# Running ACCESS-OM2 on Leonardo supercomputer **URL:** \ **Category:** Technical\ **Tags:** help, access-om2\ **Created:** [11 February 2024 19:26 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804 "2024-02-11T19:26:36Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [11 February 2024 19:26 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/1 "2024-02-11T19:26:36Z")
Dear All, First of all I’m very glad to find this forum. I’m new in the community and just started to get into the ACCESS-OM2 configuration. The task for the moment is to compile and run test experiment on Leonardo supercomputer (in Italy), I currently work at OGS in Trieste, Italy. So, the question is: is there any documentation how to run outside of Gadi? I spent half of the day looking up through repositories 🙂 But couldn’t find anything. Probably I just don’t know where to look… So I’d be very grateful for any advices. Many thanks, Natalia ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [12 February 2024 02:19 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/2 "2024-02-12T02:19:23Z")
Hi Natalia, It’s great to hear from researchers using a supercomputer other than Gadi. I have not created any documentation on compiling/running ACCESS-OM2 outside of Gadi. I will ask my colleagues if they know of any old documentation. Recently we released a version of ACCESS-OM2 that was built using Spack ([https://spack.io/](https://spack.io/)). This should allow for ACCESS-OM2 to be more easily ported to another supercomputer. The instructions are: [How to build ACCESS-OM2 on Gadi](https://forum.access-hive.org.au/t/how-to-build-access-om2-on-gadi/1545) Some modifications will need to be made to the Gadi specific files. e.g. - [spack-config/common/gadi at main · ACCESS-NRI/spack-config · GitHub](https://github.com/ACCESS-NRI/spack-config/tree/main/common/gadi) - [spack-packages/packages/nci-openmpi/package.py at main · ACCESS-NRI/spack-packages · GitHub](https://github.com/ACCESS-NRI/spack-packages/blob/main/packages/nci-openmpi/package.py) I’m happy to create instructions on how to compile on Leonardo, if you are able to test and provide feedback. Can you please sent me the output from Leonardo of: ```auto cat /etc/os-release ``` Regards, Harshula ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [12 February 2024 09:33 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/3 "2024-02-12T09:33:56Z")
Dear Harshula, Thank you for the prompt feedback! I’ll go through repos you provided. Here is an output for Leonardo: ![image](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/4/47b1407e088b1c9b0f07a26b31741bcc407ec447.jpeg) Thanks again, Natalia ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [12 February 2024 10:54 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/4 "2024-02-12T10:54:44Z")
Hi Natalia, Since Leonardo is using RHEL, please try the following instructions: ```auto git clone -c feature.manyFiles=true https://github.com/spack/spack.git --branch releases/v0.20 --single-branch --depth=1 git clone https://github.com/ACCESS-NRI/spack-packages.git --branch main git clone https://github.com/ACCESS-NRI/spack-config.git --branch main ln -s -r -v spack-config/v0.20/ci/* spack/etc/spack/ . spack-config/spack-enable.bash spack install intel-oneapi-compilers@2021.2.0 target=x86_64 spack load intel-oneapi-compilers@2021.2.0 spack compiler find spack install access-om2 ^netcdf-c@4.7.4 ^netcdf-fortran@4.5.2 ^parallelio@2.5.2 ^openmpi@4.0.2 %intel@2021.2.0 target=x86_64 spack find ``` Also, it is better to attach textual output instead of screenshots. Thanks, Harshula ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [12 February 2024 11:37 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/5 "2024-02-12T11:37:53Z")
---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [12 February 2024 12:07 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/6 "2024-02-12T12:07:04Z")
Dear Harshula, Thank you! I will try all these and will let you know here. Also I’ll use text, thanks for the hint. Natalia ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [13 February 2024 23:40 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/7 "2024-02-13T23:40:33Z")
It looks like Leonardo already supports `spack` [https://www.hpc.cineca.it/systems/software/spack/](https://www.hpc.cineca.it/systems/software/spack/) ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [20 February 2024 02:39 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/8 "2024-02-20T02:39:07Z")
Hi Natalia, Have you had an opportunity to test the ACCESS-OM2 instructions? If you need more detailed information, I’m happy to join a video chat during EU morning hours. Regards, Harshula ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [13 March 2024 09:18 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/9 "2024-03-13T09:18:34Z")
Hi Harshula and Aidan, I apologise for being silent for a while! I’m currently attending a Master in High Performance yearly course and working with ACCESS-OM2 is a part of my thesis project. The workload with classes and homework was huge until now so I had to stay with them. Thank you very much for being responsive and your readiness to help! I will keep you updated on the progress of compiling on Leonardo. Thanks, Natalia ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [13 March 2024 09:31 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/10 "2024-03-13T09:31:49Z")
At the moment spack gives the following output errors when installing the list of packages, it fails with openmpi: > > **Summary** > > > > ==\> Installing openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 > > ==\> No binary for openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 found: installing from source > > ==\> Using cached archive: /leonardo\_scratch/large/userexternal/ntilinin/spack/var/spack/cache/\_source-cache/archive/90/900bf751be72eccf06de9d186f7b1c4b5c2fa9fa66458e53b77778dffdfe4057.tar.bz2 > > ==\> Applied patch /leonardo\_scratch/large/userexternal/ntilinin/spack/var/spack/repos/builtin/packages/openmpi/fix-ucx-1.7.0-api-instability.patch > > ==\> Applied patch /leonardo\_scratch/large/userexternal/ntilinin/spack/var/spack/repos/builtin/packages/openmpi/opal\_assembly\_arch.patch > > ==\> openmpi: Executing phase: ‘autoreconf’ > > ==\> openmpi: Executing phase: ‘configure’ > > ==\> openmpi: Executing phase: ‘build’ > > ==\> Error: ProcessError: Command exited with status 2: > > ‘make’ ‘-j16’ ‘V=1’ > > > > 29 errors found in build log: > > 13831 /bin/sh …/…/…/…/libtool --tag=CC --mode=compile /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H > > -I. -I…/…/…/…/opal/include -I…/…/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/ > > autogen -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/inclu > > de -I/leonardo\_scratch/large/userexternal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include > > -I/leonardo\_scratch/large/userexternal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include > > -I/leonardo\_scratch/large/userexternal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include > > -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre\_file\_open.lo -MD -MP -MF > > $depbase.Tpo -c -o fs\_lustre\_file\_open.lo fs\_lustre\_file\_open.c && > > 13832 mv -f $depbase.Tpo $depbase.Plo > > 13833 libtool: compile: /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H -I. -I…/…/…/…/opal/include -I…/ > > …/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I…/…/…/…/opal/mca/hwloc > > /hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/include -I/leonardo\_scratch/large/userexter > > nal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include -I/leonardo\_scratch/large/userexternal/ > > ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include -I/leonardo\_scratch/large/userexternal/ntil > > inin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include -O3 -DNDEBUG -finline-functions -fno-strict > > -aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre.lo -MD -MP -MF .deps/fs\_lustre.Tpo -c fs\_lustre.c -fPIC -DPIC -o .l > > ibs/fs\_lustre.o > > 13834 libtool: compile: /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H -I. -I…/…/…/…/opal/include -I…/ > > …/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I…/…/…/…/opal/mca/hwloc > > /hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/include -I/leonardo\_scratch/large/userexter > > nal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include -I/leonardo\_scratch/large/userexternal/ > > ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include -I/leonardo\_scratch/large/userexternal/ntil > > inin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include -O3 -DNDEBUG -finline-functions -fno-strict > > -aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre\_file\_open.lo -MD -MP -MF .deps/fs\_lustre\_file\_open.Tpo -c fs\_lustre\_ > > file\_open.c -fPIC -DPIC -o .libs/fs\_lustre\_file\_open.o > > 13835 libtool: compile: /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H -I. -I…/…/…/…/opal/include -I…/ > > …/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I…/…/…/…/opal/mca/hwloc > > /hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/include -I/leonardo\_scratch/large/userexter > > nal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include -I/leonardo\_scratch/large/userexternal/ > > ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include -I/leonardo\_scratch/large/userexternal/ntil > > inin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include -O3 -DNDEBUG -finline-functions -fno-strict > > -aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre\_component.lo -MD -MP -MF .deps/fs\_lustre\_component.Tpo -c fs\_lustre\_ > > component.c -fPIC -DPIC -o .libs/fs\_lustre\_component.o > > 13836 In file included from fs\_lustre.c(42): > > > > > > 13837 /usr/include/sys/mount.h(35): error: expected an identifier > > > > 13838 MS\_RDONLY = 1, /\* Mount read-only. _/ > > > > 13839 ^ > > > > 13840 > > > > 13841 In file included from fs\_lustre.c(42): > > > > 13842 /usr/include/sys/mount.h(37): error: expected an identifier > > > > 13843 MS\_NOSUID = 2, /_ Ignore suid and sgid bits. _/ > > > > 13844 ^ > > > > 13845 > > > > 13846 In file included from fs\_lustre.c(42): > > > > 13847 /usr/include/sys/mount.h(39): error: expected an identifier > > > > 13848 MS\_NODEV = 4, /_ Disallow access to device special files. _/ > > > > 13849 ^ > > > > 13850 > > > > 13851 In file included from fs\_lustre.c(42): > > > > 13852 /usr/include/sys/mount.h(41): error: expected an identifier > > > > 13853 MS\_NOEXEC = 8, /_ Disallow program execution. _/ > > > > 13854 ^ > > > > 13855 > > > > 13856 In file included from fs\_lustre.c(42): > > > > 13857 /usr/include/sys/mount.h(43): error: expected an identifier > > > > 13858 MS\_SYNCHRONOUS = 16, /_ Writes are synced at once. _/ > > > > 13859 ^ > > > > 13860 > > > > 13861 In file included from fs\_lustre.c(42): > > > > 13862 /usr/include/sys/mount.h(45): error: expected an identifier > > > > 13863 MS\_REMOUNT = 32, /_ Alter flags of a mounted FS. _/ > > > > 13864 ^ > > > > 13865 > > > > 13866 In file included from fs\_lustre.c(42): > > > > 13867 /usr/include/sys/mount.h(47): error: expected an identifier > > > > 13868 MS\_MANDLOCK = 64, /_ Allow mandatory locks on an FS. _/ > > > > 13869 ^ > > > > 13870 > > > > 13871 In file included from fs\_lustre.c(42): > > > > 13872 /usr/include/sys/mount.h(49): error: expected an identifier > > > > 13873 MS\_DIRSYNC = 128, /_ Directory modifications are synchronous. _/ > > > > 13874 ^ > > > > 13875 > > > > 13876 In file included from fs\_lustre.c(42): > > > > 13877 /usr/include/sys/mount.h(51): error: expected an identifier > > > > 13878 MS\_NOATIME = 1024, /_ Do not update access times. _/ > > > > 13879 ^ > > > > 13880 > > > > 13881 In file included from fs\_lustre.c(42): > > > > 13882 /usr/include/sys/mount.h(53): error: expected an identifier > > > > 13883 MS\_NODIRATIME = 2048, /_ Do not update directory access times. _/ > > > > 13884 ^ > > > > 13885 > > > > 13886 In file included from fs\_lustre.c(42): > > > > 13887 /usr/include/sys/mount.h(55): error: expected an identifier > > > > 13888 MS\_BIND = 4096, /_ Bind directory at different place. _/ > > > > 13889 ^ > > > > 13890 > > > > 13891 In file included from fs\_lustre.c(42): > > > > 13892 /usr/include/sys/mount.h(57): error: expected an identifier > > > > 13893 MS\_MOVE = 8192, > > > > 13894 ^ > > > > 13895 > > > > 13896 In file included from fs\_lustre.c(42): > > > > 13897 /usr/include/sys/mount.h(59): error: expected an identifier > > > > 13898 MS\_REC = 16384, > > > > 13899 ^ > > > > 13900 > > > > 13901 In file included from fs\_lustre.c(42): > > > > 13902 /usr/include/sys/mount.h(61): error: expected an identifier > > > > 13903 MS\_SILENT = 32768, > > > > 13904 ^ > > > > 13905 > > > > 13906 In file included from fs\_lustre.c(42): > > > > 13907 /usr/include/sys/mount.h(63): error: expected an identifier > > > > 13908 MS\_POSIXACL = 1 \<\< 16, /_ VFS does not apply the umask. _/ > > > > 13909 ^ > > > > 13910 > > > > 13911 In file included from fs\_lustre.c(42): > > > > 13912 /usr/include/sys/mount.h(65): error: expected an identifier > > > > 13913 MS\_UNBINDABLE = 1 \<\< 17, /_ Change to unbindable. _/ > > > > 13914 ^ > > > > 13915 > > > > 13916 In file included from fs\_lustre.c(42): > > > > 13917 /usr/include/sys/mount.h(67): error: expected an identifier > > > > 13918 MS\_PRIVATE = 1 \<\< 18, /_ Change to private. _/ > > > > 13919 ^ > > > > 13920 > > > > 13921 In file included from fs\_lustre.c(42): > > > > 13922 /usr/include/sys/mount.h(69): error: expected an identifier > > > > 13923 MS\_SLAVE = 1 \<\< 19, /_ Change to slave. _/ > > > > 13924 ^ > > > > 13925 > > > > 13926 In file included from fs\_lustre.c(42): > > > > 13927 /usr/include/sys/mount.h(71): error: expected an identifier > > > > 13928 MS\_SHARED = 1 \<\< 20, /_ Change to shared. _/ > > > > 13929 ^ > > > > 13930 > > > > 13931 In file included from fs\_lustre.c(42): > > > > 13932 /usr/include/sys/mount.h(73): error: expected an identifier > > > > 13933 MS\_RELATIME = 1 \<\< 21, /_ Update atime relative to mtime/ctime. _/ > > > > 13934 ^ > > > > 13935 > > > > 13936 In file included from fs\_lustre.c(42): > > > > 13937 /usr/include/sys/mount.h(75): error: expected an identifier > > > > 13938 MS\_KERNMOUNT = 1 \<\< 22, /_ This is a kern\_mount call. _/ > > > > 13939 ^ > > > > 13940 > > > > 13941 In file included from fs\_lustre.c(42): > > > > 13942 /usr/include/sys/mount.h(77): error: expected an identifier > > > > 13943 MS\_I\_VERSION = 1 \<\< 23, /_ Update inode I\_version field. _/ > > > > 13944 ^ > > > > 13945 > > > > 13946 In file included from fs\_lustre.c(42): > > > > 13947 /usr/include/sys/mount.h(79): error: expected an identifier > > > > 13948 MS\_STRICTATIME = 1 \<\< 24, /_ Always perform atime updates. _/ > > > > 13949 ^ > > > > 13950 > > > > 13951 In file included from fs\_lustre.c(42): > > > > 13952 /usr/include/sys/mount.h(81): error: expected an identifier > > > > 13953 MS\_LAZYTIME = 1 \<\< 25, /_ Update the on-disk [acm]times lazily. \*/ > > > > 13954 ^ > > > > 13955 > > > > 13956 In file included from fs\_lustre.c(42): > > > > 13957 /usr/include/sys/mount.h(83): error: expected an identifier > > > > 13958 MS\_ACTIVE = 1 \<\< 30, > > > > 13959 ^ > > > > 13960 > > > > 13961 In file included from fs\_lustre.c(42): > > > > 13962 /usr/include/sys/mount.h(85): error: expected an identifier > > > > 13963 MS\_NOUSER = 1 \<\< 31 > > > > 13964 ^ > > > > 13965 > > > > 13966 fs\_lustre.c(95): warning #2330: argument of type “const char \*” is incompatible with parameter of type “char \*” (dropping qualifiers) > > > > 13967 fh-\>f\_fstype = mca\_fs\_base\_get\_fstype ( fh-\>f\_filename ); > > > > 13968 ^ > > > > 13969 > > > > 13970 compilation aborted for fs\_lustre.c (code 2) > > > > 13971 make[2]: \*\*\* [Makefile:1866: fs\_lustre.lo] Error 1 > > > > 13972 make[2]: \*\*\* Waiting for unfinished jobs… > > > > 13973 libtool: compile: /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H -I. -I…/…/…/…/opal/include -I…/ > > > > …/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I…/…/…/…/opal/mca/hwloc > > > > /hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/include -I/leonardo\_scratch/large/userexter > > > > nal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include -I/leonardo\_scratch/large/userexternal/ > > > > ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include -I/leonardo\_scratch/large/userexternal/ntil > > > > inin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include -O3 -DNDEBUG -finline-functions -fno-strict > > > > -aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre\_component.lo -MD -MP -MF .deps/fs\_lustre\_component.Tpo -c fs\_lustre\_ > > > > component.c -o fs\_lustre\_component.o \>/dev/null 2\>&1 > > > > 13974 libtool: compile: /leonardo\_scratch/large/userexternal/ntilinin/spack/lib/spack/env/intel/icc -DHAVE\_CONFIG\_H -I. -I…/…/…/…/opal/include -I…/ > > > > …/…/…/ompi/include -I…/…/…/…/oshmem/include -I…/…/…/…/opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I…/…/…/…/opal/mca/hwloc > > > > /hwloc201/hwloc/include/hwloc/autogen -I…/…/…/…/ompi/mpiext/cuda/c -I…/…/…/… -I…/…/…/…/orte/include -I/leonardo\_scratch/large/userexter > > > > nal/ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/zlib-1.2.13-4h4sif2qagfapwdnz2pn36n5whjn2vdp/include -I/leonardo\_scratch/large/userexternal/ > > > > ntilinin/release/linux-rhel8-x86\_64/intel-2021.2.0/hwloc-2.9.1-baak3vruvdy44nmzhl65g5ison3qmd24/include -I/leonardo\_scratch/large/userexternal/ntil > > > > inin/release/linux-rhel8-x86\_64/intel-2021.2.0/libevent-2.1.12-rgrznhf3oxxhgo5thjeidwlrdm2ltwds/include -O3 -DNDEBUG -finline-functions -fno-strict > > > > -aliasing -restrict -Qoption,cpp,–extended\_float\_types -pthread -MT fs\_lustre\_file\_open.lo -MD -MP -MF .deps/fs\_lustre\_file\_open.Tpo -c fs\_lustre\_ > > > > file\_open.c -o fs\_lustre\_file\_open.o \>/dev/null 2\>&1 > > > > 13975 make[2]: Leaving directory ‘/scratch\_local/ntilinin/spack-stage/spack-stage-openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7/spack-src/ompi/mca/fs/lu > > > > stre’ > > > > 13976 make[1]: \*\*\* [Makefile:3532: all-recursive] Error 1 > > > > 13977 make[1]: Leaving directory ‘/scratch\_local/ntilinin/spack-stage/spack-stage-openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7/spack-src/ompi’ > > > > 13978 make: \*\*\* [Makefile:1879: all-recursive] Error 1 > > > > See build log for details: > > /scratch\_local/ntilinin/spack-stage/spack-stage-openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7/spack-build-out.txt > > > > ==\> Warning: Skipping build of mom5-master-zoyam3ku2fv3zjhlhtz2oomcv5hmk3oc since openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 failed > > ==\> Warning: Skipping build of access-om2-latest-prktj7fy2cfv45xtqufrg64ph73a5vly since mom5-master-zoyam3ku2fv3zjhlhtz2oomcv5hmk3oc failed > > ==\> Warning: Skipping build of cice5-master-vvvil77faon7x5ri4i6yvh7smwulzgi4 since openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 failed > > ==\> Warning: Skipping build of parallelio-2.5.2-moskoupqn3l7pe3fagiifadmqbfw6qi6 since openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 failed > > ==\> Warning: Skipping build of oasis3-mct-master-yp6eluuircccqfst5zdtyhqrgucsijkx since openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 failed > > ==\> Warning: Skipping build of libaccessom2-master-r57bpljtuxmzya3xykwsq2nnuvxgbpbh since oasis3-mct-master-yp6eluuircccqfst5zdtyhqrgucsijkx failed > > ==\> Warning: Skipping build of hdf5-1.14.1-2-3msvuphu6glhjg437dp5mjtqallwhbbe since openmpi-4.0.2-luyhnohjapwuh4ejdruohnz2bk6mwmc7 failed > > ==\> Warning: Skipping build of netcdf-c-4.7.4-tfq3y6txvenhuiktt52sar736m3t7w45 since hdf5-1.14.1-2-3msvuphu6glhjg437dp5mjtqallwhbbe failed > > ==\> Warning: Skipping build of netcdf-fortran-4.5.2-5rlirjaepmq7aa5xklngh3vuhwjicncl since netcdf-c-4.7.4-tfq3y6txvenhuiktt52sar736m3t7w45 failed > > ==\> Error: access-om2-latest-prktj7fy2cfv45xtqufrg64ph73a5vly: Package was not installed > > ==\> Error: Installation request failed. Refer to reported errors for failing package(s). ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [13 March 2024 09:34 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/11 "2024-03-13T09:34:53Z")
But am I getting right that I should basically have loaded or installed locally for my user profile openmpi, intel compilers with netcdf and parallelio? Thanks, Natalia ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [13 March 2024 12:58 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/12 "2024-03-13T12:58:31Z")
Hi Natalia, I need more context. Perhaps we can schedule a live debug session during an EU morning? ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [13 March 2024 16:49 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/13 "2024-03-13T16:49:07Z")
That would be perfect! I’ll sent my contacts in the message now. Thanks Natalia ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [14 March 2024 11:39 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/14 "2024-03-14T11:39:53Z")
Summary: Openmpi 4.0.2 failed to build via Spack. Logs will be emailed. ACCESS-OM2 was built successfully with openmpi 4.1.4 via Spack. ---
**Author:** ![harshula](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/harshula/32/256_2.png) [@harshula](https://forum.access-hive.org.au/u/harshula)\ **Post date:** [14 March 2024 11:59 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/15 "2024-03-14T11:59:29Z")
Dockerfile: [build-ci/containers/Dockerfile.base-spack at main · ACCESS-NRI/build-ci · GitHub](https://github.com/ACCESS-NRI/build-ci/blob/main/containers/Dockerfile.base-spack) ```auto docker build -f Dockerfile.base-spack -t : --target dev . ``` ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [15 March 2024 10:47 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/16 "2024-03-15T10:47:23Z")
Hi @Aidan, hi @harshula! I’m in the process of setting up demo experiment on Leo (thanks again @harshula for help with building ACCESS OM2 with spack). It seems that in any case I would need to download forcing fields locally. Would that be possible to get 1deg\_jra55\_ryf fields? It seems for me that 1 deg repeated year forcing fields should be relatively small (first hundreds MBs). Thank you! Natalia ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [15 March 2024 12:05 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/17 "2024-03-15T12:05:16Z")
I can see get\_input\_data.py script in install.sh that supposed to get the data probably, but the \*.py file itself is not in the repo ---
**Author:** ![Natalia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/natalia/32/1612_2.png) [@Natalia](https://forum.access-hive.org.au/u/Natalia)\ **Post date:** [15 March 2024 12:30 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/18 "2024-03-15T12:30:39Z")
I also have just realised that I will have to convert PBS scheduling to SLURM (we use on Leonardo) 😅 > Attention > **PBS is no more available on HPC clusters in Cineca, since Jan 2018** The only thing that I found in COSIMA discussion: > AK: For IAF had a lot of daily CICE output. Not complete set of fields. > > MW: Starting to run performance tests at GFDL and want to use payu. Has it changed much? Manifest stuff hasn’t made a big difference? Will have to get slurm working. Filesystem will be a nightmare. You moved PBS stuff into a component? AH: No, you did that. Not huge differences. Will be great to have slurm support. @Aidan, is there any progress towards slurm? Would be grateful for any updates! Many thanks, Natalia ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [18 March 2024 04:48 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/19 "2024-03-18T04:48:45Z")
> [@Natalia](#): > > It seems that in any case I would need to download forcing fields locally. Would that be possible to get 1deg\_jra55\_ryf fields? It seems for me that 1 deg repeated year forcing fields should be relatively small (first hundreds MBs) The answer is yes, I think so, but it is just a matter of time to find a suitable location and upload them. Do you have a time-frame you need them by? ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [18 March 2024 04:58 UTC](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804/20 "2024-03-18T04:58:38Z")
> [@Natalia](#): > > @Aidan, is there any progress towards slurm? > > Would be grateful for any updates! There are some people who use `payu` with slurm, specifically @ChrisC28 and @john_reilly. It is talked about in this issue > > > There has been some discussion in the context of a MOM6 project about the potent…ial for porting Payu to non-NCI hardware. > > I would be interested in getting payu up and running on the Pawsey HPC system. This system uses Cray architecture with slurm as a scheduler. > > I notice that there appears to be some support for slurm in Payu, as there is a slurm schedular class. However, I'm unsure of the process for porting payu to another system. and this is an old PR I made with some changes to get `payu` working on the Pawsey HPC (which uses `slurm`) > > > Wrap ldd in try/except as executables on pawsey seem to be statically linked. > …Also for the same reason don't assume LD\_LIBRARY\_PATH is set. > > Commented out call to load\_modules. Pawsey has a lot of default modules > that it relies on, so can't reliably monkey with that. > > Removed a couple of the bespoke flags Marshall added to the slurm > scheduler, and also explicitly pass through the PAYU environment > variables. Also set in the current environment, but that didn't seem > to make it through to the submitted job. > > Closes #323 Maybe @ChrisC28 could comment about how easy (or not) this is, and if there is a better version of `payu` to use. [Next page](https://forum.access-hive.org.au/t/running-access-om2-on-leonardo-supercomputer/1804.md?page=2)