How to build ACCESS-OM2 on Gadi

Technical spack
,
1

Installation instructions for model developers on Gadi v3

[27/11/2023: Updated to use spack-config]

  1. Login to Gadi:
ssh USERNAME@gadi.nci.org.au
  1. Clone the relevant git repositories:
git clone -c feature.manyFiles=true https://github.com/spack/spack.git --branch releases/v0.20 --single-branch --depth=1
git clone https://github.com/ACCESS-NRI/spack-packages.git --branch main
git clone https://github.com/ACCESS-NRI/spack-config.git --branch main
  1. Link spack-config configurations files to the Spack instance:
ln -s -r -v spack-config/v0.20/gadi/* spack/etc/spack/
  1. Set Spack environment variables (required after each login):
. spack-config/spack-start.bash
  1. Build ACCESS-OM2 (initialisation may take a long time without any output):
spack install access-om2 ^netcdf-c@4.7.4 ^netcdf-fortran@4.5.2 ^nci-openmpi@4.0.2 %intel@19.0.5.281

Instructions for general users on Gadi v1

[30/11/2023: first version]

  1. Login to Gadi:
ssh USERNAME@gadi.nci.org.au
  1. Set Spack environment variables:
. /g/data/vk83/apps/spack/0.20/spack-config/spack-start.bash
  1. Display pre-built packages:
spack find
2

If the spack install ... command is failing with the following error:

==> Installing cmake-3.26.3-4hg4d2a6d72efysvvyfkepnvl6khes3o
==> No binary for cmake-3.26.3-4hg4d2a6d72efysvvyfkepnvl6khes3o found: installing from source
==> Fetching file:///g/data/<PROJECT>/<USERNAME>/spack-mirror-access-om2/_source-cache/archive/bb/bbd8d39217509d163cb544a40d6428ac666ddc83e22905d3e52c925781f0f659.tar.gz
==> No patches needed for cmake
==> cmake: Executing phase: 'bootstrap'
Killed

you may be using v1 of the instructions where CMake was being built by Spack. To avoid this problem, please use v2 of the instructions where the system CMake is used

3

@harshula what is meant by ... here? Where do I find what should go there?

4
  1. What’s an Ellipsis? Definition and Examples | Grammarly Blog
  2. The previous step would have auto-populated the remainder of that section.
5

I have just used the instructions to successfully build on Gadi login at /g/data/tm70/pcl851/spack

7

I couldn’t get it to work - build logs here /scratch/v45/aek156/tmp/spack-stage/

8

Hi @aekiss, I don’t have permission to access v45. Let’s go through the steps together when you’re in the office.

9

Hi @aekiss ,

Your Spack instance was unable to compile the required build tools. e.g. gmake:

==> gmake: Executing phase: 'build'
==> [2023-11-06-21:27:41.679695] '/scratch/[...]/tmp/spack-stage/spack-stage-gmake-4.4.1-h5uwuoe6h3nknjnagjbskugtwch7q5c3/spack-src/build.sh'
config.status: creating ./lib/alloca.h__
compiling lib/concat-filename.c...
In file included from /usr/include/bits/floatn.h(119),
                 from /usr/include/stdlib.h(55),
                 from /scratch/v45/aek156/tmp/spack-stage/spack-stage-gmake-4.4.1-h5uwuoe6h3nknjnagjbskugtwch7q5c3/spack-src/lib/concat-filename.h(20),
                 from /scratch/v45/aek156/tmp/spack-stage/spack-stage-gmake-4.4.1-h5uwuoe6h3nknjnagjbskugtwch7q5c3/spack-src/lib/concat-filename.c(22):
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
  typedef float _Float32;

Looks like a problem with your compiler config.

Do you remember why you made the following changes to your .spack/linux/compilers.yaml?

@@ -13,9 +13,22 @@
     environment: {}
     extra_rpaths: []
 - compiler:
+    spec: gcc@=13.2.0
+    paths:
+      cc: /g/data/hh5/public/apps/miniconda3/envs/analysis3-23.07/bin/gcc
+      cxx: null
+      f77: null
+      fc: null
+    flags: {}
+    operating_system: rocky8
+    target: x86_64
+    modules: []
+    environment: {}
+    extra_rpaths: []
+- compiler:
     spec: gcc@=8.5.0
     paths:
-      cc: /opt/nci/bin/gcc
+      cc: /bin/gcc
       cxx: /opt/nci/bin/g++
       f77: /opt/nci/bin/gfortran
       fc: /opt/nci/bin/gfortran

Perhaps move your .spack to another location and try the instructions with a new empty .spack.

10

spack find picked up gcc@13.2.0 because I had the conda/analysis3-23.07 module loaded.

It builds successfully if I do module purge before step 1.

11

Thanks @aekiss for testing and providing feedback. As a result, I’m planning on disabling local config (i.e. ~/.spack) to avoid similar issues.