ACCESS-Regional Nesting Suite software inventory

paulleopardi · 1 July 2024 02:20

This is a software inventory for the ACCESS Regional Nesting Suite.

Scope

This inventory covers only the software used by the nci-gadi site. The LFRic LAM and all LFRic-related configuration and script files are out of scope for the initial release. For example, the scripts in app/generate_weights are LFRic-related, and therefore out of scope.

The ACCESS Regional Ancillary Suite u-dg767 is identical to the upstream Regional Auxiliary Suite as at u-bu503/trunk@289144 except for the Rose configuration. It is therefore also out of scope for the current audit.

Regional Nesting Suite `u-dg768`

Module directories used

Relevant module use statements appear in the following files:

/g/data/hh5/public/modules:
- app/nci_*/rose-app.conf
/g/data/access/projects/access/modules:
- site/nci-gadi/suite-adds.rc

Modules used

Modules are listed by their origin.

/apps/Modules/modulefiles
- intel-compiler/2021.5.0
- intel-mkl/2019.3.199
- netcdf/4.7.1
- openmpi/4.0.1, openmpi/4.0.2
- python2/2.7.16
/g/data/hh5/public/modules
- conda (loads: conda/analysis3-24.01)
- conda/analysis3-23.01
/g/data/access/projects/access/modules
- ants/0.11.0x (also loads: intel-compiler/2019.3.199)
- drhook/1.1_ompi.4.0.2 (prerequisite: openmpi/4.0.2)
- eccodes/2.8.0
- fcm/2019.09.0
- gcom/7.7_ompi.4.0.2 (prerequisite: openmpi/4.0.2)
- pythonlib/mule (loads: pythonlib/mule/2022.05.1, also loads: python2/2.7.17)

Explicit `PATH` used

In site/nci-gadi/suite-adds.rc:

/g/data/du7/chs548/rmed/utils/moose-client-wrapper

Python scripting

Python scripts are listed by their origin within the u-dg768 file hierarchy.

Jinja2Filters
- date_list.py
- get_matches.py
- get_parent.py
- get_variable.py
- vert_levs.py
lib
- python/vertical_levels.py
meta
- lib/python/macros/nesting_suite.py
app (Python 2)
- rose_ana_atm/bin/mule_cumf.py
- rose_ana_atm/bin/um_stdout.py
- simim/file/simim_ff_manip.py
app (Python 3)
- nci_era5grib/bin/nci_era5grib.py
- nci_era5grib_parallel/bin/cdo_era5grib.py
- nci_era5grib_parallel/bin/nci_era5grib_parallel.py
- nci_hres_eccb/bin/hres_eccb.py
- nci_hres_eccb/bin/replace_landsurface_with_BARRA2R_IC.py
- nci_hres_eccb/bin/replace_landsurface_with_ERA5land_IC.py
- nci_hres_ic/bin/hres_ic.py
- nci_hres_ic/bin/replace_landsurface_with_BARRA2R_IC.py
- nci_hres_ic/bin/replace_landsurface_with_ERA5land_IC.py

Python packages imported

Python 2 packages imported

Python 2 packages are listed by their origin.

Built-in
- imp
- sys
/apps/python2/2.7.16/lib/python2.7
- argparse
- copy
- os
- re
- subprocess
/apps/python2/2.7.16/lib/python2.7/lib-dynload
- datetime
- math
- time
/apps/python2/2.7.16/lib/python2.7/site-packages
- pytz
/g/data/hr22/apps/cylc7/cylc_7.9.9/lib
- jinja2.filters
/g/data/hr22/apps/cylc7/rose_2019.01.8/lib/python/rose
- rose.macro
/g/data/access/projects/access/apps/pythonlib/mule/2022.05.1
- mule
- um_utils.cumf
- um_utils.trim
lib
- vertical_levels

Python 3 packages imported

Python 3 packages are listed by their origin.

Built-in
- sys
- time
/g/data/hh5/public/apps/miniconda3/envs/analysis3-23.10/lib/python3.10
- argparse
- datetime
- glob
- multiprocessing
- os
- pathlib
- random
- shutil
/g/data/hh5/public/apps/miniconda3/envs/analysis3-23.10/lib/python3.10/site-packages
- era5grib
- iris
- mule
- numpy
- pandas
- xarray
app/nci_era5grib_parallel/bin
- cdo_era5grib
app/nci_hres_ic/bin
- replace_landsurface_with_BARRA2R_IC
- replace_landsurface_with_ERA5land_IC
app/nci_hres_eccb/bin
- replace_landsurface_with_BARRA2R_IC (duplicate)
- replace_landsurface_with_ERA5land_IC (duplicate)

Programs called

Built-in `rose` commands

fcm_make, rose_arch, rose_bunch, rose_prune

The usual Gnu/POSIX utilities

cut, find, gunzip, head, ln, mkdir, mv, rev, sort, tar, xarg, etc.

Programs defined but not called

Python scripts defined as programs

bin/setup_metadata

Programs called by Cylc task Bash scripts

Python scripts defined as programs

bin/get_cycle_offset
bin/install_lbcdata
bin/install_startdata

Programs called via `rose-app.conf` `[command]`

By Rose app config file.

./app/check_hall/rose-app.conf: select_hall
./app/glm_um/rose-app.conf: um-atmos
./app/nci_era5grib_parallel/rose-app.conf: nci_era5grib_parallel.py
./app/nci_era5grib/rose-app.conf: nci_era5grib.py
./app/nci_hres_eccb/rose-app.conf: hres_eccb.py
./app/nci_hres_ic/rose-app.conf: hres_ic.py
./app/niwa_netcdf/rose-app.conf: wrapper.sh
./app/poller/rose-app.conf: true
./app/rose_ana_atm/rose-app.conf: mule_cumf.py
./app/rose_ana_atm/rose-app.conf: um_stdout.py
./app/surf_ostia2nwp/rose-app.conf: SurfScr_OSTIA2NWP
./app/surf_smc/rose-app.conf: SurfScr_SoilMoistureAnalysis
./app/um_createbc/rose-app.conf: um-createbc.exe
./app/um/rose-app.conf: um-atmos

Programs called by Python 2 scripts

Programs called via `os.system`

Called from bin/install_lbcdata
- /bin/tar
- /g/data/du7/chs548/rmed/utils/moose-client-wrapper/moo
  - (restricted read access.)

Programs called via `subprocess`

Called from bin/install_lbcdata and bin/install_startdata
- /opt/nci/bin/mdss
- /g/data/du7/chs548/rmed/utils/moose-client-wrapper/moo
  - (restricted read access.)
Called from bin/install_startdata
- /bin/gunzip

Programs called by Python 3 scripts

Programs called via `os.system`

Called from app/nci_era5grib_parallel/bin/cdo_era5grib.py
- /g/data/hh5/public/apps/miniconda3/envs/analysis3-23.10/bin/cdo
  - Uses hh5 conda/analysis3-23.01 module loaded by app/nci_era5grib_parallel/rose-app.conf

Programs called via `subprocess`

None. (The subprocess.call instances in app/generate_weights are LFRic-related and are out of scope.)

Programs and libraries built

`fcm:` source locations configured

By Rose and FCM configuration files.

app/fcm_make_createbc/rose-app.conf
- config_root_path=fcm:um.xm_tr
app/fcm_make_um/rose-app.conf is the suite Rose configuration file used to build $ROSE_SUITE_DIR/share/fcm_make_um/build-atmos/bin/um-atmos.exe and $ROSE_SUITE_DIR/share/fcm_make_um/build-recon/bin/um-recon.exe. It contains:
- COUPLER=none
  DR_HOOK=false
  eccodes=true
  gwd_ussp_precision=double
  land_surface_model=jules
  ls_precipitation_precision=double
  mpp_version=1C
  netcdf=true
  openmp=true
  platagnostic=false
  portio_version=2A
  stash_version=1A
  thread_utils=false
  timer_version=3A
  config_revision=
  config_root_path=fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile
  config_type=atmos
  casim_rev=um13.0
  casim_sources=fcm:casim.xm/branches/dev/paulfield/um13.0_um13_casim_vn1.0_aggcfl@9955
  jules_rev=um13.0
  jules_sources=
  shumlib_sources=
  shumlib_rev=um13.0
  socrates_sources=
  socrates_rev=um13.0
  ukca_rev=um13.0
  ukca_sources=
  um_rev=vn13.0
  um_sources
  = fcm:um.xm/branches/dev/paulfield/vn13.0_casim_prec_frac@112741
  = fcm:um.xm/branches/dev/adrianlock/vn13.0_CoMA_cloud_inhom_regime@112431
  = fcm:um.xm/branches/dev/adrianlock/vn13.0_comorph_radius_q_dep@111985
app/fcm_make_um/file/fcm-make.cfg is the suite FCM configuration file used to build um-atmos.exe and um-recon.exe. It contains:
- include = $config_root_path/fcm-make/$platform_config_dir/um-$config_type-$optimisation_level.cfg$config_revision
- resolving to: fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/nci-x86-ifort/um-atmos-safe.cfg which contains:
  - include = um-atmos-common.cfg
  - resolving to: fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/um-atmos-common.cfg. This is the main FCM config file used to build um-atmos.exe and um-recon.exe. It contains::
    - $casim_project_location{?} = fcm:casim.xm
      $casim_base{?} = trunk
      $casim_rev{?} = head
      $jules_project_location{?} = fcm:jules.xm
      $jules_base{?} = trunk
      $jules_rev{?} = head
      $shumlib_project_location{?} = fcm:shumlib.xm
      $shumlib_base{?} = trunk
      $shumlib_rev{?} = head
      $socrates_project_location{?} = fcm:socrates.xm
      $socrates_base{?} = trunk
      $socrates_rev{?} = head
      $ukca_project_location{?} = fcm:ukca.xm
      $ukca_base{?} = trunk
      $ukca_rev{?} = head
      $um_project_location{?} = fcm:um.xm
      $um_base{?} = trunk
      $um_rev{?} = head
      extract.location{primary}[casim] = $casim_project_location
      extract.location[casim] = $casim_base@$casim_rev
      extract.location{primary}[jules] = $jules_project_location
      extract.location[jules] = $jules_base@$jules_rev
      extract.location{primary}[shumlib] = $shumlib_project_location
      extract.location[shumlib] = $shumlib_base@$shumlib_rev
      extract.location{primary}[socrates] = $socrates_project_location
      extract.location[socrates] = $socrates_base@$socrates_rev
      extract.location{primary}[ukca] = $ukca_project_location
      extract.location[ukca] = $ukca_base@$ukca_rev
      extract.location{primary}[um] = $um_project_location
      extract.location[um] = $um_base@$um_rev\
    - resolving to:
    - extract.location{primary}[casim] = fcm:casim.xm
      extract.location[casim] = trunk@um13.0
      extract.location{primary}[jules] = fcm:jules.xm
      extract.location[jules] = trunk@um13.0
      extract.location{primary}[shumlib] = fcm:shumlib.xm
      extract.location[shumlib] = trunk@um13.0
      extract.location{primary}[socrates] = fcm:socrates.xm
      extract.location[socrates] = trunk@um13.0
      extract.location{primary}[ukca] = fcm:ukca.xm
      extract.location[ukca] = trunk@um13.0
      extract.location{primary}[um] = fcm:um.xm
      extract.location[um] = trunk@um13.0
    - fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/um-atmos-common.cfg also specifies that parts of each of the components casim, jules, shumlib, socrates, ukcaandum` are extracted, by specifying for each component:
      - extract.path-excl[component]
        extract.path-incl[component]
    - fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/um-atmos-common.cfg also contains:
      - $ATMOS_EXEC{?} = um-atmos.exe
        $RECON_EXEC{?} = um-recon.exe
        $COUPLER{?} = none # Coupler name
        $gwd_ussp_precision{?} = double
        $land_surface_model{?} = jules # land surface model
        $ls_precipitation_precision{?} = double
        $mpp_version{?} = 1C # C96
        $portio_version{?} = 2A # C95
        $stash_version{?} = 1A # C84
        $timer_version{?} = 3A # C97
        $recon_mpi{?} = parallel # Parallel/serial reconfiguration
        include = ${recon_mpi}.cfg
      - resolving to fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/nci-x86-ifort/inc/parallel.cfg which contains:
        
        $ccflags_omp_on{?} = -qopenmp
        $fcflags_omp_on{?} = -qopenmp
        $ldflags_drhook_on{?} = -ldrhook -qopenmp
        $ldflags_eccodes_on{?} = -leccodes_f90 -leccodes
        $ldflags_netcdf_on{?} = -lnetcdff -lnetcdf
        $ldflags_omp_on{?} = -qopenmp
        $ldgcom_omp{?} = -qopenmp
        include = options/common.cfg
        
        resolving to fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/common.cfg which contains:
        
        $COUPLER{?} = none # Coupler name
        $DR_HOOK{?} = false # Dr Hook library
        $eccodes{?} = false # ECCodes GRIB support
        $gwd_ussp_precision{?} = double
        $land_surface_model{?} = jules # land surface model
        $ls_precipitation_precision{?} = double
        $mkl{?} = false # Intel Math Kernel Library
        $mpp_version{?} = 1C # C96
        $netcdf{?} = false # NetCDF support
        $openmp{?} = false # OpenMP
        $platagnostic{?} = false # Platform agnostic builds
        $portio_version{?} = 2A # C95
        $recon_mpi{?} = parallel # Parallel/serial reconfiguration
        $stash_version{?} = 1A # C84
        $thread_utils{?} = false # Thread Utils Module (from SHUMlib)
        $timer_version{?} = 3A # C97
        include = $HERE/coupler/${COUPLER}.cfg
        include = $HERE/drhook/${DR_HOOK}.cfg
        include = $HERE/eccodes/${eccodes}.cfg
        include = $HERE/ussp/${gwd_ussp_precision}.cfg
        include = $HERE/land_surface/${land_surface_model}.cfg
        include = $HERE/ls_precip/${ls_precipitation_precision}.cfg
        include = $HERE/mkl/${mkl}.cfg
        include = $HERE/mpp/${mpp_version}.cfg
        include = $HERE/netcdf/${netcdf}.cfg
        include = $HERE/openmp/${openmp}.cfg
        include = $HERE/platagnostic/${platagnostic}.cfg
        include = $HERE/portio/${portio_version}.cfg
        include = $HERE/recon_mpi/${recon_mpi}.cfg
        include = $HERE/stash/${stash_version}.cfg
        include = $HERE/thread_utils/${thread_utils}.cfg
        include = $HERE/timer/${timer_version}.cfg
        
        resolving to:
        
        include = fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/coupler/none.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/drhook/false.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/eccodes/true.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/ussp/double.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/land_surface/jules.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/ls_precip/double.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/mkl/false.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/mpp/1C.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/netcdf/true.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/openmp/true.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/platagnostic/false.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/portio/2A.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/recon_mpi/parallel.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/stash/1A.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/thread_utils/false.cfg
        include = um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/options/timer/3A.cfg
        
        Including these files results in setting these variables:
        
        $cppflags_eccodes{?} = GRIB_AVAILABLE=grib_available
        $cppflags_land_surface{?} = UM_JULES=um_jules
        $cppflags_mpp{?} = C96_1C=c96_1c
        $cppflags_netcdf{?} = NETCDF_AVAILABLE=netcdf_available
        $cppflags_portio{?} = C95_2A=c95_2a
        $cppflags_stash{?} = C84_1A=c84_1a
        $cppflags_thread_utils{?} =
        $cppflags_timer{?} = C97_3A=c97_3a
        $fcflags_eccodes{?} =
        $fcflags_netcdf{?} =
        $fcflags_omp{?} = -qopenmp
        $fppflags_platagnostic{?} =
        $ldflags_eccodes{?} = -leccodes_f90 -leccodes
        $ldflags_netcdf{?} = -lnetcdff -lnetcdf
        $ldflags_omp{?} = -qopenmp
        $nodep_options = $nodep_options eccodes netcdf
        $openmp_path = openmp
      - fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/nci-x86-ifort/inc/parallel.cfg also contains:
      - $ldflags_overrides_prefix{?} =
        $ldflags_overrides_suffix{?} =
        $ldflags{?} = $ldflags_overrides_prefix \
        \ -lgcom $ldgcom_omp $ldflags_netcdf $ldflags_eccodes \
        \ $ldflags_omp $ldflags_drhook \
        \ -mcmodel=medium -shared-intel \
        \ $ldflags_overrides_suffix
      - resulting in
      - $ldflags{?} = -lgcom -qopenmp -lnetcdff -lnetcdf \
        \ -leccodes_f90 -leccodes \
        \ -qopenmp -mcmodel=medium -shared-intel
    - fcm:um.xm-br/dev/simonwilson/vn13.0_archer2_compile/fcm-make/inc/um-atmos-common.cfg also specifies the source files, compiler flags and linker flags used for each of the executables um-atmos.exe and um-recon.exe. The header and Fortran module file locations are assumed to be specified via CPATH and FPATH. The library locations are assumed to be specified by LIBRARY_PATH and LD_LIBRARY_PATH. These are set in the environment via (e.g.) environment modules.
app/fcm_make_um/file/fcm-make.cfg also contains:
- extract.location{diff}[casim] = $casim_sources
  extract.location{diff}[jules] = $jules_sources
  extract.location{diff}[shumlib] = $shumlib_sources
  extract.location{diff}[socrates] = $socrates_sources
  extract.location{diff}[ukca] = $ukca_sources
  extract.location{diff}[um] = $um_sources
- resolving to:
- extract.location{diff}[casim] = casim.xm/branches/dev/paulfield/um13.0_um13_casim_vn1.0_aggcfl@9955
  extract.location{diff}[jules] =
  extract.location{diff}[shumlib] =
  extract.location{diff}[socrates] =
  extract.location{diff}[ukca] =
  extract.location{diff}[um] =
  um.xm/branches/dev/paulfield/vn13.0_casim_prec_frac@112741
  um.xm/branches/dev/adrianlock/vn13.0_CoMA_cloud_inhom_regime@112431
  um.xm/branches/dev/adrianlock/vn13.0_comorph_radius_q_dep@111985
- FCM uses these diff locations to modify the extracted files for each component.

Programs installed

Program directories installed by `install_cold`

Configured by app/install_cold/opt/rose-app-nci-gadi.conf.

$ROSE_SUITE_DIR/share/fcm_make_um_utils
- source=/projects/access/umdir/vn13.0/linux/utilities
  - This directory does not exist, it probably should be
    - source=/projects/access/umdir/vn13.0/gadi/utilities
$ROSE_SUITE_DIR/share/fcm_make_surf
- source=/projects/access/NWP_SHARE/surf-2020.01.0/1.0.0/intel1903-ompi401
  - Built from
    - /scratch/dp9/sjr548/cylc-run/surf-2020.01.0/work/1/fcm_make_surf_x86_64_ifort_opt_mirror

atteggiani · 2 July 2024 06:23

Amazing job @paulleopardi!
Is this list comprehensive of both Regiona Ancillary Suite u-dg767 and Regional Nesting Suite u-dg768?

There a two Jinja2Filters python scripts in u-dg767 that are not included in your list:

bot_left_corner.py
rot_pole.py

paulleopardi · 3 July 2024 03:24

I still haven’t finished the list. I will do my best today, but am also working on access-esm1.5 Spack.

paulleopardi · 4 July 2024 00:52

To directly answer your question, I haven’t started on u-dg767 yet.

paulleopardi · 5 July 2024 02:28

Also, u-dg767 is identical to the upstream Regional Auxiliary Suite as at u-bu503/trunk@289144 except for the Rose configuration. I will update the scope to exclude it from the current audit.

paulleopardi · 15 July 2024 06:12

Relevant FCM configuration trees in um.xm-br/dev/simonwilson/vn13.0_archer2_compile

$ tree fcm-make/nci-x86-ifort
fcm-make/nci-x86-ifort
├── inc
│   ├── libs.cfg
│   ├── parallel.cfg
│   └── serial.cfg
├── um-atmos-debug.cfg
├── um-atmos-high.cfg
├── um-atmos-rigorous.cfg
├── um-atmos-safe.cfg
├── um-createbc-debug.cfg
├── um-createbc-high.cfg
├── um-createbc-safe.cfg
├── um-libs-high.cfg
├── um-scm-debug.cfg
├── um-scm-high.cfg
├── um-scm-rigorous.cfg
├── um-scm-safe.cfg
├── um-utils-serial-debug.cfg
├── um-utils-serial-high.cfg
├── um-utils-serial-rigorous.cfg
└── um-utils-serial-safe.cfg

1 directory, 19 files

$ tree fcm-make/inc
fcm-make/inc
├── options
│   ├── common.cfg
│   ├── coupler
│   │   ├── none.cfg
│   │   └── oasis3_mct.cfg
│   ├── drhook
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── eccodes
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── land_surface
│   │   └── jules.cfg
│   ├── ls_precip
│   │   ├── double.cfg
│   │   └── single.cfg
│   ├── mkl
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── mpp
│   │   └── 1C.cfg
│   ├── netcdf
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── openmp
│   │   ├── c_only.cfg
│   │   ├── false.cfg
│   │   ├── fortran_only.cfg
│   │   └── true.cfg
│   ├── platagnostic
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── portio
│   │   ├── 2A.cfg
│   │   └── 2B.cfg
│   ├── recon_mpi
│   │   ├── parallel.cfg
│   │   └── serial.cfg
│   ├── stash
│   │   └── 1A.cfg
│   ├── thread_utils
│   │   ├── false.cfg
│   │   └── true.cfg
│   ├── timer
│   │   ├── 1A.cfg
│   │   ├── 3A.cfg
│   │   └── 4A.cfg
│   └── ussp
│       ├── double.cfg
│       └── single.cfg
├── um-atmos-common.cfg
├── um-createbc-common.cfg
├── um-libs-common.cfg
├── um-scm-common.cfg
├── um-utils-mpp-common.cfg
└── um-utils-serial-common.cfg

17 directories, 39 files

paulleopardi · 16 July 2024 03:24

Shell scripts in u-dg768 (including LFRic):

./app/check_hall/bin/select_hall
./app/configurate/bin/configurate.sh
./app/fix_nc_hdr/bin/fix_nc_time
./app/lfric_atm_lam/bin/launch-exe
./app/niwa_netcdf/bin/wrapper.sh
./app/poller/bin/check_task_status
./app/psyclone/bin/psyclone.sh
./app/simim/bin/charts_simim.sh
./app/um_createbc/bin/convert_to_ff
./bin/mo_test_battery
./bin/umpp

Shell scripts in u-dg768 but not in u-by395@292921: none.

Python scripts in u-dg768:

./app/generate_weights/bin/create_um_grid_nml.py
./app/generate_weights/bin/create_um_grid.py
./app/generate_weights/bin/generate_weights.py
./app/generate_weights/bin/weights_gen_wrapper.py
./app/nci_era5grib/bin/nci_era5grib.py
./app/nci_era5grib_parallel/bin/cdo_era5grib.py
./app/nci_era5grib_parallel/bin/nci_era5grib_parallel.py
./app/nci_hres_eccb/bin/hres_eccb.py
./app/nci_hres_eccb/bin/replace_landsurface_with_BARRA2R_IC.py
./app/nci_hres_eccb/bin/replace_landsurface_with_ERA5land_IC.py
./app/nci_hres_ic/bin/hres_ic.py
./app/nci_hres_ic/bin/replace_landsurface_with_BARRA2R_IC.py
./app/nci_hres_ic/bin/replace_landsurface_with_ERA5land_IC.py
./app/rose_ana_atm/bin/mule_cumf.py
./app/rose_ana_atm/bin/um_stdout.py
./app/simim/file/simim_ff_manip.py
./bin/get_cycle_offset
./bin/install_lbcdata
./bin/install_startdata
./bin/linkprev_lbcdata
./bin/read_horiz_grid
./bin/setup_metadata
./Jinja2Filters/date_list.py
./Jinja2Filters/get_matches.py
./Jinja2Filters/get_parent.py
./Jinja2Filters/get_variable.py
./Jinja2Filters/vert_levs.py
./lib/python/vertical_levels.py
./meta/lib/python/macros/nesting_suite.py

Python scripts in u-dg768 but not in u-by395@292921:

./app/nci_era5grib_parallel/bin/cdo_era5grib.py
./app/nci_era5grib_parallel/bin/nci_era5grib_parallel.py
./app/nci_hres_eccb/bin/hres_eccb.py
./app/nci_hres_eccb/bin/replace_landsurface_with_BARRA2R_IC.py
./app/nci_hres_eccb/bin/replace_landsurface_with_ERA5land_IC.py
./app/nci_hres_ic/bin/hres_ic.py
./app/nci_hres_ic/bin/replace_landsurface_with_BARRA2R_IC.py
./app/nci_hres_ic/bin/replace_landsurface_with_ERA5land_IC.py

paulleopardi · 18 November 2024 04:39

SURF scripts and executables

The SURF executables used by the Regional Nesting Suite are compiled separately and then installed via install_cold as a symbolic link at $ROSE_SUITE_DIR/share/fcm_make_surf.

In the case of app/install_cold/opt/rose-app-nci-gadi.conf in both the u-dg768 and u-by395 regional nesting suites this is a link to
/projects/access/NWP_SHARE/surf-2020.01.0/1.0.0/intel1903-ompi401.

There are seven such directories in /projects/access/NWP_SHARE/:

$ ls -lhd /projects/access/NWP_SHARE/surf-*/*/*
drwxr-s---+ 4 saw562 access.admin 4.0K Mar  3  2023 /projects/access/NWP_SHARE/surf-2018.02.0/1.0.0/intel1903-ompi401
drwxrws---+ 3 wml548 access.admin 4.0K Feb 26  2020 /projects/access/NWP_SHARE/surf-2019.02.0/1.0.0/intel1903-ompi401
drwxrws---+ 3 wml548 access.admin 4.0K Apr  1  2020 /projects/access/NWP_SHARE/surf-2020.01.0/1.0.0/intel1903-ompi401
drwxrws---+ 5 wml548 access.admin 4.0K Mar  3  2023 /projects/access/NWP_SHARE/surf-2021.03.0/1.0.0/intel-1903-ompi401
drwxr-s---+ 3 dl9118 access.admin 4.0K Aug 31  2023 /projects/access/NWP_SHARE/surf-2021.03.0_bom_deploy/1.0.0/intel2003-ompi402
drwxr-sr-x+ 3 dl9118 access.admin 4.0K Sep 20  2023 /projects/access/NWP_SHARE/surf-2021.03.0_bom_deploy/1.0.1/intel2003-ompi402
drwxr-sr-x+ 3 dl9118 access.admin 4.0K Aug 31  2023 /projects/access/NWP_SHARE/surf-2022.12.0/1.0.0/intel2003-ompi402

See RMED ticket #249 for an example of the update process at the UK Met Office end.

SURF has its own MOSRS project.

The nci site within the rose-stem tests for the trunk refers to /g/data/dp9: https://code.metoffice.gov.uk/trac/surf/browser/main/trunk/rose-stem/site/nci/surf/suite-env.rc

For a list of SURF releases, see https://code.metoffice.gov.uk/trac/surf/wiki/ReleasedVersions

paulleopardi · 18 November 2024 05:13

um_createbc scripts and executables

In site/nci-gadi/suite-adds.rc in both u-by395@282921 and u-dg768 there occurs:

    [[CREATEBC]]
        [[[ environment ]]]
            PATH = ${UMDIR}/vn{{UMVER}}/gadi/utilities:$PATH
        ...

and suite-runtime/lams.rc contains:

    [[{{model(regn, resln, mod)}}_CREATEBC]]
        {% if mod["ic_lbc_src"] == DRV_MOD["name"] and DRV_MOD["name"] != "ec" %}
          inherit = CREATEBC, {{DRV_MOD["name"] | upper}}_OPER_HOOK_TASKS
        {% else %}
          inherit = CREATEBC
        {% endif %}
        ...
    [[{{model(regn, resln, mod)}}_LBCS]]
        inherit = {{model(regn, resln, mod)}}, {{model(regn, resln, mod)}}_CREATEBC, LBCS
        ...
    [[{{model(regn, resln, mod)}}_um_createbc_{{crp}}]]
        inherit = {{model(regn, resln, mod)}}_LBCS
        ...

This, rather than install_cold, sets the path that is used by the *_um_createbc_* tasks to find um-createbc.exe.

Examining the runtime of the Lismore_d1000_GAL9_um_createbc_000 job with a run of u-dg768 reveals:

$ grep "PATH=.*UMDIR\|UMDIR=" log/job/20220226T0000Z//Lismore_d1000_GAL9_um_createbc_000/01/job
    UMDIR="/g/data/access/projects/access/umdir"
    PATH="${UMDIR}/vn13.0/gadi/utilities:$PATH"

The contents of this directory is:

$ ls /g/data/access/projects/access/umdir/vn13.0/gadi/utilities
/g/data/access/projects/access/umdir/vn13.0/gadi/utilities:
um-createbc.exe  um-pptoanc  um-pptoanc.exe  um_script_functions

Corresponding directories exist for UM versions 13.0 to 13.7:

$ ls -lh /g/data/access/projects/access/umdir/vn13.?/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Jul 26  2022 /g/data/access/projects/access/umdir/vn13.0/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Dec  2  2022 /g/data/access/projects/access/umdir/vn13.1/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Apr  1  2023 /g/data/access/projects/access/umdir/vn13.2/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Jun 30  2023 /g/data/access/projects/access/umdir/vn13.3/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Nov  6  2023 /g/data/access/projects/access/umdir/vn13.4/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Apr  2  2024 /g/data/access/projects/access/umdir/vn13.5/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Jul 10 12:23 /g/data/access/projects/access/umdir/vn13.6/gadi/utilities/um-createbc.exe
-rwxr-x---+ 1 mrd599 access.admin 5.8M Nov  9 10:13 /g/data/access/projects/access/umdir/vn13.7/gadi/utilities/um-createbc.exe

paulleopardi · 18 November 2024 06:52

um_createbc software build and installation

The um_createbc software is currently built and installed on Gadi via UM rose stem. For example, see /scratch/p66/mrd599/cylc-run/vn13.7_nci_rose_stem for the runtime of the rose stem test that was used to build and install the contents of /g/data/access/projects/access/umdir/vn13.7/gadi/utilities.

In particular, the file /scratch/p66/mrd599/cylc-run/vn13.7_nci_rose_stem/suite.rc.processed contains

    [[install-gadi_intel_createbc_high_omp]]
        inherit = GADI_INSTALL_UMDIR
        [[[environment]]]
            ROSE_TASK_APP = install_utils
            INSTALL_TARGET = gadi
            SOURCES = "fcm_make_gadi_intel_createbc_high_omp"
         ...
    [[fcm_make_gadi_intel_createbc_high_omp]]
        inherit = GADI_BUILD_INTEL_SERIAL, BUILD
        [[[environment]]]
            ROSE_TASK_APP=fcm_make_createbc
            OPTIMISATION=high
            OPENMP=true
            THREADUTILS=false
            ROSE_APP_OPT_CONF_KEYS=
            PREBUILD=
        ....

Topic		Replies	Views
Joint development of the regional nesting suite Working Group regional , suite , um , model , atmosphere	3	96	15 October 2024
ACCESS-NRI Intake catalog: a way to find, load and share data on Gadi ACCESS-NRI Releases nci , python , release , catalogue , nri-updates , intake	4	293	7 May 2024
UM RNS scaling on NCI Regional Nesting Suite help , inscope , solved , ncmas , scaling	5	103	4 November 2024
About the modules category modules	0	181	21 October 2022
A centralised location for ACCESS forcing and ancil files Infrastructure	9	361	12 September 2024

ACCESS-Regional Nesting Suite software inventory

Scope

Regional Nesting Suite u-dg768

Module directories used

Modules used

Explicit PATH used

Python scripting

Python packages imported

Python 2 packages imported

Python 3 packages imported

Programs called

Built-in rose commands

The usual Gnu/POSIX utilities

Programs defined but not called

Python scripts defined as programs

Programs called by Cylc task Bash scripts

Python scripts defined as programs

Programs called via rose-app.conf [command]

Programs called by Python 2 scripts

Programs called via os.system

Programs called via subprocess

Programs called by Python 3 scripts

Programs called via os.system

Programs called via subprocess

Programs and libraries built

fcm: source locations configured

Programs installed

Program directories installed by install_cold

SURF scripts and executables

SURF has its own MOSRS project.

um_createbc scripts and executables

um_createbc software build and installation

Related topics

Regional Nesting Suite `u-dg768`

Explicit `PATH` used

Built-in `rose` commands

Programs called via `rose-app.conf` `[command]`

Programs called via `os.system`

Programs called via `subprocess`

Programs called via `os.system`

Programs called via `subprocess`

`fcm:` source locations configured

Program directories installed by `install_cold`