Building UM executable using ESMF library with fcm_make fails

Technical
1

Hi all.

TLDR; Trying to build the UM using fcm_make with a NUOPCY cap fails due to a linker error.

FAIL] /bin/ld: cannot find -lesmf

I’m trying to build a version of the UM executable used in rAM3 using fcm with the NUOPCY cap installed.

I have copied the following files to the ~/cylc-run/<suite-id>/share/fcm_make_um/extract/um/src/control/top_level/

um_esmf_grids.F90
um_esmf.F90
um_cap_import_export.F90
model.F90

Using the fcm-make-on-success.cfg file from the CM3 suite access-cm3-configs as a guide, I’ve added entries for these files to fcm-make2.cfg

build-atmos.prop{fc.flags}[um/src/control/top_level/model.F90] = -i8 -r8 -mcmodel=medium              -std08                    -g                        -traceback                -assume nosource_include -O2 -fp-model precise -qopenmp     
build-atmos.prop{fc.flags}[um/src/control/top_level/um_cap_import_export.F90] = -i8 -r8 -mcmodel=medium              -std08                    -g                        -traceback                -assume nosource_include -O2 -fp-model precise -qopenmp      
build-atmos.prop{fc.flags}[um/src/control/top_level/um_esmf_grids.F90] = -i8 -r8 -mcmodel=medium              -std08                    -g                        -traceback                -assume nosource_include -O2 -fp-model precise -qopenmp      
build-atmos.prop{fc.flags}[um/src/control/top_level/um_esmf.F90] = -i8 -r8 -mcmodel=medium              -std08                    -g                        -traceback                -assume nosource_include -O2 -fp-model precise -qopenmp

and added -lemsf to the fc flags section

build-atmos.prop{fc.flags-ld} =   -lgcom -qopenmp -lnetcdff -lnetcdf -leccodes_f90 -leccodes  -qopenmp   -mcmodel=medium -shared-intel  -lesmf

Using the suggestion from this Hive Post: Model Tools

I added the following to the job script:

module use /g/data/vk83/modules
module load esmf/v8.7.0

The full error message in fcm-make2.log is

[info] shell(256  2.6) mpif90 -obin/um-atmos.exe o/um_main.o -L/home/548/pag548/cylc-run/rCM3-test-UM-NUOPC/work/20220226T0000Z/fcm_make2_um/ywlLGL3Xr2 -lum-atmos -lgcom -qopenmp -lnetcdff -lnetcdf -leccodes_f90 -leccodes -qopenmp -mcmodel=medium -shared-intel -lesmf
[FAIL] mpif90 -obin/um-atmos.exe o/um_main.o -L/home/548/pag548/cylc-run/rCM3-test-UM-NUOPC/work/20220226T0000Z/fcm_make2_um/ywlLGL3Xr2 -lum-atmos -lgcom -qopenmp -lnetcdff -lnetcdf -leccodes_f90 -leccodes -qopenmp -mcmodel=medium -shared-intel -lesmf # rc=1
[FAIL] /bin/ld: cannot find -lesmf
[FAIL] link      12.7 ! um-atmos.exe         <- um/src/control/top_level/um_main.F90

Evidently my knowledge of configuring fcm config files isn’t up to scratch.

Thanks in advance.

2

Hi Paul

For CM3 - currently the um is built as a library in a bash script (this will be inside the fcm configuration soon), and then the executable is a separate step using a CMake build which links to the UM-library.

What’s the intended use case of the build?

3

Hi Anton.

This is to help build rCM3. As you know, Ashley is pursuing a development path which takes the CM3 executable and makes it ‘regional’. This isn’t trivial, as CM3 contains lots of features that rAM3 doesn’t.

So I’m pursuing another developmental path - taking rAM3 and making it coupled.

I have taken the cpl_build task from the access-cm3-config suite you’ve linked to and placed it in my rAM3->rCM3 suite.

But first I need to get fcm_make_um and fcm_make2_um to build a rAM3 exe including the required NUOPCY files.

I’m using the same OM3 exe as Ashley is using.

I’ve diff’ed the app/fcm_make[_um]/rose-app.conf files between access-cm3-configs and rAM3 and can see no major differences besides

  • UM version numbers
  • Debug levels (I’m building with debug flags and no optimisation)
  • Location of UM source
4

Hi @Paul.Gregory,
the CM3 build uses several modules created by spack using the configuration file here. When loaded, these modules export several paths for each dependency required in the build:

  modules:
      prefix_inspections:
        bin:
         - PATH
        lib:
         - LIBRARY_PATH
         - LD_LIBRARY_PATH
        lib64:
         - LIBRARY_PATH
         - LD_LIBRARY_PATH
        include:
         - FPATH

We’re working on transitioning to a full spack build, which will hopefully be a bit more robust and avoid steps like this. For the meantime as a workaround, could you test if manually exporting LIBRARY_PATH with the path for your version of ESMF helps with the fcm_make step?