AM3 NetCDF conversion failing- temporary fix

Atmosphere AM3
1

The AM3 netcdf_conversion job is currently failing, with the error:

Traceback (most recent call last):
  File "/home/564/lw5085/cylc-run/access-am3-test-branch/app/netcdf_conversion/bin/run_um2netcdf.py", line 17, in <module>
    import um2netcdf4
  File "/g/data/access/projects/access/apps/pythonlib/um2netcdf4/2.1/um2netcdf4.py", line 6, in <module>
    import cf_units, cftime, mule
ModuleNotFoundError: No module named 'mule'

This was caused by a combination of 2 things:

  1. A recent change to the /g/data/access/modules/pythonlib/um2netcdf/xp65 modulefile, which caused it to load the default version of conda/analysis rather than a specific version. The purpose of this modulefile was to load conda/analysis, and add /g/data/access/apps/pythonlib/um2netcdf4/2.1 to the $PYTHONPATH.
  2. Our scheduled update of the default version of conda/analysis, which changed from 26.01 to 26.03.

This version change removed mule from the Conda environment, as part of the phasing out of the old UM packages due to the untenable restrictions they were putting on the version of other packages in the environment e.g. numpy (note that ants has also been gone since 25.09).

I have made a branch of access-am3-configs called dev-n96e-temporary-postproc-fix, which contains a temporary workaround. The workaround is to create a duplicate of the original modulefile in a writable location, which specifies conda/analysis3-25.08 precisely. The location I chose for now was /g/data/gb02/lw5085/modulefiles, as a majority of the regular AM3 users are part of gb02.

This means the only required changes to the configuration are:

  • Modify the [[netcdf_conversion]] pre-script to be:
    """
module use /g/data/gb02/lw5085/modulefiles
module load um2netcdf4/xp65
"""

The previous solution I offered was much more complicated than it needs to be. I’ll leave it there for posterity.

Previous solution attempt:

  • Modify the pre-script for the [[netcdf_conversion]] task to load conda/analysis3-25.08 specifically
  • Add the python files in /g/data/access/apps/pythonlib/um2netcdf4/2.1 to the /g/data/access/modules/pythonlib/um2netcdf/xp65 modulefile in a writable location, which specifies the conda/analysis3-25.08 precisely.

The previous method was overcimplicating the problemsconfiguration’s app/netcdf_conversion/file, so that they’re available in the task’s workspace.

  • Add unset _CONDA_PYTHON_SYSCONFIGDATA_NAME to the task [[[environment]]]. The precise reason for this is beyond me, but it does cause the python job to fail.

So until we produce a permanent fix, use this branch (or apply these changes to your working branch).

@qinggangg This is effectively the second solution I suggested in the other thread- I’m fairly sure your problem was caused by the first issue I mentioned.

1 Like
2

This has been fixed by reverting the changes to the /g/data/access/modules/pythonlib/um2netcdf/xp65 to load conda/analysis3-25.08 specifically. The original configuration should now work.

1 Like