Ants module availability in xp65 analysis3 kernels

Technical Infrastructure
, ,
21

I’ve not looked at ANTS for a while, it is now public on github with the environment definition and conda lock file that it wants to be using - GitHub - MetOffice/ANTS: A Python library for generating UM ancillary files, developed at the Met Office (the contrib code that actually generates the ancillaries is still private I think?)

There are two use cases to consider:

  • If we’re generating ancillaries for future use we should make use of the lock file to generate a conda environment, this way the ancillaries can be re-created exactly the same in the future.

  • If we’re developing ANTS code then it would make sense to install it into a larger environment with Jupyter etc.

22

I agree with Scott, and I would add that we are planning to release new ANTS versions in a dedicated containerised environment on Gadi (similar to Scott’s ngm one).

However, the main issues remain:

  • ANTS strict dependency pinning (which makes it very hard to install in an environment together with other packages)
  • no distributions are available (meaning we have to manage distributions, for example Conda or PyPI, ourselves)

Now that ANTS, as other UKMO tools, are on GitHub, I think it will be easier to have released distributions of ANTS and other tools (e.g., mule, shumlib, ug-ants, etc.) directly connected to their respective GitHub repos and managed in collaboration with the MetOffice.

There is a related issue for mule, for which Scott also has a conda-forge package proposal in queue for review.

I think similar procedures should be put in place for ANTS.

23

Thanks @atteggiani, can you please confirm that analysis3-25.08 will remain active (currently throwing warning on load that it will soon be depreciated) until the release of the dedicated seperate ANTS environment? Can you also please confirm which other packages are planned to be included in the seperate ANTS environment?

24

I’ll defer to @rbeucher on this, as the ACCESS-NRI MED team is managing the analysis3 environments.

There hasn’t been a discussion about this yet.
When the deployment infrastructure is ready (hopefully within a month), I think this could be a good topic to discuss in a Working Group (maybe the Atmosphere one?).

25

Hi @mlipson

We will not deprecate the environment without providing an alternative for your workflow.
As @atteggiani said, we have flagged the issue with the developers. There have been some changes recently, and we are hopeful we will be able to make the new packages available soon.

26

FYI, ants is now released as a stand-alone containerised environment.
This will atomatically be aupdated with new ants releases.

27

Hi all. I’m building a rose/cylc task for the regional Climate model ancillary suite which regrids a MOM6 ocean mask.

My task used to

  • Load conda/analysis3-25.12 (because it contains the mule dependency)
  • Use the version of ants installed with ACCESS-rAM3

However, this now fails with the following warnings and errors

 module that was compiled using NumPy 1.x cannot be run in
NumPy 2.4.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
..
    from mule.packing import wgdos_pack_field, wgdos_unpack_field
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/packing.py", line 107, in <module>
    raise ImportError(err.args + (msg,))
ImportError: ('numpy.core.multiarray failed to import', 'SHUMlib Packing library found, but failed to import')

I note the release of the new ants package here :

However, this environment doesn’t contain xarray. (It does contain numpy, iris and espy), which means my current script won’t run : https://github.com/21centuryweather/rCM3-ancil-suite/blob/main/bin/regrid_MOM6_UM.py

My task follows the working practises of @lachlanswhyborn and the AM3/CM3 ancillary suite :

https://github.com/ACCESS-NRI/CCI-Ancillary-Suite/tree/main/app/landseamask_from_ocean_mesh

https://github.com/ACCESS-NRI/CCI-Ancillary-Suite/blob/main/flow.cylc

The default python environment used in this suite is

		init-script = """
			module use /g/data/xp65/public/modules
			module load conda/analysis3-25.08
			module load nco
		"""

Can the global CCI ancillary suite still with this environment?

If so, what am I missing?

28

The existing ancillary suite definitely still runs with conda/analysis-25.08, but that version was chosen specifically because it was the last environment that shipped with ants (or at least I thought it was- maybe it was mule that was no longer shipped). I suspect ants doesn’t specify its dependencies correctly, which is why the environment managed to concretize with an invalid combination of ants and numpy.

We can deploy a new environment that contains the UKMO modules and xarray- I don’t remember the reason we didn’t include it initially. It feels a bit superfluous to use them in the same task, since they have significantly overlapping functionality- can you load the desired modules for the respective tasks? A newer conda/analysis version for xarray tasks, and the ants package for ants/iris tasks?

29

Yeah I could split the workflow so that it creates the land-sea mask grids and required metadata and save that to netcdf.

And then write a seperate task to load the netcdf file and use the new ants library to save it as a UM ancillary.

I’ll let you know how I go.

30

Ah yep I forgot about that bit. The most recent versions of the conda environment do contain the iris package, and the ants savers do just wrap the iris savers AFAIK. I’ve been trying to move to iris where possible, as ANTS is not public and people generally do not have visibility of the contents of ants.

31

I recall that ants has extra functionality when saving ancillary files that iris doesn’t. i.e. iris won’t transfer all the cube metadata to the ancillary header file.

I’ll double check that and get back to you.

32

There is nothing stopping someone making a PR to add xarray to the ants environment:

Note that it is pretty lean at the moment, which is a good thing. The more packages you add, the harder it becomes to maintain. Eventually you could end up with an entire analysis environment with ants and a bunch of out-dated packages because ants is so prescriptive about dependencies.

But if xarray is obviously required for ancillary processing workflows (and I can see it might be) then we can add it.

33

Ok I will add an xarray issue to the repo (I don’t think I have permissions to make branches).

That would ensure that @lachlanswhyborn 's scripts will work with the new ants environment.

It also help those using the ants environment to post-process and diagnose ancillary and PP files.

34

Eventually you could end up with an entire analysis environment with ants and a bunch of out-dated packages because ants is so prescriptive about dependencies.

I don’t think this is necessarily such a bad thing- the main point of the creation of this ants environment was to be able to remove them from our analysis environments, so that everyone that doesn’t have to deal with UM nonsense are not held back by the people that do have to deal with it. I’m on board with adding xarray to this environment.

35

@Paul.Gregory I can’t replicate this issue on the current main branch of the rCM3-ancil suite, nor on the older “sots full res” branch. The regrid-mom step runs fine and produces a mask and I don’t see these errors or warnings in the output

Could something have changed with your environment? Something in your .bashrc or anything perhaps?

36

Ok I’ve made a test script here : UM_configuration_tools/scripts/test_local_ants_import.sh at main · 21centuryweather/UM_configuration_tools · GitHub

This script:

  • Loads conda/analysis3-25.12
  • Creates a test python file which
    • Adds the path of a fresh local ants install from a regional atmospheric suite to sys.path
    • Attempts to load this local copy of ants.

My .bashrc and .bash_profile are ‘clean’. They execute no commands and contain only aliases.

Here is my output from that script.

Loading xp65 module...
WARNING: The environment 'conda/analysis3-25.12' is more than 8 months old.
It will soon be deprecated. Please move to a newer 'conda/analysis3' environment.

Loading conda/analysis3-25.12
  Loading requirement: singularity
which python
/g/data/xp65/public/apps/med_conda_scripts/analysis3-25.12.d/bin/python
Creating test Python script...
Running test script...
['/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/', '/home/548/pag548/code/UM_config_tools/scripts', '/g/data/xp65/admin/analysis3', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python311.zip', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/lib-dynload', '/home/548/pag548/.local/lib/python3.11/site-packages', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages']

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.4.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/home/548/pag548/code/UM_config_tools/scripts/test_imports.py", line 7, in <module>
    import ants
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/__init__.py", line 5, in <module>
    from . import analysis, config, coord_systems, exceptions, utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/analysis/__init__.py", line 35, in <module>
    import ants.regrid
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/__init__.py", line 25, in <module>
    from . import esmf, interpolation, rectilinear
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/interpolation.py", line 9, in <module>
    import ants.utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/__init__.py", line 8, in <module>
    from . import coord, cube, dask, ndarray
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/cube.py", line 14, in <module>
    import ants.io.save as save
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/save.py", line 30, in <module>
    from ants.fileformats.ancil import _cubes_to_ancilfile, _mule_set_lbuser2
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/fileformats/__init__.py", line 13, in <module>
    import ants.io.load
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/load.py", line 65, in <module>
    import iris.fileformats
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/__init__.py", line 17, in <module>
    from . import name, netcdf, nimrod, pp, um
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/netcdf/__init__.py", line 22, in <module>
    from .._nc_load_rules.helpers import UnknownCellMethodWarning, parse_cell_methods
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/_nc_load_rules/helpers.py", line 38, in <module>
    import iris.fileformats.cf as cf
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/cf.py", line 30, in <module>
    from iris.mesh.components import Connectivity
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/mesh/__init__.py", line 12, in <module>
    from iris.fileformats.netcdf.saver import save_mesh
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/netcdf/saver.py", line 52, in <module>
    from iris.fileformats.netcdf._attribute_handlers import ATTRIBUTE_HANDLERS
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/netcdf/_attribute_handlers.py", line 25, in <module>
    from iris.fileformats.pp import STASH
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/iris/fileformats/pp.py", line 39, in <module>
    import mo_pack
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mo_pack/__init__.py", line 5, in <module>
    from ._packing import (compress_rle, compress_wgdos,
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.4.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/home/548/pag548/code/UM_config_tools/scripts/test_imports.py", line 7, in <module>
    import ants
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/__init__.py", line 5, in <module>
    from . import analysis, config, coord_systems, exceptions, utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/analysis/__init__.py", line 35, in <module>
    import ants.regrid
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/__init__.py", line 25, in <module>
    from . import esmf, interpolation, rectilinear
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/interpolation.py", line 9, in <module>
    import ants.utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/__init__.py", line 8, in <module>
    from . import coord, cube, dask, ndarray
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/cube.py", line 14, in <module>
    import ants.io.save as save
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/save.py", line 30, in <module>
    from ants.fileformats.ancil import _cubes_to_ancilfile, _mule_set_lbuser2
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/fileformats/__init__.py", line 13, in <module>
    import ants.io.load
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/load.py", line 68, in <module>
    from ants.fileformats import _grid_extract, ancil, namelist, pp, raster
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/fileformats/ancil/__init__.py", line 26, in <module>
    import mule
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/__init__.py", line 1788, in <module>
    from mule.ff import FieldsFile  # noqa: E402
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/ff.py", line 29, in <module>
    from mule.packing import wgdos_pack_field, wgdos_unpack_field
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/packing.py", line 46, in <module>
    import um_packing
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/um_packing/__init__.py", line 21, in <module>
    from .um_packing import wgdos_pack, wgdos_unpack, get_shumlib_version
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
Traceback (most recent call last):
  File "/home/548/pag548/.local/lib/python3.11/site-packages/numpy/core/_multiarray_umath.py", line 46, in __getattr__
    raise ImportError(msg)
ImportError: 
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.4.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.


Traceback (most recent call last):
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/packing.py", line 46, in <module>
    import um_packing
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/um_packing/__init__.py", line 21, in <module>
    from .um_packing import wgdos_pack, wgdos_unpack, get_shumlib_version
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: numpy.core.multiarray failed to import

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/548/pag548/code/UM_config_tools/scripts/test_imports.py", line 7, in <module>
    import ants
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/__init__.py", line 5, in <module>
    from . import analysis, config, coord_systems, exceptions, utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/analysis/__init__.py", line 35, in <module>
    import ants.regrid
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/__init__.py", line 25, in <module>
    from . import esmf, interpolation, rectilinear
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/regrid/interpolation.py", line 9, in <module>
    import ants.utils
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/__init__.py", line 8, in <module>
    from . import coord, cube, dask, ndarray
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/utils/cube.py", line 14, in <module>
    import ants.io.save as save
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/save.py", line 30, in <module>
    from ants.fileformats.ancil import _cubes_to_ancilfile, _mule_set_lbuser2
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/fileformats/__init__.py", line 13, in <module>
    import ants.io.load
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/io/load.py", line 68, in <module>
    from ants.fileformats import _grid_extract, ancil, namelist, pp, raster
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/548/pag548/cylc-run/rCM3-ancil-suite/share/fcm_make_ants/build/lib/ants/fileformats/ancil/__init__.py", line 26, in <module>
    import mule
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/__init__.py", line 1788, in <module>
    from mule.ff import FieldsFile  # noqa: E402
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/ff.py", line 29, in <module>
    from mule.packing import wgdos_pack_field, wgdos_unpack_field
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages/mule/packing.py", line 107, in <module>
    raise ImportError(err.args + (msg,))
ImportError: ('numpy.core.multiarray failed to import', 'SHUMlib Packing library found, but failed to import')

It would be great to get to the bottom of how this changed.

37

Running that script you linked, but replacing the path to ants with one generated by my main branch copy of the rCM3-ancil-suite

sys.path.insert(0,'/scratch/gb02/ab8992/cylc-run/mainbranch_rCM3-ancil-suite/share/fcm_make_ants/build/lib/')

I get:

bash paul_test 
Loading xp65 module...
WARNING: The environment 'conda/analysis3-25.12' is more than 8 months old.
It will soon be deprecated. Please move to a newer 'conda/analysis3' environment.

Loading conda/analysis3-25.12
  Loading requirement: singularity
which python
/g/data/xp65/public/apps/med_conda_scripts/analysis3-25.12.d/bin/python
Creating test Python script...
Running test script...
['/scratch/gb02/ab8992/cylc-run/mainbranch_rCM3-ancil-suite/share/fcm_make_ants/build/lib/', '/g/data/gb02/ab8992/tmp', '/g/data/xp65/admin/analysis3', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python311.zip', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/lib-dynload', '/home/149/ab8992/.local/lib/python3.11/site-packages', '/g/data/xp65/public/apps/med_conda/envs/analysis3-25.12/lib/python3.11/site-packages']
Cleaning up...
Done.

Could you try running with the same ants path as above if permissions are ok? If you’re also on main branch of rcm3 suite then that won’t make a difference

My site packages (as included in my path) do not contain any ants instances. Do you have things in your site packages that could be clashing? Try clear your site packages, or move them somewhere else temporarily?

38

Paul have you checked that nothing from your .local directory or anything is getting included when you run the script? Could be something you’ve pip installed using an old numpy version. python -s or export PYTHONNOUSERSITE=x