Hi @wghuneke, hmmm, ok, I think I understand you now. My example above is from the unreleased version of AM3, so perhaps things work differently in the next release. There’s no um2netcdf4loaded in the suite I have.
If um2netcdf4 is where the environment is loaded then I suspect the global solution would be for ACCESS-NRI to create an xp65 version of um2netcdf4 in access/modules.
Otherwise as a temporary workaround, you could create your own local copy, change it to xp65, and call your local script.
@wghuneke EDIT: I’ve just had a poke around in access/modules - there’s an xp65 version in there already:
$ ll /g/data/access/modules/pythonlib/um2netcdf4/
total 12
-rw-r-----+ 1 mrd599 access.admin 525 Mar 10 2022 2.0
-rw-r-----+ 1 mrd599 access.admin 525 Jan 16 2023 2.1
-rw-r-----+ 1 mrd599 access.admin 735 Aug 15 08:58 xp65
Can you try explicitly loading it and see what happens (unless someone from NRI jumps in to say we shouldn’t be doing that)?: module load pythonlib/um2netcdf4/xp65
Thanks @bethanwhite , your last suggestion worked!
In summary, to switch CM2 suites from hh5 to xp65, you will have to change 3 things in suite.rc :
Update -l storage flag to include gdata/xp65
[[ocean_ke_check]]: Load a specific (at least no newer than) version of the xp65 conda environment: module use /g/data/xp65/public/modules module unload python module load conda/analysis3-25.05
[[netcdf_conversion]]: change to module load pythonlib/um2netcdf4/xp65
Thank you everyone for your contributions to this topic!
It has been very helpful and some pieces of information have now been incorporated in a PR to update the “How to run ACCESS-CM2” instructions on the Hive Docs.
Is there a NetCDF compression tool equivalent to nccompress in the new xp65 space? I was routinely running it on my files but I can see it is no longer available. I have no idea what installing to a virtual environment means I know python has compression options but I also use other languages to work with NetCDFs. Thanks
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
28
I am glad you found nccompress useful @aukkola. Assuming that is what you’re referring to?
It hasn’t had substantial changes in over 6 years, so I can’t guarantee it is still fit for purpose.
There may well now be better options available. e.g. nco or cdo (though the cdo documentation seems to be publicly inaccessible currently).
Thanks Aidan. That’s a shame nccompress is no longer maintained, it was such a no-brainer to run it on files to save storage space. I’ll see if I can find something similar using NCO/CDO
I know python has compression options but I also use other languages to work with NetCDFs. Thanks