Issues when transitioning from hh5 to xp65 in suite runs

General
, ,
1

Hi all,

I’m trying to run the u-cy339 suite in ACCESS-CM2 following this instruction: Run ACCESS-CM - ACCESS-Hive Docs. Since I don’t have hh5 membership which I understand is being closed, I’m trying to transition everything regarding hh5 to xp65 in the suite. But I have encountered some issues:

1. Issue with loading conda environment:

At the beginning, I encountered an error in the [ocean_ke_check] step like:

Screenshot 2025-05-26 at 5.39.53 PM
Screenshot 2025-05-26 at 5.39.53 PM1594×1274 207 KB

Then, I found these lines in the suite.rc file:

module use /g/data3/hh5/public/modules
module unload python
module load conda/analysis3

Following the announcement: ACCESS-NRI Analysis3 Conda Environments – New Release Announcement. I replaced hh5 with xp65 in the path, and it seems to work now. Is it the correct approach to resolve this issue?

2. Issue with um2netcdf4 script:

However, I encountered another issue in [netcdf_conversion] step. The error message is as follows:

Screenshot 2
Screenshot 21528×1408 139 KB

This error comes from um2netcdf4 command in suite.rc file:

module use ~access/modules
module unload python
module load pythonlib/um2netcdf4.

The following is the script of um2netcdf4 which still seems to rely on hh5:

$ cat ~access/modules/pythonlib/um2netcdf4/2.1
#%Module

set help            "Conversion of UM fieldsfiles to netCDF"
set prefix           ~access/apps/pythonlib/um2netcdf4/2.1
set install-contact "martin.dix@csiro.au"
set install-date    "2023-01-16"
set url             "https://bitbucket.csiro.au/users/dix043/src"

conflict pythonlib/um2netcdf4

if ![ is-loaded conda/analysis3 ] {
    module use /g/data3/hh5/public/modules
    module load conda/analysis3
}

prepend-path PYTHONPATH $prefix
prepend-path PATH $prefix
setenv UMDIR /g/data/access/projects/access/umdir

Would it be possible to update um2netcdf4 to use xp65, or is there an existing version that supports xp65?

Does anyone have any suggestions? Thanks in advance!

Hao Wang

2

um2netcdf4.py uses the mule package which isn’t yet included in xp65.

I’ll look at a workaround but for the moment stay with hh5.

3

Thanks for looking into a workaround, @MartinDix . I’ll hold off now.

5

Hi all,

xp65 provides a copy of the hh5 analysis3-24.07 environment, which includes mule and other related packages. We’re currently looking into how these can be supported in the newer environments and will get back to you as soon as possible.

In the meantime, you can load the analysis3-24.07 environment on xp65 with:

module use /g/data/xp65/public/modules
module load conda/analysis3-24.07
6

Hi all,
I am also updating to xp65 and have found that when using the default analysis3 environment (analysis3-25.04), I get an error when I try to do import xesmf in python.
If I revert back to analysis3-24.07, then xesmf works. Just wanted to flag this for future updates.
Regards,
David

7

Hi @dkhutch ,

I believe it has been fixed in the recent versions of analysis3.

8

@rbeucher yes it seems to work for me thanks.

9

Hi again,
Now that I’m trying to use up-to-date analysis3, i.e. conda/analysis3-25.06, I’ve noticed that the tool ncvarinfo is no longer there. I really like that tool… I think it was written by @Aidan a long time ago. Anyway, are there plans to keep it in the conda analysis package? I can “install” it manually otherwise.

15

dkhutch

@rbeucher yes it seems to work for me thanks.

@Aidan @dkhutch
Can I mark this problem as being solved, and move the discussion after 1 July to a separate thread?

16

Hi NRI-team, @spencerwong

hh5 is going to be decommissioned very soon (next week?). It would be great if the Run ACCESS-CM2 instructions could be updated. hh5 paths are hard-coded in many of the CM2 suites as described in this post and will likely not work anymore.

Thanks,
Wilma

17

There is an open issue to update the run docs

if you have any specific feedback that would be a good place to put it.

18

Thanks, done!

19

I’m having the samenetcdf_conversionissue as mentioned in this post when moving to xp65. I don’t understand the provided recommendation and am hoping someone can clarify the suggested workflow to me.

The file that creates the problem (i.e. is trying to load hh5 modules) is part of ~access/modules/pythonlib/ so I’m confused where I need to specify the different conda environment. I assume I’m not meant to make a local copy of um2netcdf4.

Should the conda environment be specified in the netcdf_conversion section of suite.rc?

[[netcdf_conversion]]
inherit = POSTPROC, NCI, SHARE
pre-script = “”"
module use ~access/modules
module unload python
module load pythonlib/um2netcdf4

20

Hi @wghuneke , this is set in site/nci_gadi.rc, under the [[CONV]] section:

    [[CONV]]
        pre-script = """
            module use /g/data/xp65/public/modules
            module unload conda
            module load conda/analysis3-24.07

I assume you have already got everything else in your suite set up to run with xp65, but in case not, you also need to add gdata/xp65 to set storage_projects in site/nci_gadi.rc :

{% set storage_projects = [ ,… your other storage flags…], ‘gdata/xp65’, ‘scratch/’+environ[‘PROJECT’], ‘gdata/’+environ[‘PROJECT’]] %}

Handy tip, if you’re confused about where to add or edit something, then go to the top level of your rose directory and grep for it - e.g. in this case run grep -ir ‘module load conda/analysis’ from the top of your suite’s rose directory, and it will pull out all files where this line exists :slight_smile:

21

Thanks @bethanwhite for getting back to me!

I think your suggestion does not work for my suite which is set up differently. (I run a CM2 suite, copy of u-db130.) The equivalent to the site/nci_gadi.rc file in my suite is suite.rc which is where I set the storage flags and also updated to the xp65 conda environment under the ocean_ke_check section.

My question is regarding the um2netcdf4 section in suite.rc for which a different python module is loaded separately (see my last post) from a central pythonlib directory under /g/data/access - which I don’t want/should/can touch. I assume all CM2 suites call this module. Since the module unloads any existing conda environments and then loads the hh5 environment, I don’t think specifying anything in the suite.rc file will help. But my understanding might be wrong and I possibly make things more complicated then they are. (Am I making sense?)

I would very much appreciate if someone can clarify this for me and suggest a workaround.

Many thanks,
Wilma

22

Hi @wghuneke, hmmm, ok, I think I understand you now. My example above is from the unreleased version of AM3, so perhaps things work differently in the next release. There’s no um2netcdf4loaded in the suite I have.

If um2netcdf4 is where the environment is loaded then I suspect the global solution would be for ACCESS-NRI to create an xp65 version of um2netcdf4 in access/modules.

Otherwise as a temporary workaround, you could create your own local copy, change it to xp65, and call your local script.

23

@wghuneke EDIT: I’ve just had a poke around in access/modules - there’s an xp65 version in there already:

$ ll /g/data/access/modules/pythonlib/um2netcdf4/
total 12
-rw-r-----+ 1 mrd599 access.admin 525 Mar 10  2022 2.0
-rw-r-----+ 1 mrd599 access.admin 525 Jan 16  2023 2.1
-rw-r-----+ 1 mrd599 access.admin 735 Aug 15 08:58 xp65

Can you try explicitly loading it and see what happens (unless someone from NRI jumps in to say we shouldn’t be doing that)?:
module load pythonlib/um2netcdf4/xp65

24

Thanks @bethanwhite , your last suggestion worked!

In summary, to switch CM2 suites from hh5 to xp65, you will have to change 3 things in suite.rc :

  1. Update -l storage flag to include gdata/xp65

  2. [[ocean_ke_check]]: Load a specific (at least no newer than) version of the xp65 conda environment:
    module use /g/data/xp65/public/modules module unload python module load conda/analysis3-25.05

  3. [[netcdf_conversion]]: change to module load pythonlib/um2netcdf4/xp65