I’m trying to run the u-cy339 suite in ACCESS-CM2 following this instruction: Run ACCESS-CM - ACCESS-Hive Docs. Since I don’t have hh5 membership which I understand is being closed, I’m trying to transition everything regarding hh5 to xp65 in the suite. But I have encountered some issues:
1. Issue with loading conda environment:
At the beginning, I encountered an error in the [ocean_ke_check] step like:
This error comes from um2netcdf4 command in suite.rc file:
module use ~access/modules
module unload python
module load pythonlib/um2netcdf4.
The following is the script of um2netcdf4 which still seems to rely on hh5:
$ cat ~access/modules/pythonlib/um2netcdf4/2.1
#%Module
set help "Conversion of UM fieldsfiles to netCDF"
set prefix ~access/apps/pythonlib/um2netcdf4/2.1
set install-contact "martin.dix@csiro.au"
set install-date "2023-01-16"
set url "https://bitbucket.csiro.au/users/dix043/src"
conflict pythonlib/um2netcdf4
if ![ is-loaded conda/analysis3 ] {
module use /g/data3/hh5/public/modules
module load conda/analysis3
}
prepend-path PYTHONPATH $prefix
prepend-path PATH $prefix
setenv UMDIR /g/data/access/projects/access/umdir
Would it be possible to update um2netcdf4 to use xp65, or is there an existing version that supports xp65?
Does anyone have any suggestions? Thanks in advance!
Hao Wang
2 Likes
MartinDix
(Martin Dix ACCESS-NRI Associate Director for Model Development)
2
um2netcdf4.py uses the mule package which isn’t yet included in xp65.
I’ll look at a workaround but for the moment stay with hh5.
xp65 provides a copy of the hh5analysis3-24.07 environment, which includes mule and other related packages. We’re currently looking into how these can be supported in the newer environments and will get back to you as soon as possible.
In the meantime, you can load the analysis3-24.07 environment on xp65 with:
module use /g/data/xp65/public/modules
module load conda/analysis3-24.07
Hi all,
I am also updating to xp65 and have found that when using the default analysis3 environment (analysis3-25.04), I get an error when I try to do import xesmf in python.
If I revert back to analysis3-24.07, then xesmf works. Just wanted to flag this for future updates.
Regards,
David
Hi again,
Now that I’m trying to use up-to-date analysis3, i.e. conda/analysis3-25.06, I’ve noticed that the tool ncvarinfo is no longer there. I really like that tool… I think it was written by @Aidan a long time ago. Anyway, are there plans to keep it in the conda analysis package? I can “install” it manually otherwise.
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
10
Good gracious! It took me a few minutes to track it down and I wrote it!
The nccompress package has been written to facilitate compressing netcdf files. Although nccompress can work on single files, it is particularly useful to compress all uncompressed files under whole directory trees. This can allow users to compress files regularly using the same script each time.
The nccompress package consists of three python programs, ncfind, nc2nc and nccompress. nc2nc can copy netCDF files with compression and an optimised chunking strategy that has reasonable performance for many datasets. His two main limitations: it is slower than some other programs, and it can only compress netCDF3 or netCDF4 classic format. There is more detail in the following sections.
The convenience utility ncvarinfo is also included, and though it has no direct relevance to compression, it is a convenient way to get a summary of the contents of a netCDF file.
For those following along at home; can I find this on gadi (and if so, where?) or do I need to install for myself?
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
13
I think an install into a virtual environment is your best bet. The code is not currently maintained, so I can’t see a use case to put it into a shared conda environment until that status changes.
