ACCESS-CM2: changes in access-cm2-drivers from fcm_make_drivers

Earth System
1

Hi all

When the ACCESS-CM2 workflow changed from accessdev to run in persistent sessions the process of installing drivers also changed for suites that previously ran on accessdev
The fcm_make_drivers process changed from getting files from trac.nci.org.au to cloning a git repo (GitHub - ACCESS-NRI/access-cm2-drivers: Driver scripts for the ACCESS-CM2 coupled model).

See here for how I got the drivers to install and other changes required to make the coupled job run

After making these changes, I found that there are some issues from causing processing based not to work because of changes in the the scripts used from access-cm2-drivers.

The history_postprocess and netcdf_conversion jobs both fail.
image

The error for history_postprocess relates to THISYEAR variable not being found in the history_postprocess.sh from the drivers location.

In the example suite (cy399) THISYEAR is defined in suite.rc in these lines where history_postprocess is called to run.

FIX: you need to add the [[[environment]]] block shown below for the script to work.

[[history_postprocess]]
        inherit = POSTPROC, NCI, SHARE
        script = history_postprocess.sh
        [[[job]]]
            execution time limit = PT90M
        [[[environment]]]
            THISYEAR=$(cylc cyclepoint --print-year)
        [[[directives]]]
            -l mem = 20gb
            -l ncpus = 1
            -q = {{NCIEXQ}}

For the netcdf_conversion process, I was getting an the error from showing that run_um2netcdf.py is failing when trying to use um2netcdf4 library commands. This was because there are some problems with the environment that was loaded in the suite.rc

FIX: In suite.rc, you need to change
module load pythonlib/umfile_utils/access_cm2 to module load pythonlib/um2netcdf4
as below

    [[netcdf_conversion]]
        inherit = POSTPROC, NCI, SHARE
        pre-script = """
            module use ~access/modules
            module unload python
            module load pythonlib/um2netcdf4
        """ 
        [[[job]]]
              execution time limit = PT150M
          [[[directives]]]
              -l mem = 40gb
              -l ncpus = 1
              -q = normalbw

If you have any questions or feedback for this post let me know. These changes are based on what I had to do to get cy286 to work. There may be other small changes I have forgotten to list

Remaining questions

I noticed is that both um2netcdf.py and history_postprocess.sh script are slightly different in this new git version compared to my old suites. Does anyone know if these changes do anything different to the calculations made for the ouputs or why they would be different now?

The other thing I saw in cy399 suite.rc is that there is a process called [[ocean_ke_check]] which runs one of the access-cm2-drivers scripts. This script is not called in my original suite so I am wondering what this does?

 [[ocean_ke_check]]
        inherit = POSTPROC, NCI, SHARE
        pre-script = """
            module use /g/data/hh5/public/modules
            module unload python
            module load conda/analysis3
        """
        script = """
            ocean_ke_check.py ${OCN_RUNDIR}/HISTORY/ocean_scalar.nc
        """
        [[[job]]]
            execution time limit = PT5M
            batch system = background
1 Like
2

In CM2, at some point, possibly due to chaos, the grid point near Sumatra Island, at a depth of 400m, experiences an enormous current, which suddenly alters the circulation in the Indian Ocean. Although the entire model can still manage this, the overturning becomes unrealistic. Reflecting on past experiences, we encountered this issue and realized we could use MOM’s kinetic energy (KE) output to detect it. Typically, the KE value remains around 1500, but it jumps to over 2000 when this issue occurs. Interestingly, this event occurs only in CM2, not even in OM2. We are still unsure of the exact cause but are using KE as an indicator.

3

Maybe a bit off topic, but are you using Rayleigh damping @ars599?

From the ACCESS-OM2 technical paper

At 1∘ Rayleigh damping is used to improve the Indonesian throughflow transport; a damping timescale of 1.5 h is applied at all but the bottom two (three) U cells in Lombok (Ombai) Strait and for 3 ∕ 4 of the width of the Torres Strait at all depths. At 0.1∘ a damping timescale of 1.5 h is used at all depths across the full width of Kara Strait to constrain the velocity, which otherwise leads to numerical instability unless an unfeasibly small time step is used. There is no Rayleigh drag in the 0.25∘ configuration.

@sebmckenna if you want ACCESS-NRI support, or want to flag this for help in general, make sure you use the help tag.

For more information about help and support:

1 Like
4

I want to help here, but I am unsure if I will be helpful. My experience is that um2netcdf.py is a varied beast and there are somewhere up to 20 versions floating around. The bottom line though is that these variations are mostly cosmetic or suite specific, so as long as you can find the data you want, you should be OK.

If you can compare a previous version’s output against your latest output, then you should be able to be confident that the newer version is effectively the same.

</2cents>

1 Like
5

OK thanks all.

That sounds a little concerning about the ITF energy spike. Was this an issue when it ran using accessdev or only in the newer versions? It looks like the script just stops the run if it does happen @ars599

@griff Thanks, that’s good to know its mostly cosmetic. I’ll do some tests to double check outputs are the same. Thankyou so much!

6

FYI @atteggiani is working on an updated um conversion tool, um2nc

I am sure he would welcome people trying it out and giving feedback.

1 Like
7

Hahah yes!! Dave uses the Rayleigh damping to fix it, but we still keep the ocean_ke_check in case that happens again. Somehow when we run 1/4 ocean, the KE did jump again. Not sure if that is correct, but we assumed that makes sense at this stage. Higher resolution, higher KE?!?!

8

Hello Griff. I hope Martin can help on this :slight_smile: in regarding the um2netcdf.py.

1 Like
9

I would make sure to check very carefully that the Rayleigh damping is applied at the correct locations. I’ve seen lots of misconfigured Rayleigh damping warnings from MOM5 in the past. Not that I’m saying it was Dave’s configuration, just that it seems to be something that often gets overlooked, or misconfigured.

10

totally agree!! some of that are carried from mom4 … so you know how long ago :slight_smile:

Is there any way we can check it? Or need Dave?

11

The MOM model output will tell you if the points are over land

but you can also output some diagnostics to check against the data or grid

There is also no substitute for just using ncview and checking the i,j indexes against the bathymetry. Tedious though.

1 Like
12

Many thanks Aidan, I will bring to the coupled model meeting today.

1 Like