Making domain beyond 60 S in ACCESS-rAM3

Hi everyone,

I am trying to run RAS to generate some ancillary files for a domain beyond 60 S, and I encounter some issues as follows:

  • The job.err of _ancil_mean_orog shows:
raise Exception('!!!!! ERROR: SRTM data not available for regions above 60 deg. Max:'+str(North)+' Min:'+str(South))
Exception: !!!!! ERROR: SRTM data not available for regions above 60 deg. Max:2 Min:-82
  • _ancil_soils_hydr failed due to running out of time
 PBS: job killed: walltime 3697 exceeded limit 3600

I was just wondering if there is some way to extend my domain beyond 60 S? Given the SRTM data not avalable beyond 60S, could we use another source to generate _ancil_mean_orog?

Thanks!

P.S. The domain I would like to simulate

For [-40, 99] with [0.1, 0.1] and [800, 1200]: 
coords = 39.000, 159.000, -80.000, 0.000

P.P.S. For my simulation, I actually do not need very accurate orography data around the Antarctic. Would it be possible/reasonable to just mask out the orography flat beyond 60 S?

Hi @chenhui.jin, the SRTM dataset for orography only extends to -60. You can though use the GLOBE dataset instead. This is set in the regional ancillary suite. There is a true/false toggle for {}_{}_orog_srtm under the resolution setup.

Please be aware though that BARRA does not extend even to -60 so you will need to use either era5-land for the land/surface or era5 (I would use era5-land, and shift the lat/lons to a grid-corresponding to era5-land points with 0.1 degree spacing).

NOTE: You would need to switch both rg01_rs01_orog_srtm and rg01_rs02_orog_srtm to .FALSE. .

Please reach out if you need help.

1 Like

Hi @cbengel,

Thanks for your help. I had a try, it seems the _ancil_mean_orog is not triggered this time.

There are still some issues with _ancil_soils_hydr (due to running out of time), and _ancil_cap_sstice (job.err doesn’t show anything helpful to find where the issue occurred).

job.err for _ancil_soils_hydr:

Using the cylc session cylc.cj0591.if69.ps.gadi.nci.org.au

Loading cylc7/24.03
  Loading requirement: mosrs-setup/2.0.1
=>> PBS: job killed: walltime 3707 exceeded limit 3600
Terminated
2026-03-25T11:30:31Z CRITICAL - failed/TERM

job.err for _ancil_cap_sstice:

Using the cylc session cylc.cj0591.if69.ps.gadi.nci.org.au

Loading cylc7/24.03
  Loading requirement: mosrs-setup/2.0.1
gc_abort (Processor     0): ANCIL EXIT
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 9.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[FAIL] rose mpi-launch -v central_ancillary.exe # return-code=9
2026-03-25T11:08:19Z CRITICAL - failed/EXIT

Hi @chenhui.jin ,

You can change the wall time in site/nci-gadi/suite-adds.rc, line 69. Change the execution time limit from PT1H to PT2H.

For the _ancil_cap_sstice error, I would re-submit the job and try again. It might be a gadi issue.

By the way, is your grid very large? If so,@Paul.Gregory might be able to give you advice.

1 Like

Hi @chenhui.jin I was talking to @alanah.chapman yesterday at the roundtable. Alanah is going to try to run rAM3 at very high latitudes for some sea ice / aerosol studies.

We were discussing the same issues you are raising, and we discovered this old thread on the hive.

I would reach out to @ZhangchengPei to ask about using the other orography datasets that have coverage over Antarctica.

I don’t believe anyone has yet run rAM3 using those other datasets. But there is definite interest in doing so.

2 Likes

Thanks for the suggestion. I’ll have a try and let you know how it goes.

Yes, I have grid numbers 800 and 1200 in Y and X direction respectively

1 Like

Hi @cbengel,

Changing the time limit solves the _ancil_soils_hydr.

However, the _ancil_cap_sstice issue persists after a couple of attempts, with the same error message.

Hi @chenhui.jin, I suggest that you reach out to @ZhangchengPei because it might be a dataset issue near Antarctica (given it is a SST issue). Otherwise I would suggest shifting your lower boundary northward.

1 Like

Thanks, @cbengel and @Paul.Gregory, I will reach out to @ZhangchengPei.

P.S. I was also wondering if I can run the model without _ancil_cap_sstice ?

Yes. You could try setting _sstsmc_anc to .FALSE. in rg01_rs01 and rg01_rs02 and see how you go.

Chenhui - if you want to run the model with unrealistic southern boundary conditions, you could try and stay with the srtm orogoraphy and do what Chermelle has suggested above.

If you comment out lines 168-169 of /src/contrib/Apps/Orography/concant_srtm_tiles.py

if North> 60 or South < -60:
        raise Exception('!!!!! ERROR: SRTM data not available for regions above 60 deg. Max:'+str(North)+' Min:'+str(South))

You can generate SRTM orography over Australia and flat nothing all the way to the South Pole.

See here :

https://21centuryweather.discourse.group/t/flagship-domain-for-ram3/1196/67

You will need to ensure your land sea mask is also consistent (no land below 60 south), otherwise you will generate soil and vegetation ancils in this region.

I have an idea how you can do this if you want to pursue this (i.e. generation ancils using SRTM but enforce no land south of 60 degrees).

Hi @Paul.Gregory,

I feel open to testing different ways :slight_smile:

Now we have:

  • @cbengel’s method: switch off {}_{}_orog_srtm, while we will still have some issues about _ancil_cap_sstice, which I will reach out to @ZhangchengPei for a solution.
    Alternatively, I switched _sstsmc_anc to False, and all tasks are completed, and I assume I could run the model without _ancil_cap_sstice for my application?

  • Paul’s method: For my understanding, we still generate the ancils from STRM, but manually mask land flat south of 60 S?

A bit trickier but you could also blend the two - have SRTM above 60S and non-SRTM below it - should be able to do this with either ANTS or Mule

1 Like

Hello @Paul.Gregory,

Can you explain this idea a bit further? I would like have a try on this.

Hi Chermelle,

I discussed with Zhengcheng, and I managed to create some ancillaries of sstice.

Here is the successful domain:

For [-42, 81] with [0.1, 0.1] and [820, 1600]: 
coords = 1.000, 161.000, -83.000, -1.000

For [-42, 81] with [0.1, 0.1] and [800, 1580]: 
coords = 2.000, 160.000, -82.000, -2.000

I used Paul’s ancillary checker to check the ancillaries. There are NaNs around the Antarctic, mainly associated with soils

Is ok to run the model with these ancillaries?

1 Like

Hi @chenhui.jin, Did @Paul.Gregory fix the NaNs?? If not, the model may not run.

You would have to defer to your supervisor regarding the suitability of the settings from a scientific perspective.

NaNs are not fixed yet.

Thanks for the suggestion :-). I’d double-check the suitability with others

Hi Chenui.

The first task in the regional ancillary suite : <regional>_<resolution>_ancil_lct generates a land-sea mask that is used to define all downstream soil and vegetation fields.

Here is what the cylc graph for the suite looks like:

install_cold_idl => install_cold_hpc
install_cold_hpc => <Region>_<resolution>_ancil_top
install_ants & install_ugants => <Region>_<resolution>_ancil_lct
<Region>_<resolution>_ancil_top => <Region>_<resolution>_ancil_lct
<Region>_<resolution>_ancil_lct => <Region>_<resolution>_ancil_lct_prostproc_c4 => <Region>_<resolution>_ancil_cap_orog => <Region>_<resolution>_ancil_ants_housekeep
<Region>_<resolution>_ancil_lct => <Region>_<resolution>_ancil_lct_prostproc_wc_urban => <Region>_<resolution>_ancil_lct_prostproc_c4
<Region>_<resolution>_ancil_cap_orog => <Region>_<resolution>_ancil_mean_orog  => <Region>_<resolution>_ancil_mean_orog_smooth => <Region>_<resolution>_ancil_ants_housekeep

Here is it in graphical form:

So I believe you can use this suite to remove all soil and vegetation ancillary data. What I would do is

  1. Run the regional ancillary suite using your current domain definitions, but begin the suite with rose suite-run -- --hold. This will load the suite, but hold all tasks.

  2. Manually trigger the upstream tasks
    install_cold_idl, install_cold_hpc, install_ants & install_ugants and <Region>_<resolution>_ancil_top. Do this by right-clicking on the task-name and selecting ‘Trigger now’ in the pop-up menu.

  3. Manually trigger all the <Region>_<resolution>_ancil_lct tasks you have.

These will create the files qrparm.mask which is the land-sea mask used by each resolution (or nest).

  1. Manually edit this file to remove all land points south of 60 degrees. You can do this by using the iris module and the iris.load_cube() method. Scott Wales wrote an example of this on the old CLEX wiki - I’m hosting it now on the 21st Century Weather wiki here:
  1. Once you have saved your updated qrparm.mask file with all land points removed south of 60 degrees, you can ‘release’ all the downstream tasks in your suite. Use the cylc GUI pull-down menu from the top taskbar : ‘Control’ → ‘Release Suite (unpause)’.

This should generate a set of land, soil and vegetation ancillaries that are consistent with a semi-aquaplanet (i.e. no Antarctica).

BUT this may only be the first step in creating this make-believe scenario.

You may have to do some edits with
a) Your sea-surface temperature datasets, as they are probably affected by real observed sea ice
b) Your snow and ice related climatology ancillary files (although it sounds like you are already doing this).

It’s an interesting problem, but that’s how I’d go about generating a semi-aquaplanet experiment (if this is what you’re after).

2 Likes

Hi Paul,

Many thanks for your step-by-step instructions! I will have a go and report back.