Ancils for small test domain fail checking during reconfiguration due to longitude rounding errors

[Paul.Gregory]

More fun and games with a small test domain.

Outer domain is specified in the regional nesting suite as

rg01_centre=-28.5,153.5
rg01_igbp_offset=0,0
rg01_name="Lismore"
rg01_nreslns=2
rg01_rs01_delta=0.10,0.10
rg01_rs01_npts=100,100

Inner domain is specified with

rg01_rs02_name="d1100"
rg01_rs02_delta=0.10,0.10
rg01_rs02_npts=50,50

Reconfiguration fails. First there are multiple warnings with the qrclim.sea ancillary file.

????????????????????????????????????????????????????????????????????????????????
??????????????????????????????      WARNING       ??????????????????????????????
?  Warning code: -15000
?  Warning from routine: ANCIL_CHECK_HORIZONTAL_GRID
?  Warning message: Mismatch between model and ancil field x grid spacing
?          Expected x grid spacing : .1000000000
?          Ancil    x grid spacing : .0999998748
?          Ancil file : /scratch/gb02/pag548/cylc-run/rCM3-test-UM-ancil/share/data/ancils/Lismore/d1100/qrclim.sea
?          Lookup num : 1
?          Stashcode  : 96
?  Warning from processor: 0
?  Warning number: 2
????????????????????????????????????????????????????????????????????????????????

There are seven of these warnings, followed by the error

???????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 1001
?  Error from routine: ANCIL_CHECK_MOD::REPORT_ANCIL_ERRORS
?  Error message:
?        ERRORS: 1 ancil files have failed ancil checking and resulted in this abort:
?        -- /scratch/gb02/pag548/cylc-run/rCM3-test-UM-ancil/share/data/ancils/Lismore/d1100/qrclim.sea
?
?        WARNINGS: 1 ancil files have failed ancil checking. These files did not cause an abort
?        due to the setting of l_ignore_ancil_grid_check=.true. in the items namelist:
?        -- /scratch/gb02/pag548/cylc-run/rCM3-test-UM-ancil/share/data/ancils/Lismore/d1100/qrparm.orog.mn
?
?        Any fields causing an error will have produced an ereport
?        warning earlier in the run, please search the log files for
?        each ancil filename for more details on each failure.
?  Error from processor: 0
?  Error number: 14
????????????????????????????????????????????????????????????????????????????????

If you load qrclim.sea into xconv the x coords are not defined at 0.1 degree increments at sufficient precision to please the UM.

Note the y coords of the same file are all displayed in 0.1 degree increments.

For the outer domain (100 by 100 points), both x and y coordinates of qrclim.sea suffer from precision / rounding issues.

For fun, I replaced all entries in app/um/rose-app.conf of
l_ignore_ancil_grid_check=.true.
to
l_ignore_ancil_grid_check=.false.
This made no change, presumably because each entry in rose-app.conf for ancil checking is tied to an ancil filename and STASH entry, and as yet there are none for qrclim.sea

So, given that
a) The same domain size works fine with 0.11 spacing
b) I’m running a large 0.1 degree domain right across most of Australia right now without dramas
c) So has everyone else since the dawn of time
d) I’ve never seen any issues with rounding errors created by the suites in-built ancillaries before
e) There is no extra information in the um_recon pe0 file

I’m a bit stumped. We can manually fix the ancillary with ants but I’m curious as to how this an happen in the first place.

Has anyone ever encountered this before?

[Scott (Scott Wales)]

As I understand it all of the ancillaries should be basing their grid off of the land sea mask. Looking at the RAS app app/ancil_clim_sea/rose-app.conf it’s being given a target land mask. See if you can check if this target mask matches the one you’re using for the run.

[Paul.Gregory]

Yeah, I’ve been digging around the RAS suite for the past few months to understand its workflow and think of ways to build consistent regional UM and MOM6 regional masks.

The main initial task for ancillary generation in RAS is:
<Region>_<resolution>_ancil_lct

which calls the script ancil_lct.py , which in turn returns

1 lct_cube - the 9 level vegetation area fraction cube
2 [land_mask_cube , land_fraction_cube]
where the land_mask_cube is binary, land_fraction_cube is a float (i.e values b/w 0.0 to 1.0)

The land mask cube is then written to disk, which then defines the grid which all other ancillaries are built from.

In our case qrparm.mask (which links to qrparm.mask_cci) displays in xconv as

So I’m going to run the ancil regridding task through a debugger and see why it doesn’t want to respect these grid co-ordinates.

The actual task command is

ants-launch ancil_general_regrid.py --ants-config ${ANTS_CONFIG} \
       ${source} --target-lsm ${target_lsm} --invert-mask -o ${output}

where source=<path to>/GlobColour/v2/qrparm.sea.nc
and target_lsm=<path to>qrparm.mask_cci

[Scott (Scott Wales)]

Yeah, I’ve been digging around the RAS suite for the past few months to understand its workflow and think of ways to build consistent regional UM and MOM6 regional masks.

Start with a single source of truth (for global runs we used the ocean mask as it was a higher resolution) and interpolate the land mask and land fraction from that - land fraction was a conservative regrid of the ocean mask to the land grid with a bit of cleanup something like >=99% land grid points were just set to land and similar for ocean.

There’s an opt in the ancil_lct app vegfrac_only that will get it to use a target mask rather than generate the mask itsself

[Paul.Gregory]

Thanks Scott. I’ve been working with @kieranricardo to replicate their CM3 workflow, which is
a) Define a land mask in MOM6
b) Interpolate that land mask (using the ESMF algorithms used by NUOPC) onto the target UM grid definition.
c) Use that interpolated land mask as the ‘target mask’ for subsequent UM ancil tasks.

I think that reflects your suggestion above. The idea is this retains consistency b/w the NUOPC ‘CAPS’ created inside the NUOPC executable when it passes UM/MOM6 gridded information.

I’ve successfully run ancil_lct in ‘target mask’ mode.

BUT I found the subsequent downstream tasks still required a ASCII grid.nl file (i.e. the namelist definition of the grid) - having a target mask alone is not sufficient.

So now I’m just figuring out which parameters you need to define a grid.nl file.

[Scott (Scott Wales)]

Sounds good - if you’re not using a rotated pole or stretched grid the grid.nl should be pretty basic, pretty much just origin and spacing as I recall - the RAS should generate it