CABLE in AM3 development notes

Land Surface
1

This topic is to collate meeting notes around the ongoing development of ACCESS-AM3 with a focus on role of CABLE.

Key individuals: @clairecarouge @MartinDix @ben @Jhan @inh599

2

Meeting 16/1/2025

Status of developments:

  • AM3 is functional, proto-type AM3 codebase and suite has been shared with university colleagues.
  • CASA has been run using the AM3 codebase but output has not been assessed in anyway.
  • energy/water balance is reasonable (commensurate with CM2), still some energy imbalance over snow regions which could be followed up on at a later date.
  • progress to back port recent AM3 changes back into CABLE3-MAIN and ESM1.6 have resulted in a range of other concerns/technical challenges.
  • dust in proto-type AM3 is too low - attempts to rectify via changing (increasing) CABLE’s soil roughness have led to model instability (temperature oscillations). Work underway to identify and if possible resolve
  • request has been made for CABLE related ancillaries for AM3 at N216/N512 resolution and/or ANTS suite to create these.
  • restart reproducibility is commensurate with CM2 - known causes exist (%fes_cor being the key and necessary example) so this is not fully resolvable.

Additional open topics:

  • Process around release of AM3, including what documentation (science and ‘how to run’) is needed and possible.
  • CMORIsation of outputs.
  • Establishment of CABLE library

Other notes:

  • We will need to update the CASA codebase in AM3 to align with CABLE3 (there have been updates to sync ESM1.6 and CABLE3 that have occurred since AM3 branched off).
  • Suspected that the model instability/crashes relate to an odd combination of vegetation parameter values (very low LAI, high vegetation height) - points towards the need to update these ancillaries and/or enforce consistency between them inside the code [but care is needed around intersection with CASA]

To do:

  • @MartinDix To confirm exactly how the updated roughness length was implemented (there are two locations needed, likely only one was done)
  • @MartinDix @inh599 To look more closely at period of crash to see if we can confirm (narrow down) cause.
  • NRI - @ben @clairecarouge with input from @jhan - to lead design and development of workflow around the CABLE library (offline, ESM1.6 and AM3 separately first, but JAC should be also be remembered).
  • All: Review and tidy-up outstanding issues.

Next meeting: we will continue with fortnightly meetings for the moment (i.e. 30th January), but may relax to less frequently given focus on ESM1.6.

3

I am loathe to add an issue when I am trying to get rid of them BUT if this is deemed as an (reproducibility) important part of AM3 then I will add it to the STASH. My hesitation is that this alone will not give us reproducibility?

4

@Jhan It maybe better to go through the effort of adding this to STASH/restart (though whether to do that for AM3 only or both is a different question). Remember this %fes_cor issue is only a small effect acting on the first time step of each restart. I can’t guarantee that this is the only cause of the current failure of restart reproducibility, but was noting that this particular cause is known.

I’m still hoping that we can find time to rethink/assess the whole correction terms topic and see if we can find a way past them (without breaking conservation). This would simplify the syncing between offline and coupled, conceivably allow us to investigate the use of SLI, and/or allow for our soil&snow time stepping to moved out from implicit into extras (to align better with JULES).

5

Meeting notes 30/1/2025

updates

  • Investigations of temperature oscillations are ongoing. A bug (-ve sign) has been identified in the cbm() routine which may be the cause.
  • work underway to add variables to restart file - %fes_cor, %gammzz, %osnowd, and %rtsoil. This should resolve restart reproducibilty in AM3 (with current configuration - at least %ga [Penman-Monteith] would be needed to be added to ensure reproducibility across all configurations).

Other notes

  • overnight AM3 build is still functioning (no warnings).
  • CABLE library work is progressing

To do

  • cbm() bug fix to be formally lodged as an AM3 issue (@Jhan @MartinDix)
  • cbm() bug fix also to be implemented in ESM1.6
  • @Jhan add 4 variables to AM3 restart - needs care around that the existing initialisations are removed during PR process (@inh599 @ben)
  • assessment of runs with cbm() fix to determine whether oscillations/spikes have been resolved (@MartinDix)
  • @inh599 Coordinate vegetation height work with ESM1.6 vegetation distribution development work.
  • @inh599 Add issue to CABLE offline rpo around a potential move of the snow_Aging() routine which could avoid the need for osnowd to be in the restart.
6

Meeting notes 13/2/2025
updates/notes

  • -ve sign in cbm() routine has been hown to be cause of 2-time step offset. However fixing this did not resolve the increase in oscillation occurences.
  • initial work indicates that reverting to the ESM1.5 formula for rad%trad does lead to reduced oscillations - likely adopt this for AM3 provided that short testing indicates no adverse impacts on energy balance.
  • cbm() bugs will need to be implemented in ESM1.6
  • @Jhan has implemented %fes_cor and %gammzz in the restart. %rtsoil development work in progress.
  • @inh599 has investigated whether snow_aging can be moved without overly impacting model and thereby remove need to %osnowd in restart. This is highly promising - see CABLE #545
  • @ben has extended the nightly build testing to be able to trigger-on-demand to test builds (code+suite) of development branches. This is viewed as an important final step in the review process.
  • Work progressing on namelists and threading soil fractionation information through the codebases - this is necessary to make best use of the sand fraction information in the soil_thermal_fix science option with CABLE3 (ESM issue #22 and ESM issue #61))
  • We should try to co-locate analysis scripts for routine assessments of land in AM3/ESM1.6.
  • Interest from Jiachen Lu around a supported AM3-SCM suite.
  • Vegetation height work progressing slowly - @inh599 has an approach that provides a ‘reasonable’ climatology for grass/crop heights. Further consideration needed around forests and tundra.

to do

  • rtsoil restart work and linked-PRs
  • a decision to proceed with the move of snow_aging will necessitate revisions to cbm() in the coupled models; this also impacts whether %osnowd needs to be the AM3 restart.
  • @clairecarouge to reach out to evaluation team around how best to use MOPPeR for purposes of linking AM3/ESM1.6 runs to iLAMB.
  • @MartinDix looking to run a short AM3 test where variables are initialised to unphysical values or NaNs (not 0.0) to check for use before setting. There is a more general decision for CABLE around whether to enforce/revise how vars should be first filled.

For further thought
Documentation: Technical documentation of AM3/CM3 and its development is now a routine part of the process. However, the documentation of the science is lacking. What should such documentation look like? Where should it reside? What is it’s scope? How is responsible for writing and managing?

7

Update for ILAMB:

Instead of looking into MOPPeR, we will investigate the ILAMB pre-processor that Rhaegar built for raw ESM1.5 model output. It is likely to need some modifications for ESM1.6 but should be better and faster than MOPPeR for our purpose.

8

Meeting notes 27/2/2025

updates/notes

  • rtsoil restart work ready for 3-way (JULES, UM, and suite) testing (@ben)
  • snow_aging() routine has been moved successfully in CABLE3 MAIN, AM3 and ESM1.6
  • soil values threaded through, and _vec forms fully initialised so can be used.
  • PR #250 - correction of the potential ET diagnostic, %epot - we’re looking to implement the fully correct version (as per the dev branch) in all versions of CABLE (and avoid breaking the sync’ing of the repos). In offline CABLE3 this should be bitwise reproducible with MAIN, but in AM3 and ESM1.6 this impacts one of the coupling variables %wetfac_cs. Need short run tests to check that this is okay. (@Jhan)
  • automatic conversion of AM3 output to netcdf format is on the to do list of the NRI - but delay until current changes to the code base/suite are implemented.
  • work to assess/update the default canopy height ancillary has not really progressed.
  • As noted from last time - we’re looking to potential use Rhaegar’s scripts as a pre-processor to convert ESM1.6/AM3 output into a format suitable for iLAMB. In parallel @ben to investigate how to improve the efficiency of the MOPPeR process.
  • @ben has been developing the script to facilitate the great reshuffle - a necessary step towards a CABLE library. A schematic of the proposed end point would be useful as a guide to help other contribute to that development process.
  • NRI are considering whether to progress AM3 to a formal alpha or beta release. Elements to consider - i) maturity of codebase (will this turn into a barrier to future development), ii) known issues (e.g. low dust, odd default ancillaries), iii) which configurations included in release (global N96, global N216, regional, SCM etc.), and iv) what level of documentation would be required, noting that this needs to comprise multiple aspects (technical and science).

To do ESM

  • @MartinDix to undertake a short ESM run with hourly radiation steps to check our hypothesis that the energy balance issue there is largely a result of the 3-hourly call to the radiation. Dependent on that we may need to think about how to restructure the code so that the albedo is only evaluated as part of the radiation call (or live with the issue).