# CABLE in AM3 development notes **URL:** \ **Category:** Land Surface\ **Created:** [16 January 2025 01:22 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073 "2025-01-16T01:22:50Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [16 January 2025 01:22 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/1 "2025-01-16T01:22:50Z")
This topic is to collate meeting notes around the ongoing development of ACCESS-AM3 with a focus on role of CABLE. Key individuals: @clairecarouge @MartinDix @ben @Jhan @inh599 ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [16 January 2025 01:42 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/2 "2025-01-16T01:42:46Z")
**Meeting 16/1/2025** _Status of developments_: - AM3 is functional, proto-type AM3 codebase and suite has been shared with university colleagues. - CASA has been run using the AM3 codebase but output has not been assessed in anyway. - energy/water balance is reasonable (commensurate with CM2), still some energy imbalance over snow regions which could be followed up on at a later date. - progress to back port recent AM3 changes back into CABLE3-MAIN and ESM1.6 have resulted in a range of other concerns/technical challenges. - dust in proto-type AM3 is too low - attempts to rectify via changing (increasing) CABLE’s soil roughness have led to model instability (temperature oscillations). Work underway to identify and if possible resolve - request has been made for CABLE related ancillaries for AM3 at N216/N512 resolution and/or ANTS suite to create these. - restart reproducibility is commensurate with CM2 - known causes exist (`%fes_cor` being the key and necessary example) so this is not fully resolvable. _Additional open topics_: - Process around release of AM3, including what documentation (science and ‘how to run’) is needed and possible. - CMORIsation of outputs. - Establishment of CABLE library _Other notes_: - We will need to update the CASA codebase in AM3 to align with CABLE3 (there have been updates to sync ESM1.6 and CABLE3 that have occurred since AM3 branched off). - Suspected that the model instability/crashes relate to an odd combination of vegetation parameter values (very low LAI, high vegetation height) - points towards the need to update these ancillaries and/or enforce consistency between them inside the code [but care is needed around intersection with CASA] _To do_: - @MartinDix To confirm exactly how the updated roughness length was implemented (there are two locations needed, likely only one was done) - @MartinDix @inh599 To look more closely at period of crash to see if we can confirm (narrow down) cause. - NRI - @ben @clairecarouge with input from @jhan - to lead design and development of workflow around the CABLE library (offline, ESM1.6 and AM3 separately first, but JAC should be also be remembered). - All: Review and tidy-up outstanding issues. _Next meeting_: we will continue with fortnightly meetings for the moment (i.e. 30th January), but may relax to less frequently given focus on ESM1.6. ---
**Author:** ![Jhan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/jhan/32/721_2.png) [@Jhan](https://forum.access-hive.org.au/u/Jhan)\ **Post date:** [16 January 2025 07:36 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/3 "2025-01-16T07:36:01Z")
> [@inh599](#): > > restart reproducibility is commensurate with CM2 - known causes exist (`%fes_cor` being the key and necessary example) so this is not fully resolvable. I am loathe to add an issue when I am trying to get rid of them BUT if this is deemed as an (reproducibility) important part of AM3 then I will add it to the STASH. My hesitation is that this alone will not give us reproducibility? ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [16 January 2025 22:26 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/4 "2025-01-16T22:26:28Z")
@Jhan It maybe better to go through the effort of adding this to STASH/restart (though whether to do that for AM3 only or both is a different question). Remember this `%fes_cor` issue is only a small effect acting on the first time step of each restart. I can’t guarantee that this is the only cause of the current failure of restart reproducibility, but was noting that this particular cause is known. I’m still hoping that we can find time to rethink/assess the whole correction terms topic and see if we can find a way past them (without breaking conservation). This would simplify the syncing between offline and coupled, conceivably allow us to investigate the use of SLI, and/or allow for our soil&snow time stepping to moved out from implicit into extras (to align better with JULES). ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [30 January 2025 01:18 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/5 "2025-01-30T01:18:14Z")
**Meeting notes 30/1/2025** _updates_ - Investigations of temperature oscillations are ongoing. A bug (-ve sign) has been identified in the `cbm()` routine which may be the cause. - work underway to add variables to restart file - `%fes_cor`, `%gammzz`, `%osnowd`, and `%rtsoil`. This should resolve restart reproducibilty in AM3 (with current configuration - at least `%ga` [Penman-Monteith] would be needed to be added to ensure reproducibility across all configurations). _Other notes_ - overnight AM3 build is still functioning (no warnings). - CABLE library work is progressing _To do_ - `cbm()` bug fix to be formally lodged as an AM3 issue (@Jhan @MartinDix) - `cbm()` bug fix also to be implemented in ESM1.6 - @Jhan add 4 variables to AM3 restart - needs care around that the existing initialisations are removed during PR process (@inh599 @ben) - assessment of runs with `cbm()` fix to determine whether oscillations/spikes have been resolved (@MartinDix) - @inh599 Coordinate vegetation height work with ESM1.6 vegetation distribution development work. - @inh599 Add issue to CABLE offline rpo around a potential move of the `snow_Aging()` routine which could avoid the need for `osnowd` to be in the restart. ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [13 February 2025 03:51 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/6 "2025-02-13T03:51:43Z")
**Meeting notes 13/2/2025** _updates/notes_ - -ve sign in `cbm()` routine has been hown to be cause of 2-time step offset. However fixing this did not resolve the increase in oscillation occurences. - initial work indicates that reverting to the ESM1.5 formula for `rad%trad` does lead to reduced oscillations - likely adopt this for AM3 provided that short testing indicates no adverse impacts on energy balance. - `cbm()` bugs will need to be implemented in ESM1.6 - @Jhan has implemented `%fes_cor` and `%gammzz` in the restart. `%rtsoil` development work in progress. - @inh599 has investigated whether `snow_aging` can be moved without overly impacting model and thereby remove need to `%osnowd` in restart. This is highly promising - see [CABLE #545](https://github.com/CABLE-LSM/CABLE/pull/545) - @ben has extended the nightly build testing to be able to trigger-on-demand to test builds (code+suite) of development branches. This is viewed as an important final step in the review process. - Work progressing on namelists and threading soil fractionation information through the codebases - this is necessary to make best use of the sand fraction information in the `soil_thermal_fix` science option with CABLE3 (ESM issue #22 and ESM issue #61)) - We should try to co-locate analysis scripts for routine assessments of land in AM3/ESM1.6. - Interest from Jiachen Lu around a supported AM3-SCM suite. - Vegetation height work progressing slowly - @inh599 has an approach that provides a ‘reasonable’ climatology for grass/crop heights. Further consideration needed around forests and tundra. _to do_ - `rtsoil` restart work and linked-PRs - a decision to proceed with the move of `snow_aging` will necessitate revisions to `cbm()` in the coupled models; this also impacts whether `%osnowd` needs to be the AM3 restart. - @clairecarouge to reach out to evaluation team around how best to use MOPPeR for purposes of linking AM3/ESM1.6 runs to iLAMB. - @MartinDix looking to run a short AM3 test where variables are initialised to unphysical values or NaNs (not 0.0) to check for use before setting. There is a more general decision for CABLE around whether to enforce/revise how vars should be first filled. _For further thought_ Documentation: Technical documentation of AM3/CM3 and its development is now a routine part of the process. However, the documentation of the science is lacking. What should such documentation look like? Where should it reside? What is it’s scope? How is responsible for writing and managing? ---
**Author:** ![clairecarouge](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/clairecarouge/32/380_2.png) [@clairecarouge](https://forum.access-hive.org.au/u/clairecarouge)\ **Post date:** [13 February 2025 04:41 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/7 "2025-02-13T04:41:45Z")
Update for ILAMB: Instead of looking into MOPPeR, we will investigate the ILAMB pre-processor that Rhaegar built for raw ESM1.5 model output. It is likely to need some modifications for ESM1.6 but should be better and faster than MOPPeR for our purpose. ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [27 February 2025 03:22 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/8 "2025-02-27T03:22:40Z")
**Meeting notes 27/2/2025** _updates/notes_ - `rtsoil` restart work ready for 3-way (JULES, UM, and suite) testing (@ben) - `snow_aging()` routine has been moved successfully in CABLE3 MAIN, AM3 and ESM1.6 - soil values threaded through, and `_vec` forms fully initialised so can be used. - PR #250 - correction of the potential ET diagnostic, `%epot` - we’re looking to implement the fully correct version (as per the dev branch) in all versions of CABLE (and avoid breaking the sync’ing of the repos). In offline CABLE3 this should be bitwise reproducible with MAIN, but in AM3 and ESM1.6 this impacts one of the coupling variables `%wetfac_cs`. Need short run tests to check that this is okay. (@Jhan) - automatic conversion of AM3 output to netcdf format is on the to do list of the NRI - but delay until current changes to the code base/suite are implemented. - work to assess/update the default canopy height ancillary has not really progressed. - As noted from last time - we’re looking to potential use Rhaegar’s scripts as a pre-processor to convert ESM1.6/AM3 output into a format suitable for iLAMB. In parallel @ben to investigate how to improve the efficiency of the MOPPeR process. - @ben has been developing the script to facilitate the great reshuffle - a necessary step towards a CABLE library. A schematic of the proposed end point would be useful as a guide to help other contribute to that development process. - NRI are considering whether to progress AM3 to a formal alpha or beta release. Elements to consider - i) maturity of codebase (will this turn into a barrier to future development), ii) known issues (e.g. low dust, odd default ancillaries), iii) which configurations included in release (global N96, global N216, regional, SCM etc.), and iv) what level of documentation would be required, noting that this needs to comprise multiple aspects (technical and science). _To do ESM_ - @MartinDix to undertake a short ESM run with hourly radiation steps to check our hypothesis that the energy balance issue there is largely a result of the 3-hourly call to the radiation. Dependent on that we may need to think about how to restructure the code so that the albedo is only evaluated as part of the radiation call (or live with the issue). ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [13 March 2025 04:30 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/9 "2025-03-13T04:30:29Z")
**Meeting notes 13/3/2025** _updates/notes_ - have identified that `%wetfac` also needs to be in AM3 restart. Code and suite changes have been developed and 3-way testing underway by @ben - work on the reshuffle has been ongoing - it works however subsequent testing of the applications has produced some failures. Likely because of changes to MAIN since the reshuffle branch occurred. - it’s hard to assess AM3 developments without automatic conversion to netcdf. @MartinDix will look to add the conversion step that sits in the CMIP6-CM2 suite in the default AMIP suite. WE can then look to improve/modify that as the downstream needs (e.g. MOPPeR) become clearer. - @MartinDix’s ESM1.6 runs with hourly radiation strongly indicates that the residual energy balance issues in the ESM1.6 are because the surface albedo seen by the atmosphere and land are not in step with each other. @inh599 and @Jhan have devised a potential solution - current development occurring in ESM1.6 but the approach should be transferable across to AM3. - We’d like to return to the issue of the value of the baresoil roughness - this is currently a combination of a `PARAMETER` and hard-wired constants in the `roughness` module. The current value (1.e-4) is too small and likely a contributor to both too little dust in AM3 and a poor diurnal cycle in desert and other low vegetation regions. The optimal solution would be to extract this parameter to the cable namelist - however this is a little dangerous without first modifying the codebase to ensure that changing the parameter value won’t cause issues. - _for the record:_ Recent effort on the Ground Water model has identified that this capability takes a subtly different interpretation of CABLE’s soil moisture variables than the default `soil_snow` model. This difference will necessitate extreme care and additional code if/when we attempt to activate the code in AM3. - _for the record:_ Early discussions have occurred around the approach to get _UCLEM_ in ACCESS-AM3. A key early decision will involve whether we can allow for a separate _UCLEM_ restart (and only pass diagnostic outputs back out through the UM) or whether more substantive reorgnisation of the exsiting CABLE implementation and coupling will be needed. ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [13 March 2025 05:06 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/10 "2025-03-13T05:06:22Z")
> [@inh599](#): > > it’s hard to assess AM3 developments without automatic conversion to netcdf. @MartinDix will look to add the conversion step that sits in the CMIP6-CM2 suite in the default AMIP suite. WE can then look to improve/modify that as the downstream needs (e.g. MOPPeR) become clearer. @paulleopardi has implemented direct netCDF output from the UM in the regional suite. I’d suggest trying this in the first instance. ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [27 March 2025 04:05 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/11 "2025-03-27T04:05:02Z")
**Meeting notes 27/3/2025** _updates/notes_ - work to implement `%wetfac` in the AM3 restart is ongoing. This has been somewhat delayed because of an issue with STASH item 839 which may actually originate in the process of building the new executable. - work on reshuffle continues. - netCDF conversion is now incorporated within the default AM3 suite. - baresoil roughness work has not advanced. - ACCESS-NRI (as part of release strategy) will shortly move the AM3 repository, from the CABLE-LSM github space, across to the NRI github space. - ACCESS-NRI are developing work plan for FY25/26, including in the AM3 space. This involves a release of AM3, a release of CM3 and work in the ANTS-space to facilitate the easy use of ACCESS-AM3 at resolutions other than N96. - work to facilitate the use of the iLAMB-benchmarking from both ESM1.6 and AM3 is hopefully on a 2-3 week timeline. - there is a task to review the conversion from ACCESS outputs to CF/CMOR compliant output. Recent science changes within CABLE may require changes in the MOPPeR/AP4 step. - _for the record:_ ESM1.6 land researchers are looking to add two Australian PFTs to CABLE. This does involve code changes to CABLE3 (which will need to brought across to AM3) and may involve additional work in the AM3 configuration/suite and initialisation. ---
**Author:** ![inh599](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/inh599/32/3400_2.png) [@inh599](https://forum.access-hive.org.au/u/inh599)\ **Post date:** [10 April 2025 00:50 UTC](https://forum.access-hive.org.au/t/cable-in-am3-development-notes/4073/12 "2025-04-10T00:50:27Z")
**Meeting notes 10/4/2025** _updates/notes_ - work to get `%wetfac` into the restart is complete - @MartinDix has identified one further source of failure of restart reproducibility. The conversion from soil moisture mass (JULES) to soil moisture by volume (CABLE) is currently only done on first time step - this section introduces round off differences which breaks restart reproducibility. Potential solution is to bring/duplicate the conversion step (in `cbl_um_init_soilsnow`) into `cbl_um_update_soilsnow` routine so that it is done every timestep. - Alongside work towards release of AM3 and the CABLE library - and science developments in AM3 such as ongoing urban work - there is an emerging need to build automatic tests to check for run-to-run and restart reproducibility. - AM3 has been moved from the CABLE-LSM github repo across to ACCESS-NRI github repos. - Documentation for AM3 will be in form of a set of MarkUp documents - under github/version control. - _for info:_ Increasing interest from W21C around use of AM3 in regional, high-resolution applications (including engaging new software engineer). Important tasks in this space are 1) creation/testing of land ancillaries (note new islands) and 2) checking that the reconfiguration/initialisation produces sensible values for the CABLE state variables (note tiled soil temperature and moisture but also all the new state vars added). **Ongoing meetings** ACCESS-NRI will be commencing a series of AM3/CM3 stand-up meetings (weekly, short) to progress work towards AM3 release. To accommodate this we will cease this particular series of meetings and co-opt/extend the stand-up meetings as necessary (in the first instance expect this to be monthly).