ACCESS-ESM1.5: ACCESS-NRI flagship release plans

Earth System Earth System Model
,
1

Introduction

ACCESS-NRI is targeting mid to late May end of July for a supported ACCESS-ESM1.5 flagship release.

A flagship release needs to meet some minimum criteria:

  • Source code under version control on GitHub
  • Reproducible build infrastructure (all models built with spack)
  • Automated build infrastructure (CI) in place to ensure code correctness
  • Experiment configurations under version control on GitHub (e.g. ACCESS-OM2)
  • Automated reproducibility checking in place. Both for CI testing of experiment configuration updates, but also scheduled reproducibility checking of specific experiment configuration versions marked for long-term support
  • User documentation

Additionally there should be

  • Full experiment provenance
  • Model documentation

This is a more difficult task, and so will be an on-going process of improvement

Source code

ACCESS-ESM1.5 contains four major components:

  • UMv7: atmosphere model
  • CABLE2: Land surface model
  • MOM5: ocean model
  • CICE4: sea-ice model
  • OASIS3: coupler

Some other important dependencies include gcom, fcm and netCDF.

CICE4

Currently available on the ESM_1.5 branch in the cice4 repo on the ACCESS-NRI GitHub organisation:

MOM5

MOM5 version is currently in a separate COSIMA repository

UMv7

Access to the the Unified Model (UM) is restricted by a UK Met Office license, which limits where it can be stored, and who can access it.

CABLE2

Version 2.4 is included as part of the UM source code

OASIS3-mct

This is available on the ACCESS-NRI GitHub organisation

Build infrastructure

All components of ACCESS-ESM1.5 have been ported to use the spack build infrastructure by adding or modifying their spack package definitions. Thanks to @paulleopardi for this work, and @harshula who assisted with reviews and additional complementary updates.

There is an active pre-release deployment of ACCESS-ESM1.5 that is being tested and is close to being released

Build CI

Build CI has been added to model components that are open source and unencumbered by licensing issues. The UM and gcom components do not currently have build CI enabled as the current approach could expose compiled components to access by unlicensed persons.

Experiment Configurations

Initially ACCESS-NRI plans to release and support three ACCESS-ESM1.5 experiment configurations: pre-industrial and concentration driven historical and an emissions driven historical.

Emissions driven configurations will be part of subsequent follow-up releases of ACCESS-ESM.

The intention is to adopt, and where required modify, the CLEX CMS developed payu configurations:

Emissions driven configurations will be from configurations developed by CSIRO (predominantly @tiloz).

Configuration work is predominantly being done by @spencerwong and @MartinDix.

Pre-industrial

The pre-industrial configuration is being developed in this branch

Historical Concentration Driven

The concentration driven historical configuration is being developed in this branch

Reproducibility testing

Modular reproducibility testing developed for ACCESS-OM2 will be applied to ACCESS-ESM1.5.

2

Is this thread the best place to discuss the Spack packages for the ACCESS-ESM build, or should I create a new topic?

3

Or should I move the discussion to the issue: Spack package for ACCESS-ESM1.5 Ā· Issue #59 Ā· ACCESS-NRI/spack-packages Ā· GitHub ?

4

Yes I think that sounds best. The intention was to make this a ā€œsuper topicā€ and link to other activities/topics.

5

Hi @Aidan,
Iā€™m pretty sure ACCESS ESM1.5 uses MOM5.1, as per the pre-industrial git repository that is available and discussed in a recent thread.
Otherwise, great to hear you are planning a new release with additional provenance and reproducibility.
Regards,
David

1 Like
6

Yes youā€™re right, thanks. It was a mistake on my part. I have updated it and pointed to the GitHub repo where the current version of the source code resides.

7

@tiloz mentioned at the ESM Working Group meeting that it would be good to have an emissions driven Pre-Industrial (PI) control to match the emissions driven historical that is planned.

@tiloz also pointed out there are payu based experiments developed by others in the community. It would be good to have an accounting of all that is currently available in case there are some that would be good, and simple, candidates for the ACCESS-NRI to adopt and support.

8

Hi @aidan et al,

Following from a lunchtime discussion today and also our WG meeting:

A notable issue with the current setup of ACCESS-ESM1.5 is that it churns too much raw data relative to whatā€™s useful to be saved. For instance, the atmosphere saves out a restart file every monthā€¦ one every 10 years is usually plenty. It also saves both monthly and daily data by default - many current users (like myself and others in paleo) will delete the daily data without even looking at it. It also saves atmosphere output in double precision, without compression.

Also in the ocean, the monthly diagnostics are rather extensive, saving lots of 3D fields at monthly resolution, that could easily be made more compact by switching to annual outputs for most things other than surface ocean fields which are the ones that vary most on seasonal timescales.

While some tools exist to help with this, such as the excellent ACCESS Archiver tool, I think a fresh release of ACCESS-ESM1.5 could go much further in helping to enable users to keep their outputs more compact. A ā€œcompact diagnosticā€ option that new users could select would be helpful. In such an option I would suggest using the payu workflow to automatically enable:

  1. Conversion of atmosphere files to netcdf
  2. Change the STASHC file to get rid of atmosphere daily outputs (i.e. donā€™t generate them in the first place)
  3. Shrink down the ocean outputs to mostly annual frequency for the purposes of spinup
  4. Automatically remove monthly restarts

The ACCESS-Archiver could be repurposed into payu for (1). I could share some examples for (2) and (3). (4) could be done via a postscript to run through payu.

Cheers, Dave

3 Likes
9

I donā€™t want to derail the discussion from Davidā€™s request but is it time to update the first post on the intended timeline? The first sentence currently reads:

ACCESS-NRI is targeting mid to late March for an NRI supported ACCESS-ESM1.5 flagship release.

1 Like
10

All excellent suggestions @dkhutch.

I believe @MartinDix has already fixed for the problem with the monthly restarts being archived, and that is now available on gadi.

I agree automated post-processing (netCDF conversion) is a must.

I agree that the model outputs waaaay too much data by default. The access-om2 models have multiple configurations in different branches in the configs repo

Using the same approach we can support different versions of configurations for different purposes. Or maybe we could have a tool/script that does a wholesale conversion of diagnostics from one mode to another. Iā€™m not too familiar with what would be required for that.

And please do share your configs and what youā€™ve found to be useful settings for outputs etc in a paleo context.

1 Like
11

Hi @aidan,
That is really great to hear that the new version of payu has fixed the monthly restart problem. And also about the automation of archiving to netcdf. In regards to my personal changes to diagnostics: My edited STASHC with no daily outputs can be found here:

/home/157/dkh157/ACCESS/mio_v4/atmosphere/STASHC

Here is a modified version of my ocean diagnostic outputs, trimmed down to mostly annual outputs with just a few 2D fields saved in monthly frequency:

/home/157/dkh157/ACCESS/mio_v4/ocean/diag_table

(Note, Iā€™m not saving any biogeochemistry!! This is not going to suit everybody.) Let me know if you canā€™t access those files.

One year of output using these ocean diagnostics is only 228 MB. The original ocean_month.nc for a yearā€™s output is 4.9 GB. Getting rid of daily atmosphere outputs shrinks the atmosphere data by about a factor of 3x.

Cheers, Dave

1 Like
12

Thanks @dkhutch. Are they available in a GitHub repository?

13

Ermā€¦ not currently. The trouble is my full collection of input files is too large for a github repo. I could make a zenodo repo, but Iā€™m inclined to do that later when Iā€™ve finished tinkering with settings.

I could make a mini repo just for the diagnostic input changes, if this is important to do.

14

We definitely want to encourage everyone in the community to make the most of GitHub, to make our science as open as discoverable as possible. So Iā€™m interested in finding out the technical reasons why this is difficult, but I fear we are somewhat off-topic. So I have made another topic. I you had the time Iā€™d be interested to hear your experience

15

Hi @aidan, I put the diagnostic files I mentioned in a repo here.
(I did this prior to seeing your last post.)

Hmmmā€¦ I never quite got my head around payuā€™s github integration. This is because I am not very competent with github. I would be curious to learn, and will read the post you have linked.

1 Like
16

There is an open issue to decide on a naming scheme for the ESM1.5 configurations.

Along the the lines of

<atmosphere_resolution>_<ocean_resolution>_<scenario>+<modifier>
Field Possible values
atmosphere resolution n48, n96, n512
ocean resolution 1deg, 2deg, 025deg
scenario historical, preindustrial, pliocene, lgm, ssp585
modifiers emiss, conc

and we would value feedback from @tammasloughran and @tiloz, particularly around possible values of scenario, and if the idea of modifiers makes sense.

17

Should probably be named something like concentration-driven vs. interactive carbon cycle, because the pre-industrial simulation does not have emissions but can have an interactive carbon cycle enabled. The emissions, whether 0, constant or variable are indicated by the ā€œscenarioā€ descriptor.

1 Like
18

Possible value for modifier: no land-use change. eg. historical+noluc
However this could be implied by the scenario name also.