ACCESS-ESM1.5: ACCESS-NRI flagship release plans

Earth System Earth System Model
,
1

Introduction

ACCESS-NRI is targeting mid to late May for an NRI supported ACCESS-ESM1.5 flagship release.

A flagship release needs to meet some minimum criteria:

  • Source code under version control on GitHub
  • Reproducible build infrastructure (all models built with spack)
  • Automated build infrastructure (CI) in place to ensure code correctness
  • Experiment configurations under version control on GitHub (e.g. ACCESS-OM2)
  • Automated reproducibility checking in place. Both for CI testing of experiment configuration updates, but also scheduled reproducibility checking of specific experiment configuration versions marked for long-term support
  • User documentation

Additionally there should be

  • Full experiment provenance
  • Model documentation

This is a more difficult task, and so will be an on-going process of improvement

Source code

ACCESS-ESM1.5 contains four major components:

  • UMv7: atmosphere model
  • CABLE2: Land surface model
  • MOM4: ocean model
  • CICE4: sea-ice model
  • OASIS3: coupler

Some other important dependencies include gcom, fcm and netCDF.

CICE4

Currently available on the ESM_1.5 branch in the cice4 repo on the ACCESS-NRI GitHub organisation:

MOM5

MOM5 version is currently in a separate COSIMA repository

UMv7

Access to the the Unified Model (UM) is restricted by a UK Met Office license, which limits where it can be stored, and who can access it.

CABLE2

Version 2.4 is included as part of the UM source code

OASIS3-mct

This is available on the ACCESS-NRI GitHub organisation

Build infrastructure

There are existing spack packages for oasis3-mct, gcom and fcm.

Additionally there are spack packages for cice5 and mom5, which should make creating spack packages for the closely related cice4 and mom4 relatively straightforward.

Build CI

Adding build CI should be relatively straightforward. The Model Release team have created reusable CI workflows and actions that can be used by any model that has a spack build package. So this steps is blocked until the build infrastructure is complete.

Experiment Configurations

Initially ACCESS-NRI plans to release and support three ACCESS-ESM1.5 experiment configurations: pre-industrial (PI), concentration driven historical (Hist-c) and an emissions driven historical (Hist-e).

The intention is to adopt, and where required modify, the CLEX CMS developed payu configurations (PI and Hist-c):

and create a Hist-e configuration by adapting the Hist-c configuration with input from emissions driven configurations developed by CSIRO (predominantly @tiloz).

Pre-industrial

Historical Concentration Driven

Historical Emissions Driven

Reproducibility testing

Reproducibility testing is still being finalised for the ACCESS-OM2 experiment configurations, but this will be adapted to be as modular as possible for use with ACCESS-ESM1.5 and other models.

2

Is this thread the best place to discuss the Spack packages for the ACCESS-ESM build, or should I create a new topic?

3

Or should I move the discussion to the issue: Spack package for ACCESS-ESM1.5 · Issue #59 · ACCESS-NRI/spack-packages · GitHub ?

4

Yes I think that sounds best. The intention was to make this a “super topic” and link to other activities/topics.

5

Hi @Aidan,
I’m pretty sure ACCESS ESM1.5 uses MOM5.1, as per the pre-industrial git repository that is available and discussed in a recent thread.
Otherwise, great to hear you are planning a new release with additional provenance and reproducibility.
Regards,
David

1 Like
6

Yes you’re right, thanks. It was a mistake on my part. I have updated it and pointed to the GitHub repo where the current version of the source code resides.

7

@tiloz mentioned at the ESM Working Group meeting that it would be good to have an emissions driven Pre-Industrial (PI) control to match the emissions driven historical that is planned.

@tiloz also pointed out there are payu based experiments developed by others in the community. It would be good to have an accounting of all that is currently available in case there are some that would be good, and simple, candidates for the ACCESS-NRI to adopt and support.

8

Hi @aidan et al,

Following from a lunchtime discussion today and also our WG meeting:

A notable issue with the current setup of ACCESS-ESM1.5 is that it churns too much raw data relative to what’s useful to be saved. For instance, the atmosphere saves out a restart file every month… one every 10 years is usually plenty. It also saves both monthly and daily data by default - many current users (like myself and others in paleo) will delete the daily data without even looking at it. It also saves atmosphere output in double precision, without compression.

Also in the ocean, the monthly diagnostics are rather extensive, saving lots of 3D fields at monthly resolution, that could easily be made more compact by switching to annual outputs for most things other than surface ocean fields which are the ones that vary most on seasonal timescales.

While some tools exist to help with this, such as the excellent ACCESS Archiver tool, I think a fresh release of ACCESS-ESM1.5 could go much further in helping to enable users to keep their outputs more compact. A “compact diagnostic” option that new users could select would be helpful. In such an option I would suggest using the payu workflow to automatically enable:

  1. Conversion of atmosphere files to netcdf
  2. Change the STASHC file to get rid of atmosphere daily outputs (i.e. don’t generate them in the first place)
  3. Shrink down the ocean outputs to mostly annual frequency for the purposes of spinup
  4. Automatically remove monthly restarts

The ACCESS-Archiver could be repurposed into payu for (1). I could share some examples for (2) and (3). (4) could be done via a postscript to run through payu.

Cheers, Dave

3 Likes
9

I don’t want to derail the discussion from David’s request but is it time to update the first post on the intended timeline? The first sentence currently reads:

ACCESS-NRI is targeting mid to late March for an NRI supported ACCESS-ESM1.5 flagship release.

1 Like
10

All excellent suggestions @dkhutch.

I believe @MartinDix has already fixed for the problem with the monthly restarts being archived, and that is now available on gadi.

I agree automated post-processing (netCDF conversion) is a must.

I agree that the model outputs waaaay too much data by default. The access-om2 models have multiple configurations in different branches in the configs repo

Using the same approach we can support different versions of configurations for different purposes. Or maybe we could have a tool/script that does a wholesale conversion of diagnostics from one mode to another. I’m not too familiar with what would be required for that.

And please do share your configs and what you’ve found to be useful settings for outputs etc in a paleo context.

1 Like
11

Hi @aidan,
That is really great to hear that the new version of payu has fixed the monthly restart problem. And also about the automation of archiving to netcdf. In regards to my personal changes to diagnostics: My edited STASHC with no daily outputs can be found here:

/home/157/dkh157/ACCESS/mio_v4/atmosphere/STASHC

Here is a modified version of my ocean diagnostic outputs, trimmed down to mostly annual outputs with just a few 2D fields saved in monthly frequency:

/home/157/dkh157/ACCESS/mio_v4/ocean/diag_table

(Note, I’m not saving any biogeochemistry!! This is not going to suit everybody.) Let me know if you can’t access those files.

One year of output using these ocean diagnostics is only 228 MB. The original ocean_month.nc for a year’s output is 4.9 GB. Getting rid of daily atmosphere outputs shrinks the atmosphere data by about a factor of 3x.

Cheers, Dave

1 Like
12

Thanks @dkhutch. Are they available in a GitHub repository?

13

Erm… not currently. The trouble is my full collection of input files is too large for a github repo. I could make a zenodo repo, but I’m inclined to do that later when I’ve finished tinkering with settings.

I could make a mini repo just for the diagnostic input changes, if this is important to do.

14

We definitely want to encourage everyone in the community to make the most of GitHub, to make our science as open as discoverable as possible. So I’m interested in finding out the technical reasons why this is difficult, but I fear we are somewhat off-topic. So I have made another topic. I you had the time I’d be interested to hear your experience

15

Hi @aidan, I put the diagnostic files I mentioned in a repo here.
(I did this prior to seeing your last post.)

Hmmm… I never quite got my head around payu’s github integration. This is because I am not very competent with github. I would be curious to learn, and will read the post you have linked.

1 Like