ACCESS-OM2 Control Experiments

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

ACCESS-OM2 Control Experiments

Introduction

COSIMA has done a number of control experiments and made them available to the COSIMA community. This saves resources, enables research and provides a common basis for collaboration.

For the control experiments COSIMA has a suite of three resolutions with the same model components, global 1.0, 0.25 and 0.1 degree nominal grid spacing.

The models are forced with a data atmosphere from a modern-day reanalysis. In all cases described here the forcing product is JRA55-do, though the versions of the product do vary.

Broadly there are two major types of control experiment: inter-annnual forcing (IAF) and repeat-year forcing (RYF).

The IAF experiments use all years available in the data product, so typically 1958-2018, and cycle the forcing a number of times (following the OMIP-2 protocol; see Griffies et al. 2016 and Tsujino et al. 2020) until the model reaches equilibrium taking into account resource constraints.

Downsides of the IAF approach is that the system undergoes a large shock every time the forcing goes from present day to 1958, and can take up to 20 years for ocean heat content to stabilise to the shock.

RYF uses the same single 12-month period (generally not a calendar year) from the forcing product and repeats it over and over. Years are chosen based on the neutrality of major climate modes of variability, as described in Stewart et al. 2020. RYF has the benefit of much smaller (but more frequent) shocks, and can be spun up “indefinitely”, but does not contain an increasing global warming signal. There is no interannual variability in RYF experiments, making them good candidates for perturbation experiments. The RYF experiments below use 1 May 1990 - 30 April 1991 repeat-year forcing (RYF9091), so a forcing jump occurs at midnight of 30 April every year.

The RYF control experiments use older configurations that are very close to those described in Kiss et al. (2020). The IAF configurations are newer than the RYF ones; they are based on those described in Kiss et al. (2020) but have numerous improvements as described by Solodoch et al. (2022), Hayashida et al. (2023), Menviel et al. (2023) and Wang et al. (2023). Further details can be found in the draft ACCESS-OM2 technical report, which is a little out of date and incomplete but still useful. The github repos for the control experiments show you exactly how they were configured.

These datasets have underpinned many papers, listed here. We recommend using the COSIMA Cookbook to access and analyse this data, which is all catalogued in the default cookbook database. A good place to start is the data explorer, which will give an overview of the data available.

Notes:

1. Exe version relates to the version of the codebase, e.g. 97e3429:d3e8bdf:1bb8904 are the short git hashes for the ACCESS-OM2 components: fms, cice5, libaccessom2, which are incorporated into the binary pathname, e.g. fms_ACCESS-OM_97e3429_libaccessom2_1bb8904.x, cice_auscom_360x300_24p_d3e8bdf_libaccessom2_1bb8904.exe
2. Inputs is the version of the input data used and is the name of the sub-directory in /g/data/ik11/inputs/access-om2/, e.g. input_20200530 refers to /g/data/ik11/inputs/access-om2/input_20200530

1 degree

RYF

Name:	1deg_jra55_ryf9091_gadi
GitHub repo:	GitHub - rmholmes/1deg_jra55_ryf at base
Exe versions:	97e3429:d3e8bdf:1bb8904
Inputs:	input_20200530
Outputs:	/g/data/ik11/outputs/access-om2/
Restarts:	/g/data/ik11/outputs/access-om2/
Brief description:	1 degree ACCESS-OM2 physics only global configuration with JRA55-do v1.3 RYF9091 Repeat Year Forcing. 780+ years of spinup from WOA. See metadata.yaml in the data directories for more information.
Simulation started:	December 2019 (all run on Gadi)
People:	Ryan Holmes (@rmholmes)
Linked articles:

IAF

Name:	1deg_jra55_iaf_omip2_cycle[1-6] (replace [1-6] with cycle of choice)
GitHub repo:	GitHub - hakaseh/1deg_jra55_iaf at omip2
Outputs:	/g/data/ik11/outputs/access-om2/; also via OMIP on ESGF (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2/omip2/r1i1p1f1)
Restarts:	/g/data/ik11/outputs/access-om2/
Brief description:	1 degree ACCESS-OM2-BGC global configuration with JRA55-do v1.4 IAF Interannual Forcing. Six 61-year cycles of 1 Jan 1958 to 1 Jan 2019 following the OMIP-2 protocol. Run for and included in CMIP6’s Ocean Model Intercomparison Program (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2/omip2/r1i1p1f1). See metadata.yaml in the data directories for more information.
Simulation started:	late 2020
People:	Hakase Hayashida
Linked articles:	Mackallah, C. et al. (2022): ACCESS Datasets for CMIP6: Methodology & Idealised Experiments. Journal of Southern Hemisphere Earth Systems Science .

Name:	1deg_jra55_iaf_omip2spunup_cycle[1-45] (replace [1-45] with cycle of choice)
GitHub repo:	GitHub - hakaseh/1deg_jra55_iaf at omip2-spunup
Outputs:	/g/data/ik11/outputs/access-om2/; also via OMIP on ESGF (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2/omip2-spunup/r1i1p1f1)
Restarts:	/g/data/ik11/outputs/access-om2/
Brief description:	1 degree ACCESS-OM2-BGC global configuration with JRA55-do v1.4 IAF Interannual Forcing. 45 61-year cycles of 1 Jan 1958 to 1 Jan 2019 following the OMIP-2 protocol for OMIP-2 spunup. Run for and included in CMIP6’s Ocean Model Intercomparison Program (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2/omip2-spunup/r1i1p1f1). See metadata.yaml in the data directories for more information. *NOTE: This uses an older version of initial conditions and topography than 1deg_jra55_iaf_omip2_cycle[1-6]
Simulation started:	early 2021
People:	Hakase Hayashida
Linked articles:	Mackallah, C. et al. (2022): ACCESS Datasets for CMIP6: Methodology & Idealised Experiments. Journal of Southern Hemisphere Earth Systems Science .

0.25 degree

RYF

Name:	025deg_jra55_ryf9091_gadi
GitHub repo:	GitHub - rmholmes/025deg_jra55_ryf at ryf9091_gadi
Exe versions:	97e3429:d3e8bdf:1bb8904
Inputs:	input_20200530
Outputs:	/g/data/ik11/outputs/access-om2-025/
Restarts:	/g/data/ik11/outputs/access-om2-025/
Brief description:	0.25 degree ACCESS-OM2 physics only global configuration with JRA55-do v1.3 RYF9091 Repeat Year Forcing. 600+ years of spinup from WOA. Ideal for perturbation runs. See metadata.yaml in the data directories for more information.
Simulation started:	December 2019 (run all on Gadi)
People:	Ryan Holmes (@rmholmes)
Linked articles:	Holmes, R. M. , Zika, J.D., Griffies, S.M., Hogg, A. McC., Kiss, A.E. and England, M.H. (2021): The geography of numerical mixing in a suite of global ocean models. Journal of Advances in Modeling Earth Systems , 13 (7), e2020MS002333. (might be an earlier one?)

IAF

A	B
Name:	025deg_jra55_iaf_omip2_cycle[1-6] (replace [1-6] with cycle of choice).
	Extension of last cycle through to end of 2023: 025deg_jra55_iaf_omip2_cycle6_jra55v150_extension
GitHub repo:	omip_amoctopo_cycle[1-5] and omip_amoctopo_cycle6_redo_mon_budget branches of GitHub - rmholmes/025deg_jra55_iaf: One quarter degree ACCESS-OM2 experiment using JRA55-do IAF forcing.
Exe versions:	e837d05d:597e4561:4198e150
Inputs:	input_236a3011
Outputs:	/g/data/ik11/outputs/access-om2-025/; also via OMIP on ESGF (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2-025/omip2/r1i1p1f1)
Restarts:	/g/data/ik11/outputs/access-om2-025/
Brief description:	0.25 degree ACCESS-OM2 physics only global configuration with JRA55-do v1.4 IAF Interannual Forcing. Six 61-year cycles of 1 Jan 1958 to 1 Jan 2019 following the OMIP-2 protocol. Run for and included in CMIP6’s Ocean Model Intercomparison Program (CMIP6/OMIP/CSIRO-COSIMA/ACCESS-OM2-025/omip2/r1i1p1f1). See metadata.yaml in the data directories for more information. NOTE: An extension of the last cycle through to the end of 2023 using JRA55 v1.5.0.1 forcing is available under 025deg_jra55_iaf_omip2_cycle6_jra55v150_extension. Configuration files at GitHub - rmholmes/025deg_jra55_iaf at omip_amoctopo_cycle6_redo_mon_budget_jra55v150_extension
Simulation started:	early 2021
People:	Ryan Holmes (@rmholmes)
Linked articles:	Mackallah, C. et al. (2022): ACCESS Datasets for CMIP6: Methodology & Idealised Experiments. Journal of Southern Hemisphere Earth Systems Science .

0.1 degree

RYF

Name:	01deg_jra55v13_ryf9091
GitHub repo:	GitHub - COSIMA/01deg_jra55_ryf at ryf9091-on-gadi
Exe versions:	97e3429:d3e8bdf:1bb8904
Inputs:	input_08022019
Outputs:	/g/data/ik11/outputs/access-om2-01/01deg_jra55v13_ryf9091
Restarts:	/g/data/ik11/outputs/access-om2-01/01deg_jra55v13_ryf9091
Brief description:	0.1 degree ACCESS-OM2 global model configuration under JRA55-do v1.3 RYF9091 Repeat Year Forcing. Forcing strategy outlined by Stewart et al. (2020). The configuration is based on that described in Kiss et al. (2020), but with many improvements. Initial conditions are WOA13v2 potential temperature and practical salinity. The JRA55-do v1.3 RYF9091 atmospheric forcing runs from 1 May 1990 to 30 April 1991. The simulation starts from a nominal year of 1 Jan 1900 and runs to 31 Dec 2179. See metadata.yaml in the data directories for more information.
Simulation started:	2019
People	Andy Hogg (@AndyHoggANU)
Linked articles:	Many of the papers listed here.

IAF

Name:	01deg_jra55v140_iaf (first cycle with v1.4 forcing)
	01deg_jra55v140_iaf_cycle2 (second cycle with v1.4 forcing)
	01deg_jra55v140_iaf_cycle3 (third cycle with v1.4 forcing)
	01deg_jra55v140_iaf_cycle4 (fourth cycle with v1.4 forcing and WOMBAT BGC)
	01deg_jra55v140_iaf_cycle4_jra55v150_extension (extension of cycle 4 to present day with WOMBAT BGC using v1.5.0 and v1.5.0.1 forcing)
	01deg_jra55v150_iaf_cycle1 (first cycle with v1.5.0 forcing; limited output - run was to generate data and restarts for forcing regional models and running DA systems)
GitHub repo:	These are all in separate branches on GitHub - COSIMA/01deg_jra55_iaf; use the links in the names above to see the relevant branch.
Outputs:	/g/data/ik11/outputs/access-om2-01/
	/g/data/cj50/access-om2/raw-output/access-om2-01/
	a large subset is also publicly available via the NCI Data Catalogue
Restarts:	/g/data/ik11/restarts/access-om2-01/
Description:	0.1 degree ACCESS-OM2 global model configuration with interannual forcing, looped over 4 cycles following the OMIP-2 protocol. The configuration is based on that described in Kiss et al. (2020), but with many improvements that are detailed in the Supplementary Materials of Solodoch et al. 2022. Cycle 4 and its extension include WOMBAT biogeochemistry as described by Hayashida et al. (2023) and Menviel et al. (2023). See metadata.yaml in the data directories for more information. Further information, errata, and a list of outputs are provided here and there’s also some information here.
Simulation started:	mid-2020
People:	Andrew Kiss (@aekiss), Adele Morrison (@adele157)
Linked articles:	Many of the papers listed here.

Other ACCESS-OM2 projects and simulations

A list of projects using ACCESS-OM2 simulations, data and code can be found here

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

Awesome, thanks for filling in all those blanks @rmholmes. I had left out People and dates, which are important, so thanks.

@Aidan

GitHub - rmholmes/1deg_jra55_ryf at base, see comment

I’m afraid I don’t understand, even after reading the comment on that commit.

With the two 1 deg RYF spin-ups, I guess the motivation for doing 1deg_jra55v14_ryf was to use the newer JRA55 RYF. Did it make much difference? If not, is there any point in having that as an official reference run as well as the much longer 1deg_jra55_ryf9091_gadi? Is this an example where the change in code meant the older RYF run no longer works correctly for the latest code versions?

[rmholmes (Ryan Holmes)]

Yes that comment wasn’t clear. I’ve removed.

I think the issue with this run was that there was a change in the input file locations (from input_rc to input_20200530 I guess, from reading the git history) and when I changed to the new location there were a few input file changes marked in the manifests (see Update input_rc to input_20200530 for reproduction. Note: 1 change is… · rmholmes/1deg_jra55_ryf@21fa920 · GitHub). This would be a good case for testing reproducibility.

With regards to the two RYF spinups - I just provided the second one because it’s a more up-to-date version. Yes the spinup is shorter but it’s the 1-degree so wouldn’t cost much to get it up to the same point. Maybe I should just do this?

@Aidan

no longer works correctly for the latest code versions?

My theory has always been to use the same executable as used for the original run (or if you want to add any specific code changes, add those specifically starting from the original code commit and recompile). Otherwise you’re running the risk of something changing that you didn’t intend no?

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

@rmholmes

I think the issue with this run was that there was a change in the input file locations (from input_rc to input_20200530 I guess, from reading the git history) and when I changed to the new location there were a few input file changes marked in the manifests (see Update input_rc to input_20200530 for reproduction. Note: 1 change is… · rmholmes/1deg_jra55_ryf@21fa920 · GitHub). This would be a good case for testing reproducibility.

You can ignore any changes that are just the binhash checksum, that is just a change-detecting hash that is sensitive to path and modification time. If it changes the md5 checksum is also recalculated and if it shows no change then the file is identical.

The one change that is substantive is to the cice executable.

The difference is this

Comparing...

which was reverting a previous change that introduced a bug, see:

floating invalid and atm and ice out of sync in IAF run

@rmholmes

With regards to the two RYF spinups - I just provided the second one because it’s a more up-to-date version. Yes the spinup is shorter but it’s the 1-degree so wouldn’t cost much to get it up to the same point. Maybe I should just do this?

NO! Sorry … maybe not. We’re looking for runs that it would useful to re-do, so we can do it with all the provenance and reproducibility checks etc. So it would be better if we targeted that as one of them.

Speaking of cheap and running new versions JRA55-do is now up to v1.5.0.1, so if it is being re-run, it would probably be best to use the most recent version. And speaking of which, if there is a more recent version, do we still need the v1.3 run included in this category for forking off new experiments? I would prefer if there was just a single run in each category unless there was a very good reason.

@rmholmes

My theory has always been to use the same executable as used for the original run (or if you want to add any specific code changes, add those specifically starting from the original code commit and recompile). Otherwise you’re running the risk of something changing that you didn’t intend no?

Absolutely. Possibly best to ignore my comment above. I was trying to understand why the new run was done, and should have just asked one question, the second was a red herring and not useful.

[rmholmes (Ryan Holmes)]

@Aidan

The one change that is substantive is to the cice executable.

Ok thanks. I think there was also a change in i2o.nc, I’m not sure if that was important.

@Aidan

Speaking of cheap and running new versions JRA55-do is now up to v1.5.0.1, so if it is being re-run, it would probably be best to use the most recent version. And speaking of which, if there is a more recent version, do we still need the v1.3 run included in this category for forking off new experiments? I would prefer if there was just a single run in each category unless there was a very good reason.

Yes I agree that it would be best to have one case in each section (except for the major CMIP runs such as Hakase’s omip2 and omip2spunup). I think this would be a great (cheap) case to run to test your systems. For now I’ve removed the v1.4 case from the wiki.

[PSpence (Paul Spence)]

For the greater good - we could use an 01-RYF-BGC control run. I am working on it. I am trying to use the ~200 year RYF restarts ocean sea ice restarts and start the BGC with year=0 initial conditions.

Query: Are there *nml toggles to do this (i.e. reading in ocean restart files and Wombat BGC initial conditions from input files)? Or do I need to hack the BGC into the restart files?

Thank you,
Paul

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

@PSpence

For the greater good - we could use an 01-RYF-BGC control run. I am working on it. I am trying to use the ~200 year RYF restarts ocean sea ice restarts and start the BGC with year=0 initial conditions.

I would strongly suggest that this should be discussed in a COSIMA Meeting before you do a lot of work and burn significant compute resources. If this is something the community needs then it might well make more sense for ACCESS-NRI to be responsible for generating and supporting this control run.

[PSpence (Paul Spence)]

Thanks for the feedback Aidan. We’ve discussed the 01-RYF-bgc idea at a few meetings (e.g. this years COSIMA meeting), but I agree that it is an expensive run and priorities often change. The main advantage is that it allows for shorter, more isolated perturbation experiments that can better focus on mesoscale bgc dynamics.

I did a 200yr 025-ryf-bgc simulation (not initiated from restarts) last year. Currently, I am just trying to play around a bit with how to set it up at 01. Perhaps over time maybe we can use the fleeting computing scraps that come up to get a few decades of 01-bgc-ryf control state. I’m seeking some advice advice on the engineering required for its initialization.

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

@PSpence

Currently, I am just trying to play around a bit with how to set it up at 01. Perhaps over time maybe we can use the fleeting computing scraps that come up to get a few decades of 01-bgc-ryf control state. I’m seeking some advice advice on the engineering required for its initialization.

My point is that if this is something the ACCESS-NRI takes on as a released product, i.e. runs the experiment and publishes it, then we will be able to support more effectively than if it is an experiment conducted by you, or someone else. This is because we will have control over how it is run and the metadata that is collected, the provenance that is included etc.

[PSpence (Paul Spence)]

ACCESS-NRI support would be awesome :). Regarding initializing the BGC, I’m thinking that using the cycle 4 IAF bgc restarts from 1990 is a good option.

[rmholmes (Ryan Holmes)]

@AndyHoggANU can you confirm that the config version that you used for the 01deg_jra55v13_ryf9091 run is the branch contained at GitHub - COSIMA/01deg_jra55_ryf at 01deg_jra55v13_ryf9091? It would be worth pushing up the latest version of this configuration in your home directory to the github repo (I can see the directory, but I can’t check the git stuff because of permissions).

@aekiss the same would be great for the 01deg_jra55v140_iaf run/s (i.e. add the appropriate github repo to the wiki above). Thanks!

[AndyHoggANU (Andy Hogg)]

Done: GitHub - COSIMA/01deg_jra55_ryf at ryf9091-on-gadi

[rmholmes (Ryan Holmes)]

Thanks @AndyHoggANU! I’ve added it to the wiki.

[aekiss (Andrew Kiss)]

@rmholmes

@aekiss the same would be great for the 01deg_jra55v140_iaf run/s (i.e. add the appropriate github repo to the wiki above). Thanks!

done; also added a final paragraph in the intro section

[mauricehuguenin (Maurice F. Huguenin)]

Name:	1deg_jra55_rdf
GitHub repo:	GitHub - mauricehuguenin/1deg_jra55_iaf/PI_Offset/
Outputs:	/g/data/e14/mv7494/access-om2/archive/1deg_jra55_rdf_spinup_PI_Offset_continued, /g/data/e14/mv7494/access-om2/archive/1deg_jra55_rdf_spinup_PI_Offset_scaling and /g/data/e14/mv7494/access-om2/archive/1deg_jra55_rdf_spinup_PI_Offset_continued
Restarts:	In the same folder as outputs.
Brief description:	1910-year spin-up using repeat cycles of 1961-72 JRA55-do-1-3 forcing from which a pre-industrial offset (0.7 W/m2 in the downward longwave radiation and 0.133°C in the surface air temperatures) was removed. Relative humidity was kept constant to avoid overly impacting evaporation and the latent heat fluxes in a colder climate. Then, continuation of the spin-up while linearly reducing the offsets over 90 years to simulate the emergence of the global ocean from the pre-industrial times back to the 1960s. Lastly, repeat cycles of the 1960s forcing (with no offset) through to present times to establish a control simulation.
Simulation started:	8th April 2022
People:	Maurice Huguenin (@mauricehuguenin)
Linked articles:	Huguenin et al. (2022): Drivers and distribution of global ocean heat uptake over the last half century. Nature Communications.

[mauricehuguenin (Maurice F. Huguenin)]

I have just added some info here about the repeat decade forcing spin-up from Ryan, Matt and I. Would you recommend I put the output on /g/data/ik11? The 1° model has been used and the output from the 2000 year spin-up is in the range of about 260 GB.

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

@mauricehuguenin

I have just added some info here about the repeat year forcing spin-up from Ryan, Matt and I

Repeat decade forcing?

[mauricehuguenin (Maurice F. Huguenin)]

Yes - repeat decade forcing. Muscle memory got the better of me.

[rmholmes (Ryan Holmes)]

For those wishing to start new perturbation runs using restarts from these simulations, please see the instructions/tutorial at Tutorials · COSIMA/access-om2 Wiki · GitHub