Reproducibility tolerances for ACCESS-CM2 piControl?

I’m running a series of perturbed ensembles with ACCESS-CM2. I’m branching these from the piControl every 10 years between 950 and 1050. I had planned to use the published CMIP6 data as my control simulation, however I’ve rerun a couple of short periods of the piControl (no perturbations) on Gadi and noticed some differences with the CMIP6 data. I don’t expect the runs to be identical but they should represent the same climate state for me to be able to use the CMIP6 data as my control.

I’m new to coupled models and so don’t have a good grasp on what would be considered tolerable differences. The surface ocean and sea ice are clearly starting from the same state but the anomalies in sea ice extent become pretty large within the first year. The surface air temperature anomalies also seem quite large (to me).

Figure showing maps of SAT, SST and SSS in the first month of the piControl rerun. The top row is the CMIP6 data, the middle row is my rerun on Gadi, and the bottom row is anomalies.

Figure showing a timeseries of anomalies between my reruns (started in year 950 (blue) and 1040 (red)) and the published CMIP6 data.

My question to other Earth System Users is: Do I need to rerun the piControl on Gadi before branching my ensembles? Or are these differences considered tolerable?

Your TAS anomalies are very large, but in case you are using CMORized data for CMIP, make sure you have used the same variable for surface air temperature: tas is fld_s03i236
We have not used the CM2, but there is a lot of internal variability in the ESM1.5 and you would need to run a few decades to assess reproducibility.

The original PI control was run on the previous NCI machine raijin and is not exactly reproducible on gadi because of a newer compiler. At the time of the change I did a new 200 year control on gadi and was happy that the climate was the same.


That run was never part of the CMIP6 publication and has now been archived, but I do have some results still on gadi.

The UM is quite sensitive to perturbations (including floating point roundoff from different compilers etc) and runs diverge quickly.

Thanks @MartinDix and @LaurieM.

@LaurieM, I was using the wrong SAT variable - do you have a good resource for checking what each of the UM diagnostics are (rather than printing the metadata for each)? Here’s the updated plots. Reduced SAT biases, though they still seem considerable in the Arctic given this is the first month of the run. Maybe that size of divergence is to be expected given Martin’s comment about UM sensitivity?

950 rerun:

1040 rerun:

Monthly anomalies:

@MartinDix thanks for the info. If I check my reruns against the data that you still have on Gadi, and they reproduce it, is your position that the CMIP6 piControl publication would be fine for my control? I want to branch across 100 years of the piControl so would be looking at using 950 to 1050 as my control period.

Differences from my run c.f. CMIP6 have the same magnitude as yours but a slightly different pattern which is odd.

What’s the name of your suite?

u-cy339 is the one that I continued from year 950 for 6 months. u-dl525 is one that I continued from year 1040 for 1 year.

The output is stored here: /scratch/e14/hd4873/archive

@MartinDix , The size of the different signal over the Arctic sea ice could just be from a different phase of the Arctic Oscillation (linked to the NAO) so the pressures are different leading to different local temperature anomalies, Though its steange that it can develop such a strong anomaly in just one month.

I’ve updated my plots above. In a globally averaged sense, there is little difference in SAT, SST, SSS and sea ice, at least initially. In the 1040 rerun, the SIE winter maximum does diverge a bit from the CMIP6 data. Enough to be concerned about (given I’m interested in the sea ice response)? - I’m not sure…

I have assigned Martin as he replied, but feel free to re-assign to someone else to follow up if you don’t have time Martin.

@MartinDix just confirming the piControl rerun that you completed is no longer available for use?

We have decided to rerun 100 years of the piControl starting from 950 for the following reasons:

  1. Whilst the Gadi simulation represents the same climate as the CMIP6 run over centennial timescales, we are interested in the ~10-year sea ice response following a perturbation. We therefore want a control simulation that has been run on the same machine so that our ensembles represent the response to a perturbation only, rather than a response to a perturbation + new machine.
  2. We also want to branch 10 ensembles over the first 100 years (950 - 1050) of the piControl. We’ve chosen this period because Sea Ice Extent in CM2 is biased low and the piControl shows a steady Southern Ocean warming of around 0.3 C and a concurrent SIE reduction over the 500 years of the piControl. Additionally, the decadal restarts for 960-980 throw errors due to missing two cice variables (alvl and vlvl). So to branch from these decades we need to rerun a portion of the control anyway.

I’ve started rerunning the piControl. I’m not sure if this will be useful for others, but it will be available should anyone need. If there is an existing Gadi rerun that is still available, please let me know!

I haven’t yet got to the bottom of why my piC rerun doesn’t quite match @MartinDix’s.