CMIP7 Hackathon: General diagnostics & Coupled

Hi everyone,

A few points about the ACCESS-NRI CMIP7 hackathon. Here are three important topics we discussed:

  • ESMValTool tasks: things that needs to be addressed to improve our workflow.
  • Sanity checks: parameters to monitor to quickly detect when something is wrong.
  • Evaluation: checking how the new configuration compares to references (analysis/reanalysis and/or previous configurations).

Here are editable tables concerning these three topics.
Feel free to edit to modify or add tasks/checks/metrics.
Do not delete existing lines, rather upgrade / downgrade the priority:
1 = upmost priority (needs to be done quickly)
2 = essential (needed for the development of the model)
3 = low importance (we’ll do it when we have time)
4 = not important (it was nice to think about it but we’ll most likely never do it)

ESMValTool tasks Detailed description Priority
Use native ACCESS-output in ESMValTool capacity to analyse ACCESS output without CMORization 1
ERA5 we could not use ERA5 during hackathon 1
Simple ESMValTool standard recipes we need very simple plug and play recipes for which we only need to change input variable, input simulation, input region [current ipcc recipes may be a good starting point but they are all in ncl] 2
Running ESMValTool from a jupyter notebook important for testing or on a research basis 3
Sanity checks Detailed description Priority
Earth energy balance incoming (rsdt) minus outgoing (rlut + rsut) energy 1
Earth water conservation global mean sea level 1
Earth surface temperature global mean surface temperature 1
Earth surface fluxes net surface fluxes (rsds - rsus + rlds - rlus - hfls - hfss) 2
Earth surface moisture budget water flux balance (pr - evspsbl - delta[mrtws] = 0) 2
Evaluation Detailed description Reference Priority Timeframe
standard recipe Taylor diagram see task comparison with old model, other models, obs to see how new runs look different 1 asap
standard recipe maps see tasks; (can be part of the taylor diagram recipe) as above 1 asap
Climate sensitivity 1 when model stable
monsoon seasonal cycle rainfall seasonality in SH summer monsoon regions (~Brazil, South Africa, North Australia) GPCP 2 when ready
ITF transport ? 2 when ready
modes of varibility (ENSO, IOD, IPO, SAM) seasonal cycle ERSSTv5, HadISST 2 when ready
modes of varibility amplitude as above 3 when ready
modes of varibility spectrum as above 3 when ready
MJO East/West power ratio 3 when ready
ENSO processes/feedback 3 long simulations / large ensembles needed
Tropical cyclones 3 when model stable
Position of Jet / Rossby waves 3 long simulations / large ensembles needed

We think that for a proper evaluation of modes of variability a long simulation (or a large ensemble) is needed. However, it is still important to evaluate them as we develop the model by comparing them to a long simulation (or a large ensemble) computed with a previous configuration of the model.


An example has been uploaded at:


This example was generated using recipe_perfmetrics_CMIP5.yml. However, the original script can only handle historical runs. I have used ACCESS-ESM1-5 and CESM2 control as references to replace the observation/reanalysis datasets. The base year spans from model year 1060 to 1062. The comparison data consists of recent Dave’s PPE 19 experiments, specifically the present-day control run.

This is mainly from a technical standpoint. We are currently working on fixing technical issues, primarily related to NCL. The output plots are summarized in an index.html web format. The main progress is that we can demonstrate multi-model comparisons (mostly atmospheric and land variables) using RMSD metrics and Taylor diagrams.

Special thanks to Felicity Chun and Rheager.