CM3 Dev-Eval Working group: Meeting minutes 2025

Earth System Working Group
, , ,
1

Meeting guidelines

Below are posts for each CM3 Dev-Eval Working group meeting currently focused on CM3. Before each meeting, we’ll create a placeholder post where participants are welcome to post:

  1. discussion items;
  2. links to GitHub issues in the CM3 evaluation repository that contain Figures;
  3. figures or analysis that can then be discussed at the meeting.

The posts below are “wiki” posts mean anyone with a forum account can edit it (changes are tracked).

During the meeting, the post will be updated by a note taker such that the post forms the minutes for the meeting.

Meeting table of contents

@spencerwong role immediately following each meeting

  • Create new post in this thread with boilerplate [1];
  • make it a wiki post;
  • add it to the meeting table of contents as a link (section above);
  • rollover any content from the previous meeting that we didn’t get to.

@spencerwong role before each meeting (say Monday before)

  • announce next meeting on the ESM announce page;
  • include any current agenda items and remind people they can suggest items/figures.

@kieranricardo role before each meeting

  • emphasise / find contributions from previous meeting;
  • quick summary of workflow options for contributions;
  • summary of the last CM3 technical meeting;
  • look over GitHub cm3-paper-1 repo and see if there are any updates;
  • check in with Chris/Heidi for the status of community contributions

@kieranricardo role during meeting

  • quick summary of workflow options for contributions;
  • emphasise contributions from previous meeting;
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

[1]

CM3 Dev-Eval Working group meeting minutes

Date: INSERT_DATE at 10:30 am
Participants:
Chair: @kieranricardo
Minutes: @spencerwong

Agenda:

  • shout-out props for Figures from:
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

GitHub issue links:

GitHub issue links to OM3 figures for CM3:

Community figures

Please paste here and include a caption!

2

First CM3 Dev-Eval Working group

Date: Tuesday on the 11th November at 10:30 am. Meeting link.

Participants: @cbull, @jemmajeffree, @wghuneke, @spencerwong, @rbeucher, @tiloz, @MartinDix, @ezhilsabareesh8, @paulleopardi, @CharlesTurner, @dgwyther, @aekiss, @heidi, @ZhiLiUNSW, @RachelLaw, Sienna Blanckensee, @Harun_Rashid, @lishx, @ongqingyee, @HIMADRI_SAINI, @peterdobro, @DeepashreeDutta

Coordination: @cbull

Chair: @cbull

Minutes: @spencerwong

Apologies: @nicolamaher @ctychung @dave

Agenda:

Scheduling:

  • See this post for the schedule of ESM WG and CM3 dev-eval meetings for the rest of the year. There will be an ESM WG meeting with a science presentation on Thursday 20 November at 1pm AEDT, and a CM3 dev-eval meeting on Tuesday 9 December at 10:30 am AEDT.

CM3 technical developments

  • @MartinDix provided an update on the latest CM3 technical developments. Changes being tested include bathymetry changes to address Baltic salinity drift, and UM GC5 parameter changes. The parameter changes led to a worsening of the radiative imbalance, and tests using newer UM code will be carried out.
  • Full minutes from the last technical meeting are available here, and anyone is welcome to get involved in the meetings.

Python stats functions

  • @Harun_Rashid has added a collection of python functions for statistical analysis to the repository here. These might be useful in for anyone doing analysis, and feel free to reach out if you have questions.

Datastores for CM3 evaluation

  • @CharlesTurner provided an update on creating datastores from CM2 and CM3 output. Most of the data from the latest CM3 run has been placed in a virtualised datastore, which significantly speeds up access and analysis.
  • Virtualisation may not currently be viable for the CM2 data, and more specific workarounds will be required to optimise the speed of accessing the data.
  • @MartinDix noted plans to split CM3 atmospheric output into single variable files which may also help.

Figure discussions:

Discussion highlighted changes in variability in the maximum salinity timeseries, though this could be due to changes at individual points. It was noted that adding observations to the comparisons would be useful.

GitHub links to OM3 figures for CM3 (please include OM3 comparison links where practical) – @ezhilsabareesh8 :

  • Drake Passage Transport
    The streamfunction is too high in CM3. OM3 matches observations well, while CM3 is too high and still increasing. @ezhilsabareesh8 will update the figures to use the CM2 025 degree simulation.

    This could point to a problem in the atmosphere, either too strong westerlies or too strong coupling. It was noted that Southern ocean wind stresses appear similar between 1 degree CM2 and 025 degree CM3, and so similar wind stresses could be resulting in different ocean forcing. Plots of SSH vs latitude at the Drake passage were raised as a possible way to investigate this further.

  • Global Timeseries. These diagnostics are not quite ready yet.

  • SSS and SSS difference from WOA23
    Large SSS biases were noted in the Arctic

  • SST and SST difference from WOA23
    Large warm biases in the Southern Ocean were noted

Community figures

Rolled over to next meeting
@Harun_Rashid

today I’d like to briefly talk about the ENSO power spectrum (slide 4) and the ENSO diagnostics table (slide 5). The table compares various ENSO diagnostics computed from the latest CM3 run with those calculated from observations, ACCESS-CM2 and other CMIP6 model simulations.

The run length is 37 years, with seven years of data missing
The simulation was done in August by Kieran at ACCESS-NRI

image
image1485×821 151 KB

image
image1378×834 358 KB

Others

Please paste here and include a caption!

3

CM3 Dev-Eval Working group meeting minutes

Date: Tuesday 9 December at 10:30 am
Participants:
Chair: @kieranricardo
Minutes: @spencerwong

Agenda:

  • shout-out props for Figures from:
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

GitHub issue links:

GitHub issue links to OM3 figures for CM3:

Community figures

Harun Rashid

@Harun_Rashid

today I’d like to briefly talk about the ENSO power spectrum (slide 4) and the ENSO diagnostics table (slide 5). The table compares various ENSO diagnostics computed from the latest CM3 run with those calculated from observations, ACCESS-CM2 and other CMIP6 model simulations.

The run length is 37 years, with seven years of data missing
The simulation was done in August by Kieran at ACCESS-NRI

image
image1485×821 151 KB

image
image1378×834 358 KB

Please paste here and include a caption!

4

Meeting Summary

Meeting summary of the ACCESS-NRI internal meeting on 25/11/2025, on the managing and long term plans on the CM3/OM3/AM3 evaluation efforts.

Quick Recap

The team discussed integrating OM3, AM3 and CM3 evaluation workflows using a submodule approach, where OM3 and AM3 would be submodules in the CM3 repository. Ideally notebooks written for AM3/OM3 would also work on CM3, but not a trivial exercise.

Summary

OM3/AM3 as submodules in CM3 repository

  • Plan is to make OM3 a submodule within the CM3 repository. Prevents desync of scripts between repositories.
  • Where would AM3 sit in this? Worked well for OM3, as evaluation workflow was already in place, but AM3 does not yet have this community effort. Most likely, we would have atmosphere analysis contained in CM3 repo for now, and moved to AM3 repo when model is established.
  • How to handle inconsistencies between CM and OM/AM? Apply pre-processing step, depending on which repo was running the analysis?
  • This could be done by MOPPy, but this would be some time in future- MOPPy work focused on ESM1.6 currently.

OM3–CM3 Workflow Integration

  • There are a number of issues which make generalisation of notebooks difficult- different variable names, time axes.

Maintenance

  • How do we make it easy for users to test their scripts across models and configurations? It’s likely some work would fall to us, but ideally scripts would be quite general before being added to the repo.

AM3 / CM3 Meeting Integration

  • Agreed to trial joint meetings with AM3 participants attending CM3 sessions.
  • Considered structuring agendas so ocean discussion comes first, then atmosphere, so people can leave/show up when necessary.

Next Steps

  • AM3 Team

    • How to put AM3 outputs in a format compatible with the intake catalogue?
    • Invite 21st Century Weather team to CM3 evaluation meetings

  • OM3 Team

    • Write a clear list of OM3/CM3 notebook generalization issues
    • Use same variables for ocean outputs in OM3 and CM3 by using same diag table
1 Like