CM3 Dev-Eval Working group: Meeting minutes 2025

Meeting guidelines

Below are posts for each CM3 Dev-Eval Working group meeting currently focused on CM3. Before each meeting, we’ll create a placeholder post where participants are welcome to post:

  1. discussion items;
  2. links to GitHub issues in the CM3 evaluation repository that contain Figures;
  3. figures or analysis that can then be discussed at the meeting.

The posts below are “wiki” posts mean anyone with a forum account can edit it (changes are tracked).

During the meeting, the post will be updated by a note taker such that the post forms the minutes for the meeting.

Meeting table of contents

@spencerwong role immediately following each meeting

  • Create new post in this thread with boilerplate [1];
  • make it a wiki post;
  • add it to the meeting table of contents as a link (section above);
  • rollover any content from the previous meeting that we didn’t get to.

@spencerwong role before each meeting (say Monday before)

  • announce next meeting on the ESM announce page;
  • include any current agenda items and remind people they can suggest items/figures.

@kieranricardo role before each meeting

  • emphasise / find contributions from previous meeting;
  • quick summary of workflow options for contributions;
  • summary of the last CM3 technical meeting;
  • look over GitHub cm3-paper-1 repo and see if there are any updates;
  • check in with Chris/Heidi for the status of community contributions

@kieranricardo role during meeting

  • quick summary of workflow options for contributions;
  • emphasise contributions from previous meeting;
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

[1]

CM3 Dev-Eval Working group meeting minutes

Date: INSERT_DATE at 10:30 am
Participants:
Chair: @kieranricardo
Minutes: @spencerwong

Agenda:

  • shout-out props for Figures from:
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

GitHub issue links:

GitHub issue links to OM3 figures for CM3:

Community figures

Please paste here and include a caption!

First CM3 Dev-Eval Working group

Date: Tuesday on the 11th November at 10:30 am. Meeting link.

Participants: @cbull, @jemmajeffree, @wghuneke, @spencerwong, @rbeucher, @tiloz, @MartinDix, @ezhilsabareesh8, @paulleopardi, @CharlesTurner, @dgwyther, @aekiss, @heidi, @ZhiLiUNSW, @RachelLaw, Sienna Blanckensee, @Harun_Rashid, @lishx, @ongqingyee, @HIMADRI_SAINI, @peterdobro, @DeepashreeDutta

Coordination: @cbull

Chair: @cbull

Minutes: @spencerwong

Apologies: @nicolamaher @ctychung @dave

Agenda:

Scheduling:

  • See this post for the schedule of ESM WG and CM3 dev-eval meetings for the rest of the year. There will be an ESM WG meeting with a science presentation on Thursday 20 November at 1pm AEDT, and a CM3 dev-eval meeting on Tuesday 9 December at 10:30 am AEDT.

CM3 technical developments

  • @MartinDix provided an update on the latest CM3 technical developments. Changes being tested include bathymetry changes to address Baltic salinity drift, and UM GC5 parameter changes. The parameter changes led to a worsening of the radiative imbalance, and tests using newer UM code will be carried out.
  • Full minutes from the last technical meeting are available here, and anyone is welcome to get involved in the meetings.

Python stats functions

  • @Harun_Rashid has added a collection of python functions for statistical analysis to the repository here. These might be useful in for anyone doing analysis, and feel free to reach out if you have questions.

Datastores for CM3 evaluation

  • @CharlesTurner provided an update on creating datastores from CM2 and CM3 output. Most of the data from the latest CM3 run has been placed in a virtualised datastore, which significantly speeds up access and analysis.
  • Virtualisation may not currently be viable for the CM2 data, and more specific workarounds will be required to optimise the speed of accessing the data.
  • @MartinDix noted plans to split CM3 atmospheric output into single variable files which may also help.

Figure discussions:

Discussion highlighted changes in variability in the maximum salinity timeseries, though this could be due to changes at individual points. It was noted that adding observations to the comparisons would be useful.

GitHub links to OM3 figures for CM3 (please include OM3 comparison links where practical) – @ezhilsabareesh8 :

  • Drake Passage Transport
    The streamfunction is too high in CM3. OM3 matches observations well, while CM3 is too high and still increasing. @ezhilsabareesh8 will update the figures to use the CM2 025 degree simulation.

    This could point to a problem in the atmosphere, either too strong westerlies or too strong coupling. It was noted that Southern ocean wind stresses appear similar between 1 degree CM2 and 025 degree CM3, and so similar wind stresses could be resulting in different ocean forcing. Plots of SSH vs latitude at the Drake passage were raised as a possible way to investigate this further.

  • Global Timeseries. These diagnostics are not quite ready yet.

  • SSS and SSS difference from WOA23
    Large SSS biases were noted in the Arctic

  • SST and SST difference from WOA23
    Large warm biases in the Southern Ocean were noted

Community figures

Rolled over to next meeting
@Harun_Rashid

today I’d like to briefly talk about the ENSO power spectrum (slide 4) and the ENSO diagnostics table (slide 5). The table compares various ENSO diagnostics computed from the latest CM3 run with those calculated from observations, ACCESS-CM2 and other CMIP6 model simulations.

The run length is 37 years, with seven years of data missing
The simulation was done in August by Kieran at ACCESS-NRI

Others

Please paste here and include a caption!

CM3 Dev-Eval Working group meeting minutes

Date: Tuesday 9 December at 10:30 am
Participants: @cbull, @jemmajeffree, @paocorrales, @RachelLaw, @ctychung, @kieranricardo, @MartinDix, @clairecarouge, @DeepashreeDutta, @Dietmar_Dommenget, @paulleopardi, @lachlanswhyborn, @sofarrell, @zoegillett27, @MATTENGLAND, @ShayneM, @heidi, @ClaireT, @gab563, @HIMADRI_SAINI, @ezhilsabareesh8, @lishx, @JulieA
Chair: @kieranricardo
Minutes: @spencerwong
Coord: @cbull

Agenda:

  • shout-out props for Figures from: @Harun_Rashid @DeepashreeDutta @ctychung @DeepashreeDutta
  • quick summary of workflow options for contributions: see [here] for instructions for contributing scripts and notebooks to the repository, and reach out if you have any questions!
  • summary of the dev-eval meetings (Nov 19, Dec 3)
  • community contributions
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

Updates from the last CM3 technical development meeting:

Minutes from the last technical development meeting are available here

  • Currently investigating issues with the Arctic sea ice. Both CM3 and OM3’s Arctic sea-ice is ~2x larger than observations and CM2, but the causes may be different. OM3 has an issue with stuck ice, which isn’t as prevalent in CM3. Other possible causes are differences in bottom melt, issues with where the winds are pushing the ice, meltpond differences, and possible CICE stability issues.
  • @ezhilsabareesh8’s updates to the bathymetry have successfully resolved the Baltic salinity drift.
  • Work going into updating the UM version to 13.8. This will allow for the full set of GC5 central parameter changes to be copied over, which should reduce the radiative imbalance.
  • @CharlesTurner’s virtualised dataset significantly speeds up accessing the CM3 atmospheric data. Issues have come up applying this approach to the CM2 runs, and we are currently trying to work around this.

GitHub issue links:

  • @ctychung: Monthly mean precipitation and surface temperature
    The precipitation maps show that the model still has a double ITCZ bias, though precipitation in the Western Pacific may be slightly better in CM3 than CM2. Surface temperatures show several differences between the two models, especially in the northern hemisphere and over Antarctica.

  • @ctychung: ENSO and IOD power spectra
    We may need more data for a reliable power spectrum, however the amplitude of the spectrum is much higher for CM2 compared to both CM2 and HadISST. Likewise, Niño3.4 standard deviation is almost twice that from observations.

    Using the Planton Matrix was raised as a useful way to compare CM3’s ENSO against other models.

  • @DeepashreeDutta: Comparison of cloud radiative effect (CRE) between ACCESS-CM3, ACCESS-CM2, and CERES observations
    Zonal mean cloud radiative effects are roughly similar for CM2, CM3 and CERES observations, though CM3 shows some improvements compared to CM2. Maps show similar biases in many areas for both models, though CM3 may perform slightly better for high clouds, with reduced RMS error. CRE also looks better in CM3 compared to ESM1.5/1.6.

    It was noted that the total CRE is too negative while the total TOA radiation imbalance is too positive, and improving the CRE alone may worsen the radiation imbalance. Comparison with other CMIP models was raised, and it’s likely that the errors in CM3 are within the range from other models.

GitHub issue links to OM3 figures for CM3:

Community figures

Harun Rashid

@Harun_Rashid

today I’d like to briefly talk about the ENSO power spectrum (slide 4) and the ENSO diagnostics table (slide 5). The table compares various ENSO diagnostics computed from the latest CM3 run with those calculated from observations, ACCESS-CM2 and other CMIP6 model simulations.

These figures also showed a high ENSO amplitude, and the fact that the Niño3.4 peak is higher than the Niño3 peak in the power spectrum suggested that it occurs too far to the west.

The run length is 37 years, with seven years of data missing
The simulation was done in August by Kieran at ACCESS-NRI

Please paste here and include a caption!

Discussion

Timeline for new experiments:

Work is underway to upgrade the UM version to 13.8, incorporate the GC5 Central parameter changed (including cloud tuning), and incorporate changes from the latest release of OM3. The plan is to start a new run in January which includes these changes.

CMORisation

The ad-hoc nature of the current analysis was discussed, and it was noted that this may have to change each time that the model is updated. The long term strategy is to have CMORised output, and to create analysis scripts that work seamlessly across runs and models. Progress is being made, including using CMOR names for OM3 output, however there is still work with the MED team to be done.

Meeting Summary

Meeting summary of the ACCESS-NRI internal meeting on 25/11/2025, on the managing and long term plans on the CM3/OM3/AM3 evaluation efforts.

Quick Recap

The team discussed integrating OM3, AM3 and CM3 evaluation workflows using a submodule approach, where OM3 and AM3 would be submodules in the CM3 repository. Ideally notebooks written for AM3/OM3 would also work on CM3, but not a trivial exercise.


Summary

OM3/AM3 as submodules in CM3 repository

  • Plan is to make OM3 a submodule within the CM3 repository. Prevents desync of scripts between repositories.
  • Where would AM3 sit in this? Worked well for OM3, as evaluation workflow was already in place, but AM3 does not yet have this community effort. Most likely, we would have atmosphere analysis contained in CM3 repo for now, and moved to AM3 repo when model is established.
  • How to handle inconsistencies between CM and OM/AM? Apply pre-processing step, depending on which repo was running the analysis?
  • This could be done by MOPPy, but this would be some time in future- MOPPy work focused on ESM1.6 currently.

OM3–CM3 Workflow Integration

  • There are a number of issues which make generalisation of notebooks difficult- different variable names, time axes.

Maintenance

  • How do we make it easy for users to test their scripts across models and configurations? It’s likely some work would fall to us, but ideally scripts would be quite general before being added to the repo.

AM3 / CM3 Meeting Integration

  • Agreed to trial joint meetings with AM3 participants attending CM3 sessions.
  • Considered structuring agendas so ocean discussion comes first, then atmosphere, so people can leave/show up when necessary.

Next Steps

  • AM3 Team

    • How to put AM3 outputs in a format compatible with the intake catalogue?
    • Invite 21st Century Weather team to CM3 evaluation meetings
  • OM3 Team

    • Write a clear list of OM3/CM3 notebook generalization issues
    • Use same variables for ocean outputs in OM3 and CM3 by using same diag table

CM3 Dev-Eval Working group meeting minutes

Meeting link
Date: Tuesday 20 January 2026 at 10:30 AM AEDT
Participants: @cbull, @wghuneke, @Dietmar_Dommenget, @kieranricardo, @MartinDix, @Harun_Rashid, @paocorrales, @jemmajeffree, @ctychung, @aekiss, @ariaan, @sofarrell, @ezhilsabareesh8, @paulleopardi
Chair: @kieranricardo
Minutes: @spencerwong

Agenda:

  • Reminder about program committee nominations for ESM/CMIP7 focussed workshop in September.
  • shout-out props for Figures from: @jemmajeffree @ctychung @wghuneke
  • CM3 technical meeting updates (Dec 17th, …)
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

Program committee nominations for CMIP7/ESM focussed workshop in September

  • ACCESS-NRI will be hosting a CMIP7/ESM focused workshop in early September in Melbourne. We are currently putting together a program committee to help plan the workshop. We are looking for people who would bring valuable expertise, perspectives, and commitment to shaping the workshop’s program. We also want to build a diverse committee that includes representation across career stages and institutions.
  • If you are interested in being part of the program committee, please contact @spencerwong or the co-chairs for the ESM and Land working groups by January 22, 2026.

Updates from CM3 technical meetings

Minutes from previous technical meetings are available here.

  • @kieranricardo has been working on updating component versions for CM3’s ocean to bring them into line with OM3. The UM is also being updated to version 13.8 to allow for the full GC5 central configuration to be used.
  • Tests with the UM13.8 and full GC5 central configuration showed sudden cooling, and took cloud forcing results further away from observations. Results will be compared with AMIP simulations from the UKMO.
  • A new run will be done once the updates are completed, and this will also include improvements to the wind stress remapping which will likely have large influences on the model’s ENSO.
  • There are still issues with excessive ice volume in the Northern Hemisphere. A similar problem exists in OM3 due to non-advective points in the grid, however the distribution of excessive ice in CM3 is quite different, indicating a seperate problem.

GitHub issue links:

  • @jemmajeffree ENSO and IOD power spectra & seasonal cycle re-visited
    The Niño3.4 autocorrelation suggests too strong a 4 year cycle, while the ENSO amplitude is also much too high.

  • @ctychung time series of CM3 (blue), 0.25 CM2 (green) and obs (orange)
    Timeseries of the Niño3.4 anomaly show that a very strong El Niño and La Niña both occur in the first 10 years of the CM3 run. Plots omitting the first 10 years show that the Niño3.4 standard deviation remains too high even without these events.

  • @wghuneke OM3 vs CM3 Bottom age
    OM3 shows improvements in DSW overflow compared to OM2. While CM3 also has some improvements in the DSW overflow compared to CM2, however it also has significantly excessive open ocean convection. This could be related to model spinup, and we’ll need to check how it looks later in a longer run.

Community figures

@Harun_Rashid

Comparing CM3’s ENSO statistics with CM2 and the CMIP6 ensemble. Two statistics that stand out are the thermocline depth and the thermocline depth standard deviation, both which are much higher than observations

Please paste here and include a caption!

CM3 Dev-Eval Working group meeting minutes

Meeting link
Date: Tuesday 3 March at 10:30 am AEDT
Participants: @peterdobro, @kieranricardo, @JulieA, @RachelLaw, @wghuneke, @paulleopardi, @aekiss, @sofarrell, @ezhilsabareesh8, @zoegillett27, @DeepashreeDutta, @Harun_Rashid, @spencerwong, @Dietmar_Dommenget, @ShayneM
Chair: @kieranricardo
Minutes: @spencerwong

Subscribe to the ESM Working Group calendar as follows:

  1. ESM Working Group Meeting Calendar feed (right click on the link and copy the URL)
  2. Use calendar feed URL to subscribe to the calendar

Agenda:

  • Description of new simulation. (Output at /g/data/zv30/non-cmip/ACCESS-CM3/cm3-run-20-01-2026-om3-update/cm3-datastore/cm3-datastore.json)
    • The new simulation updates the ocean to the latest OM3 release, including parameter changes.
    • Remapping wind stresses via the patch method removes artefacts in the ocean currents.
    • Land ice fluxes improve the water balance and drift in sea surface hight, however increase the sea ice extent especially in the southern hemisphere.
    • Work on the GC5 configuration is continuing to improve the energy balance.
  • shout-out props for Figures from: @ctychung, @Harun_Rashid, @ezhilsabareesh8
  • CM3 technical meeting updates (Jan 28, Feb 11, Feb 25)

GitHub issue links:

  • @ctychung (new run) Monthly mean precipitation and surface temperature. GHLINK
    • Mean precipitation figures show a double ITCZ bias. Flux correction runs were suggested, where the coupled heat exchange would be manipulated to bring the SSTs closer to the observed mean state, as a way to address this bias, or as a way to investigate the sources of model biases. It may be possible to set up a flux correction run via the MOM settings.
    • An AM3 evaluation repository is being set up, which will make it simpler to compare biases in coupled and atmosphere only simulations.
  • ENSO and IOD variability Evaluation: ENSO and IOD power spectra & seasonal cycle · Issue #15 · ACCESS-Community-Hub/access-cm3-paper-1 · GitHub

GitHub issue links to OM3 figures for CM3:

Community figures

@Harun_Rashid presented plots on:

  • Sea surface height: The sea surface height drift seen in the previous CM3 run no longer occurs. This would be due to the addition of land ice fluxes which close the water budget.
  • Biases in SAT: Compared to ERA5, CM3 SAT is much cooler, especially in the Arctic. This is similar to the biases in CM2 and other CMIP6 models.
  • Linear trends in SAT: The CM3 run (using present day forcings) is currently cooling. The model contains an initial cool bias, and is expected to warm as it spins up. @wghuneke’s figures show similar behaviour in CM2 and 0.25 degree CM2, where an initial period of cooling was followed by a long period of warming.
  • Regional means, standard deviations, and trends: SSTs in several regions show higher variability than observations, especially in the Niño3/3.4 regions of the Pacific.
  • Power spectra: Niño power spectra shows a spectral peak at ~3.3 years, however the period is too regular and the overall variability is too high.
  • ENSO metrics: Comparing several ENSO metrics against several CMIP6 models shows that the current CM3 run performs quite well. Biases in most metrics are small, though larger biases exist in the thermocline feedback, zonal wind forcing, sst standard deviation and sst skewness.

Please paste here and include a caption!

CM3 Dev-Eval Working group meeting minutes

Meeting link
Date: Tuesday 31 March at 10:30 am AEDT
Participants: @RachelLaw, @MartinDix, @JulieA, @kieranricardo, @Harun_Rashid, @AndyHoggANU, @wghuneke, @Heather_Leasor, @Dietmar_Dommenget, @cbull, @sofarrell, @spencerwong, @DeepashreeDutta, @joshuatorrance, @ctychung, @zoegillett27, @jemmajeffree, @HIMADRI_SAINI
Chair: @kieranricardo
Minutes: @spencerwong

Subscribe to the ESM Working Group calendar as follows:

  1. ESM Working Group Meeting Calendar feed (right click on the link and copy the URL)
  2. Use calendar feed URL to subscribe to the calendar

Agenda:

  • Updates from CM3 technical meetings: The previous simulation completed 100 years and has now been stopped. Work is going on to prepare for a new simulation which will include improvements to masks in the polar regions, and fixes for issues with wind stresses. The reproducibility bug has now been resolved, meaning that repeating identical runs will produce identical results.
  • CMIP7 fast track and the IPCC Assessment Report:
    • @AndyHoggANU discussed the timeline for evaluation papers to be included in the AR7 report. Fast track data will be coming out soon, and papers need to be submitted by March 2027 in order to be cited by the report. We would like to understand how NRI can support, encourage or help researchers write papers based on the fast track.
    • Meetings/workshops are being planned later in the year, with one focused on the Australian region, and a NRI ESM and Land workshop planned for September, which will include CMIP as a focus.
    • For ESM1.6, NESP will be planning an evaluation paper on ENSO, drivers, and mean states.
  • shout-out props for Figures from: @AndyHoggANU, @sofarrell
  • ESM and land community workshop:
    • We are planning a 3 day ESM and Land workshop for early September, which will include science presentations, training, and likely a hackathon. If you have any ideas of things you would like to see, or ideas for invited speakers (for example someone visiting from abroad, or someone working in Australia) contact any member of the program committee (@wghuneke, @JulieA, @alexnorton, @lachlanswhyborn, @HIMADRI_SAINI, @spencerwong, @adisen99, @zoegillett27).
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

GitHub issue links:

GitHub issue links to OM3 figures for CM3:

Community figures

@AndyHoggANU

Freshening of Labrador sea and North Atlantic, from Arctic, Greenland and North America (via St Lawrence River) stabilizes the region and shuts off convection cools surface, showed strong trend since in last month trend plots, Arctic ice thickens to > 8m. Also seen in CM2-025 but centuries to occur and FW flux to build up, different ocean mixing scheme used in CM2 (KPP). In CM3 (ePBL) mixing tends to increase stable ocean conditions with the FW flux.

Further to Wilma’s analysis from January, here is a figure I showed at the CMIP workshop recently. Confirms that OM3 gives improved Antarctic dense water at 25km resolution, but that CM3 is still not great. Not only is there excessive open ocean convection, but it is also very salty on the shelf.

MLD shows the open ocean convection.

Sea ice is also pretty different (causality isn’t clear to me).

There was discussion around whether the mixed layer depth figures suggest open ocean convection, or instead excessive mixing that is unrelated to large polynyas. Plotting the DSW exports was suggested as a way to confirm whether the younger bottom water in CM3 is caused by the deep offshore mixed layers, or if it’s instead coming from the shelves.

Discussion also centred around the iceberg spreading scheme. Currently, all freshwater is spread over a large pattern around the ice sheets, which may be contributing to the excessive salinity over the shelves. An updated scheme which allocates a fraction of the freshwater to the coast is being worked on. There is interest in performing a test run without the iceberg spreading, so that all freshwater is input around the coast. Additionally, interest was raised in comparing the total iceberg fluxes in CM3 and OM3.

@sofarrell



Over the coarse of the run there is strong surface freshening, especially in the Arctic. The CM2 025 simulation had a similar freshening kick in near the end of the run, whereas it happens quite quickly with CM3. Convection in the Labrador sea gets shut down and surface temperatures become much colder.


The age tracer at 30E shows ventilation at the coast, but not further out. Mixing extends to 500m in some regions but doesn’t appear to extend further down.


40W also shows water coming down the slope, but doesn’t show mixing to the bottom away from the coast.

At 160W 50 years into the run, a coastal polynya did occur around 76S. There is evidence of quite strong mixing down to ~400m in the temperature data. However there isn’t evidence of brine mixing in the salinity data.

The EPBL mixing scheme appears to keep things more stably stratified.

Please paste here and include a caption!

CM3 Dev-Eval Working group meeting minutes

Meeting link
Date: Tuesday 28 April 10:30 am
Participants:
Chair: @kieranricardo
Minutes: @spencerwong

Subscribe to the ESM Working Group calendar as follows:

  1. ESM Working Group Meeting Calendar feed (right click on the link and copy the URL)
  2. Use calendar feed URL to subscribe to the calendar

Agenda:

  • shout-out props for Figures from:
  • offer support (breakout room?) to people wanting to get involved / could set aside 10 minutes at end of meeting to answer questions.

GitHub issue links:

GitHub issue links to OM3 figures for CM3:

Community figures

Please paste here and include a caption!