ESM working group: Meeting notes 2024

Notes from each meeting of the ESM Working Group are added as a reply to this topic.

Make each reply a wiki post, that way other attendees can edit the notes. This can greatly reduce the burden of note-taking allowing that person to participate fully in discussions without worrying too much about missing important details.

Archived meeting notes:

2023

13 posts were merged into an existing topic: ESM working group: Meeting notes 2023 (Archived)

Date: 7/12/2023

Participants: 13

Chair: @LaurieM

Admin

  • For the next meeting (29th) we will have a shared meeting with Forecasting and Prediction Working Group to discuss the Triple La Niña experiment, and an ACCESS-S2 User Training Workshop. Will spend 20 minutes with some discussion of where to find the data, how to analyse and some discussion of analysis and ideas for where to focus that analysis.
  • @sebmckenna happy to talk about ACCESS-CM2 pacemaker experiments.

Shared experiments/ACCESS-NRI resources

  • ACCESS-NRI Merit Allocation guidelines are available. The requirements to access time are very minimal compared to MAS applications. Come and use the resources on offer!
  • ESM project lg87 /g/data storage: 30TB used out of 50TB. 116KSU used 758KSU remaining.
  • @ariaan could use some SUs. Will put up some information about them.
  • WG needs more /g/data storage space. Take to SAC.

Freshwater forcing in sub-polar North-Atlantic

Status of @gpontes experiment.

(Apologies, the top and bottom of the video have been clipped)

Freshwater forcing in sub polar northatlantic. Just esm-ssp585-45 computed under lg87. Including historical also under /g/data/lg87.

Some issues restarting historical experiments from other runs. Reported on hive. Ran another historical ensemble for historical. Also available.

Started 4-member ensemble for historical and SSP. HI-X-grm, ssp585-X-dgrm. 2-member ensemble ssp585-X-cgrm.

@tiloz is historical a new ensemble member? Restarted from a different time? Yes.

@tiloz asking if ssp585 is concentration driven. Confirmed yes, not emissions driven.

Was using the warm-start.sh to rebuild experiment for payu. Got a calendar error. Works fine if all processes using payu.

@ariaan did restart historical runs using payu from older non-payu runs. Might have been a date issue. @holger assisted.

@ariaan Are ensemble members of 2 and 4 enough? Need larger ensembles? Would it be good to have historical members branched further apart to have different AMOC states? Would be good to have more ensemble members.

@gpontes Could do so if community thinks it is useful. Ideally would be 10 members. Restarted experiments from 1950. Perturb IC and run 10 years, then begin FW perturbation.

@ariaan GFDL models have strong convection in southern ocean (Amundsen and Ross Seas). For some things in the deep ocean models have a specific path.

@gpontes Now investigating teleconnections between AMOC and southern ocean.

@RachelLaw ssp585 is the highest scenario. Plans for a more moderate scenario?

@gpontes Other scenarios not set up for payu. Mostly looking at coming decades. 20-30 years doesn’t matter too much. All simulations go to 2100. Focus on coming decades

@LaurieM Simuations quite idealised. Contentious about how to design the FW experiment.

@RachelLaw also having issues restarting payu from a different restart.

@gpontes could increase number of members If group wants to do it.

ACCESS-ESM1.5 – ACCESS-NRI supported release

@Aidan updated us on plans for an ACCESS-NRI supported version of ACCESS-ESM1.5. It will have:

  • spack build system
  • build-CI testing
  • released experimental configurations utilising payu
  • full experiment provenance
  • automated reproducibility testing

This topic on the forum gives an overview of the project and will be used as the main place to find information about the progress of the release

(Apologies, the top and bottom of the video have been clipped)

What’s going on?

@RachelLaw: the CMIP panel want feedback on the AR7 fast-track design

This is a wiki post , if you want to update any of the details in these meeting notes, or to add your own recollection of what was discussed then edit me rather than replying.

Date: 29/02/2024

Participants: 26

Apologies: @JulieA

Chair: @eunpalim

Shared Section

1. Introductions from each group

@debbie_h: Forecasting and Prediction Working Group incorporates Machine Learning (ML), Data Assimilation (DA), seasonal forecasting. Quite disparate groups, but try to do things for each group. Had a good ML session at last workshop, has led to good collaboration since.

No common infrastructure. Not supported by ACCESS-NRI. Working on how ACCESS-NRI could support those areas. Drafting two short papers for guidance: how ACCESS-NRI could support DA into the future, and another on how to support ML into the future.

@nathan-e difficult getting cohesion with these challenges.

@LaurieM co-chair. Earth System Modelling Working Group also somewhat disjointed. Focus on the use of the coupled models: ACCESS-ESM1.5, ACCESS-CM2, also ACCESS-S2. Also interested in coupled climate. Past, future and paleo climate. Ocean/atmos interactions for El Nino. Also for future climate.

Big focus on carbon cycle. Ocean and land carbon cycles.

Last few months trying to get to know each other, find out what others are doing. All want to push forward with ACCESS suite of models. One focus has been adding flexibility of framework. Modifying forcing, particularly ancillary files. Land-sea mask and bathymetry and orography.

Collaborating closely with ACCESS-NRI. Pushing for a more user friendly workflow to expand range of activities in models.

Also want to facilitate collaboration using ACCESS suite. All co-chairs from different institutions with different scientific focus.

2. Triple La Niña experiment

The Triple La Niña experiment that was a joint initiative of the ESM Working Group (Working Group) and F&P Working Group (Working Group )

Triple La Nina of 2020-2022/23. Unusual to persist for three years. Not particularly strong, but 3rd year Australia experience severe flooding and rainfall events.

Preceeded by black summer and Huna Tonga in Jan 2022 could have contributed to persistence.

ACCESS-S real-time forecasts skilfully capture the triple dips of the La Nina event at up to 4-month lead time.

Examine effect on forecast skill of atmospheric initial conditions. Examining forecast differences in topical SST and persistent SAM.

Need 3 datasets:

  • Hindcasts (retrospective forecasts). Used to compute climatology and std dev. Also can compare to previous La Nina to assess skill
  • Standard real-time forecasts with realistic Atmosphere initial conditions
  • Experimental forecasts with unrealistic Atmosphere initial conditions
  • How to access the outputs

Hindcasts are in

/g/data/ux62/access-s2/hindcast/raw_model/{atmos,ice,ocean}

Standard

/g/data/dx2/access-s[12]/rt/raw_model/atmos/{has,psl,ta,ts,ua,va,zg}

Experimental forecasts

/g/data/lg87/experiment/triple_la_nina/{atmos,ice,ocean}

18 week forecasts (4 months). 23-member ensembles for experimental forecast.

Should we pre-compute climatology and ensemble mean and store. Would require more space.

What should we do with this data?

Peer-reviewed paper? La Nina, SAM, Australian Rainfall? Encourage ECRs?

Announce availability?

Half-day hackathon?

Who might be interested in a group study?

@dkhutch: don’t want to speak for them, but @nicolamaher and @jemmajeffree couldn’t make it, but might be interested?

NESP planning a special effort on Triple Dip La Nina. This experiment came from discussion in NESP meeting. High customer demand to understand TLN and impact on Australian Climate.

Wanted to contribute to NESP

@eunpalim will Add slides to hive.

@ctychung Plan to have stakeholder based webinar around July/August. Would be good to have results from this presented.

@eunpalim Similar experiments done by @lynnzhou for wet 2022. Large magnitude of wetness was atmosphere driven. If not having large atmospheric impacts (smoke, volcano) how much of the La Nina can we reproduce? Which is most important driver? Atmosphere or Ocean. What more important IOD, SAM? Good opportunity for ECR to understand nature of forecast data and systems.

@aidan: How important is collaboration?

@debbie_h: ACCESS-S hindcast already available. Coordination not required, but coordination is an option if that is people’s wish.

Contributing this effort to NESP would be good.

Not thinking too deep an investigation at this stage. Any contribution welcome.

Discuss possible ACCESS-S2 Training. Gauge interest and requirements

Half-day hackathon? Look at some major events and analyse together?

Ensemble and climatology need more space. Certainly possible. @Aidan:ACCESS-NRI can support training activities.

@debbie_h: Some people want to run ACCESS-S2. Tricky. No documentation. Only 3 people at Bureau can run model. Only set up for forecasting. Limited knowledge of how to do this for sensitivity and scientific experiments. Getting ACCESS-NRI involvement depends on demand. Get in touch with @debbie_h to register interest in using ACCESS-S2.

ESM Working Group Section

3. ESM WG Admin

  • @sebmckenna has volunteered to talk on "SST and ENSO biases in ACCESS-CM2 pacemakers” for the next meeting (2024-03-14T02:00:00Z)

  • We need more speakers! Message any of the co-chairs or @Aidan if you’re keen

@eunpalim: Saw good talks at AMOS. Please feel free to recycle AMOS talks in this forum. Even if seen before, always good to see again and have better understanding.

@ShayneM: putting shallow water model into ACCESS coupled model. Will talk in next few months.

@dkhutch Maybe @Dietmar_Dommenget could present on the paper he is leading and which @dkhutch is collaborating. Collaboration came from ESM WG meetings.

@rachellaw also just had a paper accepted for CMIP7 overview. Could present some slides.

  • Report from Scientific Advisory Committee (@LaurieM). Next SAC Meeting 2024-03-21T03:00:00Z

Reminder of CMIP7 ESMValTool Hackathon. March 12-14 at Aspendale. Probably too late to register. Drop-in session tomorrow.

  • If you want to suggest a topic to cover in the ESM WG meetings reply to this topic and add your idea

  • Do you know anyone who you think would benefit from coming to these meetings? Please invite them! We definitely want ECRs and students to attend too.

4. ESM Shared experiments/ACCESS-NRI resources

  • See the guidelines for accessing working resources. The requirements to access time are very minimal compared to MAS applications. Come and use the resources on offer!

  • lg87 /g/data storage quota has been doubled to 100TB. Note that this is intended for short term storage only (generally maximum of 1 year).

  • ESM project lg87 /g/data storage: 27TB used out of 100TB. 117KSU used 757KSU remaining.

Probably still don’t have a clear process for how experiments that are proposed get resources to run. Filled in proposal. then what? @Aidan: bring it up at a meeting, volunteer to begin running experiment.

5. What’s going on?

CMIP7 fast-track consultation

Need survey by the middle of next week. Put feedback on hive, contact @RachelLaw or fill in survey.

@JulieA three level B positions for new CoE. UNSW advertised now. Another at UniMelb(?) one at Monash(?).

ACCESS Hackathon clashes with next meeting. @aidan will put up a poll to gauge interest.

This is a wiki post , if you want to update any of the details in these meeting notes, or to add your own recollection of what was discussed then edit me rather than replying.

1 Like

Date: 28/03/2024

Participants: 13

Apologies: @eunpalim @ShayneM @dkhutch

Chair: @tiloz

Agenda:

1. Admin

We have no speakers lined up for our next meeting or any subsequent meetings. Please volunteer to speak. Message @Aidan to discuss.

@tiloz Please advertise and invite people to talk. Particularly early career researchers.

@Dietmar_Dommenget can talk about change land-sea mask results with GFDL model. Might be able to talk about the ACCESS equivalent in May.

Remember to add any items you want this WG to discuss to ideas for topics

@tiloz SAC Meeting last week. All SAC meeting notes are publicly available on website slightly delayed.

@tiloz Will be ACCESS-NRI Workshop in September (2nd-5th). Also a COSIMA workshop beginning of July. Might reduce attendance to workshop. Call for people to join workshop program committee. Structure still to be decided.

Progress towards CMIP7 and model development plans. Largely on-track. Fast track timeline very challenging.

CMIP7 Hackathon hands-on meeting to evaluate ACCESS model output using ESMValTool and iLamb. Good intro and setup. Overview of existing recipes, look at what is required. May be a follow up at the workshop.

@LaurieM Would be good to have someone from ESM WG involved. @dkhutch and @wghuneke did it last year. Can be anyone, ECR or senior scientist.

2. Shared experiments/ACCESS-NRI resources

ESM project lg87 underutilised. 195KSU remaining out of 405KSU grant (rest reallocated to other projects). 30TB our of 100TB used of quota on /g/data/

@tiloz storage allocation is short-term for duration of experiment. Need to have a long term plan for where it will reside.

@clairecarouge presented plan to manage lg87 /g/data storage. Easier to manage storage if people write into a specific location for an experiment. Experiment folder created, can work however you want underneath.

Do want to have some data folders for slightly longer term storage. Would need to be of use to the community and decided by the working group.

To help with organisation will have a writer group, only those who are members can write by default. Others can only write to their own experiment folders to prevent data loss.

Was hoping to be ready for Q2, not quite ready yet. All admins will have write access to all data to enable clean-up.

Writer group will be some ACCESS-NRI, and hopefully some from community who want to be involved. Won’t be a large workload. No previous knowledge required.

Please put your experiment proposals on the hive. Great opportunity to access NCI compute resources. See guidelines for how to do this.

David (@dkhutch) and Dietmar (@Dietmar_Dommenget) are running experiments under lg87. @Dietmar_Dommenget: Running 4x400 CPU experiments. First test runs to confirm results obtained from GFDL model. Idealised ocean basin configurations. Seems to be working. Will do at least 15 experiments. Roughly 1MSU. Need to run 100 years. About ENSO. First 50 years trending. Changed ocean basin sizes. Used GFDL model. Submitted paper with interesting El Niño. Wanted to do with low-res ACCESS model. @dkhutch has looked at similar problems. Managed to run Miocene configuration in ACCESS-ESM1.5. Also testing with no continents in tropics, 1, 2 or 3 oceans basins. Only managing 10-15 model years/day.

@tiloz do current model proposals meet needs of community? Need to consult with community.

@JulieA is assistance to run models in scope for ACCESS-NRI? @clairecarouge if it is an experiment that is required for model development. Otherwise no.

@Dietmar_Dommenget new experiments should have ACCESS-NRI input. Learn what users are struggling with. More than providing CPU time. Particularly novel simulations. Spending months to just move land/sea boundaries.

@LaurieM Have to be careful what we suggest. Difference between running experiments versus flexibility/application of a model. Maybe not run the experiment, but be involved in helping/documenting and moving forward. @dkhutch invested a lot of time in learning how to configure the model. Spread the knowledge on GitHub. Should we include someone from ACCESS-NRI. Should be able to take it and improve it. Could help in the community.

@tiloz model development requires substantial changes to the code and model configuration. Support is required.

@Dietmar_Dommenget ACCESS-NRI needs the level of knowledge that @dkhutch has developed. Will not help someone with the next problem. ACCESS-NRI should do this sort of work. These are essential skills that we need.

@JulieA where is the boundary between science and infrastructure. Can be very difficult to configure the model.

@tiloz boundary is blurry between scientific and infrastructure. @LaurieM ACCESS-NRI has quite clear definition of what is in-scope.

3. Science talk

Sebastian McKenna (@sebmckenna) presented “SST and ENSO biases in ACCESS-CM2 pacemakers”

[VIDEO WILL BE INCLUDED WHEN AVAILABLE]

ENSO appears to be a more dominant factor in Indian Ocean variability than Pacifici Ocean mean state.

Added SST perturbation to restoring file to create ensembles.

Just finished all ensemble members, still need to analyse it all.

Initial results:

  • IOD doesn’t change under different El Niño conditions
  • Basin wide warming more likely with EP El Niño

More analysis required. Determine if we need more ensemble members.

@Dietmar_Dommenget biases in Indian Ocean seem more important? Can’t test with this experimental setup? @sebmckenna can’t test with this pacemaker experiment.

@Dietmar_Dommenget what do biases look like? Mean state winds

Too warm if east Pacific and cool around Arabian Sea. Biases are seasonal. A lot of mean-state bias related to biases in wind.

@gpontes Have you evaluated teleconnection between ENSO and IOD in the model? Correct strength of feedback. @sebmckenna did look at this. Do have internally forced, ones that happen during ENSO and not. Gets both type of events. Model is more independent in the model.

@Dietmar_Dommenget IOD independent modes are weak, too strong in the models? Too sensitive/unstable. @sebmckenna most coupled models have overly strong IOD bias. Not due to remote teleconnections. Some problem with the Indian Ocean in the model.

@gpontes investigated Indian Ocean dynamics? @sebmckenna only looked at surface ocean/atmosphere interaction. Not looked below the surface. Would be good to investigate.

@Dietmar_Dommenget Atlantic could also play a role. Warm biases could also influence the tropics.

@sebmckenna in literature a lot more about Pacific/Indian Ocean interactions. @Dietmar_Dommenget what matters is the relative differences in biases. It might not be relevant, but if you have the setup interesting to see if Atlantic mean state has any influence.

@sebmckenna study by @dhb599 of Atlantic pacemakers didn’t look at Indian Ocean. Might be interesting to look at the data from that.

4. What’s going on?

@Aidan ACCESS-NRI ACCESS-ESM1.5 release is proceeding, but slower than hoped. Initially will target three configurations: pre-industrial control and two historical, concentration driven and emissions driven. Will consult the community in the near future about what configurations should be targeted once the intial release is done. This release is different from what is currently available as it will be a ground-up restructure of how the model is built, deployed and tested. Have developed the infrastructure with the ACCESS-OM2 release which is being finalised. Includes experiment provenance and tracking. In the medium term this will mean offering databases with experiment metadata that is searchable by anyone. Will apply the same methodology to ACCESS-ESM1.5. Configurations will use pre-built and deployed model binaries, but the system is very flexible and users in the community will benefit from the ability to easily build the model and change dependencies and model component versions easily. In addition these is an ACCESS-NRI supported model and configurations, so the community can ask for assistance. @tiloz this is very welcome for the development of future versions of ACCESS-ESM.

This is a wiki post , if you want to update any of the details in these meeting notes, or to add your own recollection of what was discussed then edit me rather than replying.