ESM Working Group Resource Allocation Guidelines

Introduction

ACCESS-NRI allocates compute and storage resources to the Working Groups to use to support their activities.

Currently those resources consist of computational resources (KSUs) and storage (/g/data allocations) on NCI HPC systems. These are allocated through the lg87 project, so you must join it to be able to access them.

The process for allocating resources amongst working groups is still being finalised before being approved through the ACCESS-NRI governance structures. In the interim the Earth System Modelling Working Group has been allocated 1.25MSU for the second quarter of 2023. No specific storage allocations have been allocated to working groups at this stage, but there may be some scope to get some disk allocation to facilitate sharing data from shared experiments.

The ESM working group is allocated at least 875KSU per quarter and is currently allocated 50TB of storage on /g/data and /scratch at NCI.

Proposal for distributing resources

The ESM Working Group needs a process to allocate these resources amongst the ESM Working Group community that prioritises the needs of the community, is fair and not onerous.

Proposal:

Create a topic with a brief outline of the proposed experiment (see example). Other members of the community who want to collaborate, or use the results of the experiment, or otherwise think it worth prioritising should reply indicating their support. The topic is where discussions should occur around how the experiment is configured, how it might be shared etc.

Requirements for anyone who is allocated working group resources for an experiment:

  1. Engage in good-faith collaboration to align with community requirements, e.g. save output diagnostics required by others
  2. Report back to the community: update their experiment topic with results including sufficient meta-data to enable others in the community to use their results, and communicate this at the next appropriate WG meeting
  3. Make data from experiments available to other members of the community (this could be facilitated by ACCESS-NRI if shared storage is not available)
  4. Must use an ACCESS model
  5. Should be ESM/Coupled-model related

Allocations will be determined by the Working Group Chairs. Time is tight, and human resources are limited, so depending on demand it may be first-come, first-served. If there is more demand than resources available prioritisation will be done based on:

  1. Shared interest: the more people who indicate interest in using the experiment the better
  2. Early career researchers and/or those without compute resources such that the experiment could not be run without these resources
  3. Ability to use the allocated resources (time is short, if the experiment isn’t already configured and ready to go, or already running, there is probably not time to get it working)

There is just over a month left in the current NCI accounting quarter, so decisions will need to be made rapidly.

TL;DR

If you want to use some of the ESM Working Group allocation join lg87 and create a topic outlining your experiment. Share your proposal widely in the ESM Working Group community, engage with questions and convince your fellow group members of the merits of your experiment and urge them to reply to the topic indicating their support.

At the WG meeting today @LaurieM noted that no-one had made any proposals to use the compute resources ACCESS-NRI has made available to the ESM Working Group. As there is so little time left @LaurieM suggested we should ask members of the group who are currently running simulations if they could make use of this time.

Specifically @tiloz is running a number of ensembles that he would like to make available to the community to use.

Also @ars599 is working on CM2 ensemble runs, and @ShayneM has made contact to see if they are short on resources for the remaining quarter.

I am running large ensembles for DAMIP/LESFMIP and I have almost completed 10 ensemble members for 6 different single-forcing set-ups (GHGs, aerosols, solar, volcanic, natural and land use). I started total Ozone simulations and I have completed 4 ensemble members so far for this experiment and if it is okay with this group then I could set-up another 6 ensemble members for total Ozone which would use about 1 MSU. This is not urgent and I am happy for others to use available compute and I can scale back accordingly.