Compute and Storage Resources
Introduction
ACCESS-NRI allocates compute and storage resources to the Working Groups (WGs) to use to support their activities.
Currently, those resources consist of computational resources (KSUs) and storage (/g/data allocations) on Gadi. For the Machine Learning for Climate and Weather WG, they are allocated through the nm47 project, which you must join the project to be able to access them.
Proposal for distributing resources
Aimed at prioritising the needs of the community, a process for allocating resources amongst WGs has been put in place according to the ACCESS-NRI Merit Allocation Guidelines.
Proposal:
Create a topic with a brief outline of the proposed experiment (see example).
Other community members who want to collaborate or use the results of the experiment should reply indicating their support. The topic is also where to have discussions around how the experiment is configured, how it might be shared, etc.
Requirements for anyone allocated WG resources for an experiment:
- Engage in good-faith collaboration to align with community requirements,
e.g., Save output diagnostics required by others - Report back to the WG community:
Update their experiment topic with results, including sufficient metadata, to enable others to use their results; communicate this at the next WG meeting. - Make data from experiments available to other community members.
This could be facilitated by ACCESS-NRI if shared storage is not available.
Short Version
If you would like to use some of the Machine Learning for Climate and Weather WG compute and storage allocations:
- join nm47 and
- create a topic outlining your experiment.
- Share your proposal in the Machine Learning for Climate and Weather WG community, engage with questions and convince your fellow members of the merits of your experiment and urge them to reply to the topic indicating their support.
Technical Support
Introduction
ACCESS-NRI also provides technical support to the Machine Learning for Climate and Weather WG. This is done by assigning a Research Software Engineer from the Software Transformation Team to work with members of the community on selected projects for a certain period of time.
Submiting a project
What the project should aim to achieve
- Create clear collaboration opportunities with other Working Groups, and/or
- Contribute to community resources by making part or all of the code available when appropriate (for example, after a paper is published), ideally including a jupyter tutorial in PyEarthTools
- Use PyEarthTools, if possible.
Submitting a proposal
- Submit a short project description by creating a topic in the forum.
- Discuss with ACCESS-NRI’s Software Transformation team the specific technical tasks that align with the team’s expertise and availability. Examples of the types of support include:
- Developing hyperparameter tuning pipelines, or
- Developing inference pipelines (e.g. when many inputs need to be processed), or
- “Brushing up” scripts and data for packaging/distribution/publication.
- As part of this discussion, it is expected that the Software Transformation team may provide technical support for a few hours per week for up to 3 months, depending on the project’s scope and mutual agreement.
- This arrangement should be reviewed to ensure that the level of support is fair across projects and consistent with ethical allocation of ACCESS-NRI resources.
- The expected start date of the support should be agreed upon during this discussion.
- The time that the Software Transformation team will be able to allocate to these projects until the end of FY25-26 will be up to 4 hours per week, on average. This availability will be reviewed every six months.
- The co-chairs will then review the proposal and accept/make suggestions
- Present the proposed project at the ML Working Group monthly meetings.
- If the project runs for multiple months, provide occasional progress updates to the community.
- When the project is completed, present it again at the ML Working Group monthly meetings.