Payu: a workflow manager for some ACCESS models

ACCESS-NRI Releases
, , ,
17

09/11/2026

:rocket: Release: 1.4.0

https://github.com/payu-org/payu/releases/tag/1.4.0

Getting Started

This version of payu is now available on gadi:

module use /g/data/vk83/modules
module load payu/1.4.0

Updates

What’s Changed

For a full list of changes, please see Payu Github 1.4.0 release.

Notes

1. A new feature: build an intake datastore for your experiment output

Payu can now build an intake catalog datastore for model output using the builders from access-nri-intake-catalog. This makes your experiment output directly searchable and loadable with intake, without having to build the datastore by hand. The datastore is written to the archive directory by default, but if a sync enabled to a local directory, e.g. a location in /g/data, the datastore is generated for the data in the sync location and saved there.

The datastore is built as a separate post-processing job that runs after the sync job, so it joins the existing workflow: run -> collate -> postscript -> sync -> catalog.

It is disabled by default and can be enabled in config.yaml by:

catalog:
    enable: true

You can also customise the resources allocated to the catalog generation job:

catalog:
    enable: true
    queue: normal
    cpus: 8
    mem: 50GB
    walltime: 0:05:00

The values shown above are the default resource settings. It can also be run manually with:

payu catalog

For more details, please refer to this section in payu documentation.

2. payu status tracks remaining jobs

payu status now shows how many runs are still waiting to be submitted after payu run -n N. For example:

$ payu run -n 5
$ payu status
========================================
Run: 3
  ------------- Run Info -------------
  Job ID:            1000.gadi-pbs
  Run ID:            xxxx
  Stage:             archive
  Total Queue Time:  0h 1m 14s
  Model Finish Time: 1951-12-01T00:00:00
  Model Exit Code:   0 (Success)
  Exit Status:       0 (Success)
  Output Log:        xxxx
  Error Log:         xxxx
  Job File:          1000.gadi-pbs.json
========================================
Remaining run to be submitted: 1
========================================

The “remaining” line is only shown when at least one run is still waiting to be submitted. If a run fails, subsequent runs won’t be submitted, accordingly the remaining line is hidden.

3. Bug fix: correct PAYU_CURRENT_RUN in postscript jobs when runspersub > 1.

Thanks to the community feedback, we identified an issue where PAYU_CURRENT_RUN was not correctly incremented in postscript jobs when runspersub was greater than one.

As a result, postscript jobs could reference the wrong output directory.

This is now fixed in Payu 1.4.0. PAYU_CURRENT_RUN is correctly updated for each run, and postscript jobs locate the corresponding output directory.

4. A fresh look for payu documentation

The payu documentation now uses the furo theme. The new layout is cleaner and easier to navigate, has a sidebar with all sections at a glance, and supports both light and dark mode.

Support

Replies to this topic are disabled.

If you have specific questions about this release follow the guidelines for requesting help from ACCESS-NRI.

If you have questions about payu create a topic in a category that best matches the model you are using, or in the Technical category and tag it with payu. If you require assistance follow the guidelines for requesting help from ACCESS-NRI.

Credits

Developments by @Qianhui_Chen, @minghangli, @anton @Aidan.

Reviewers included @jo-basevi, @atteggiani, @jasmeen_kaur, @spencerwong, @CharlesTurner .