Payu: a workflow manager for some ACCESS models

ACCESS-NRI Releases
, , ,
16

09/09/2026

:rocket: Release: 1.3.6

https://github.com/payu-org/payu/releases/tag/1.3.6

:warning:

There is a bug in this release for ACCESS-ESM1.5 models that returns the error

[ERROR] ‘Experiment’ object has no attribute ‘repeat_run’

Work-around is to use another payu version for ACCESS-ESM1.5 models
:warning:

Getting Started

This version of payu is now available on gadi:

module use /g/data/vk83/modules
module load payu/1.3.6

Updates

What’s Changed

For a full list of pull requests that includes additional bug fixes, please see Payu Github 1.3.6 release.

Notes

1. Bug fix: payu was collating output001 instead of output000 for run 0.

For each run N, payu is expected to collate output00(N-1). However, @minghangli spotted that for run 0, payu was attempting to collate output001 instead of output000. This resulted in a FileNotFoundError.

This bug was introduced in payu 1.3.1 and now is fixed in payu 1.3.6. It only affects run 0 and does not impact any other runs.

2. Add job dependencies to run → collate → postscript → sync workflow

Previously, payu submitted the postscript and sync jobs at the same time. This meant that the sync job could start before the postscript job had finished. For this reason, previous payu doesn’t sync the last run if postscript is enabled.

With payu 1.3.6, job dependencies are added to ensure that each stage only starts after the previous stage has exited successfully. The sync job can now safely sync the last run, ensuring that the postscript has completed before syncing.

All stages can be monitored via payu status, a feature requested by the community. For example:

$ payu status
========================================
Run: 37
  ------------- Run Info -------------
  Job ID:            1700.gadi-pbs
  Run ID:            989xxxxx3
  Stage:             archive
  Total Queue Time:  0h 1m 57s
  Model Finish Time: 1903-12-01T00:00:00
  Model Exit Code:   0 (Success)
  Exit Status:       0 (Success)
  Output Log:        /home/189/xxx
  Error Log:         /home/189/xxx
  Job File:          /scratch/tm70/xxx/run/1700.gadi-pbs.json
  ------------- Collate Info -------------
  Job ID:            1710.gadi-pbs
  Job Dependencies:  1700.gadi-pbs
  Stage:             exited
  Total Queue Time:  0h 1m 12s
  Exit Status:       0 (Success)
  Output Log:        /home/189/qsss
  Error Log:         /home/189/ssss
  Job File:          /scratch/tm70/xxx/collate/1705.gadi-pbs.json
  ------------- Postscript Info -------------
  Job ID:            1706.gadi-pbs
  Job Dependencies:  1705.gadi-pbs
  Stage:             queued/running
  Job File:          /scratch/tm70/xxx/postscript/1706.gadi-pbs.json
  ------------- Sync Info -------------
  Job ID:            1707.gadi-pbs
  Job Dependencies:  1706.gadi-pbs
  Stage:             queued
  Job File:          /scratch/tm70/xxx/sync/1707.gadi-pbs.json
========================================

All post-processing jobs (collate, postscript, and sync) can be disabled in config.yaml. If enabled, they will be submitted all together after the model run.

To remove killed jobs from the payu status output run:

payu status --update

3. Write minimal provenance information into restart directory

Payu now records minimal provenance information in the restart directory as restart_metadata.yaml.

name: mom6
payu_current_run: 3
experiment_uuid: xxx
payu_run_id: yyy0000
model_finish_time: '1901-03-22T00:00:00'
parent_experiment: <parent_experiment_uuid_if_exists_in_metadata>

This provides important experiment provenance information if the restart directory is moved out of the original archive directory.

For any information that cannot be retrieved (e.g., when runlog or metadata is disabled), the value will be recorded as __FAILED_TO_RETRIEVE__.

4. Validate the parent uuid format before adding it to metadata

In payu 1.3.6, the parent uuid is validated to ensure that it is in UUID4 format before being added to metadata.yaml, e.g.,

payu checkout -b new_branch ctrl_branch -p FAKE_UUID

will raise an error because FAKE_UUID does not match UUID4 format.

5. Bug fix: empty-string timestamp causing errors

As reported by the community and ACCESS-NRI model developers, the cached PBS node file may contain an empty timestamp (""), which can cause payu to error out.

This bug has been fixed in payu 1.3.6. When an empty-string timestamp is found, payu will generate a new cached pbsnode.json file.

Support

Replies to this topic are disabled.

If you have specific questions about this release follow the guidelines for requesting help from ACCESS-NRI.

If you have questions about payu create a topic in a category that best matches the model you are using, or in the Technical category and tag it with payu. If you require assistance follow the guidelines for requesting help from ACCESS-NRI.

Credits

Developments by @Qianhui_Chen, @minghangli.

Reviewers included @jo-basevi, @Aidan, @spencerwong.