# Payu: a workflow manager for some ACCESS models

**URL:** <https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098>\
**Category:** ACCESS-NRI Releases\
**Tags:** nci, release, payu, nri-updates\
**Created:** [17 August 2023 06:07 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098 "2023-08-17T06:07:22Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [17 August 2023 06:07 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/1 "2023-08-17T06:07:22Z")

</div>

# What is this?

A topic that will be used to announce updates to `payu` at [NCI](http://nci.org.au).

# What is payu?

`payu` is the tool used to run a number of ACCESS models on NCI hardware

> **[GitHub - payu-org/payu: A workflow management tool for numerical models on...](https://github.com/payu-org/payu)**
>
> A workflow management tool for numerical models on the NCI computing systems

# How is this topic used?

This topic will be the place where updates to `payu` at NCI will be announced, with details of what those updates entail.

# What should I do to be notified of any updates?

You can [watch this topic](https://forum.access-hive.org.au/docs?topic=792), and be notified by email of _every_ update.

# Where do I ask questions about releases?

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

---

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [17 August 2023 06:35 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/2 "2023-08-17T06:35:28Z")

</div>

# 17/08/2023

`payu` has been updated to v1.0.29 in the [`conda/analysis3-unstable` `conda` environment](http://climate-cms.wikis.unsw.edu.au/Conda) maintained by the CLEX CMS team.

> **[Release 1.0.29 · payu-org/payu](https://github.com/payu-org/payu/releases/tag/1.0.29)**
>
> Fix solo models required libs error (#354)

## Updates:

- Added support for `module use` to add module paths in `config.yaml` [Issue #347](https://github.com/payu-org/payu/issues/347)
- Removed redundant code intended to support `MVAPICH` [Issue #350](https://github.com/payu-org/payu/issues/350)
- Supports models built with #spack [Issue #341](https://github.com/payu-org/payu/issues/341)
- Fixed bug introduced with #spack support update (version 1.0.28) and cleaned up internal code logic [Issue #354](https://github.com/payu-org/payu/issues/354)

## Notes

#### Enhanced module support

`payu` already supported loading user-specified modules when a model is run.

`payu` now also supports `module use` paths in the `config.yaml` file. This is [documented](https://payu.readthedocs.io/en/latest/config.html#miscellaneous) but in essence if you have an existing `config.yaml` that has something like this:

```yaml
modules:
  - netcdf-c-4.9.0
  - parallel-netcdf-1.12.3
  - xerces-c-3.2.3

```

that only works if you you first do

```auto
module use /path/to/module/directory

```

then the `module use` is no longer required, it can be incorporated in the `config.yaml` by changing to:

```yaml
modules:
   use:
      - /path/to/module/directory
   load:
      - netcdf-c-4.9.0
      - parallel-netcdf-1.12.3
      - xerces-c-3.2.3

```

This has a couple of benefits:

1. It makes the configuration more portable. Someone else can clone your configuration and run it without additional steps required
2. The paths specified in the `use` section are parsed for storage points and `/g/data/` and `/scratch` directories are automatically added to `-l storage` options when `payu` submits the job to the PBS queue

Note: the previous syntax is still supported.

### Support for spack built executables

For the most part `spack` uses a mechanism to embed within an executable information about the location of library dependencies that were used to compile the executable.

In these cases it doesn’t rely on some of the module inspection `payu` does to make sure the correct modules are loaded. For libraries outside the `/apps` hierarchy `payu` no longer attempts to do this sort of inspection.

### Credits

This work was all done by @jo-basevi. Thanks!

---

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [14 September 2023 07:51 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/3 "2023-09-14T07:51:08Z")

</div>



---

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [16 February 2024 06:53 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/4 "2024-02-16T06:53:17Z")

</div>

# 18/02/2024

**NOTE** : ACCESS-NRI is now supporting `payu` in a dedicated `conda` environment in the `vk83` project. `vk83` is the project ACCESS-NRI will be using to release all climate models in the future. See [below for more information](#how-to-access-8).

This release adds support for experiment UUIDs and marks a major change in the way `payu` names the `work` and `archive` laboratory directories to support experiment UUIDs and `git` branching. The naming scheme incorporates a portion of the experiment UUID and the `git` branch name which prevents namespace clashes, and allows experiments with the same name to co-exist. More importantly it means `git` branches can be utilised seamlessly as independent experiments.

As a result the minor version has been incremented (from 1.0 to 1.1): the changes are backward compatible, but from now on by default new experiments will use the new naming scheme.

> **[Release Experiment IDs and Branching · payu-org/payu](https://github.com/payu-org/payu/releases/tag/1.1)**
>
> Notable changes
> This release adds support for Experiment UUIDs and marks a major change in the way payu names the work and archive laboratory directories to support Experiment IDs and git branching...

## Updates:

- Automatic generation of unique Experiment IDs ([UUIDv4](https://en.wikipedia.org/wiki/Universally_unique_identifier))
- Automatic creation and population of `metadata.yaml` file (compatible with [ACCESS-NRI Intake Catalogue](https://access-nri-intake-catalog.readthedocs.io/en/latest/))
- Uniquely named `work` and `archive` directories, allows the same experiment repository to be used for multiple unique experiments in separate branches
- New `payu checkout` command: wraps `git checkout` to facilitate changing between experiments stored in separate branches
- New `payu branch` command: lists available branches, and their experiments, that exist in the repository
- New`payu clone` command: wrapper for `git clone` that updates metadata and makes sure the experiment directory is correctly configured
- New `sync` support: uses `rsync` to copy outputs and restarts to a specified location (can be a local long-term storage disk, or a completely remote machine). Incredibly useful functionality when model outputs are saved to short-term storage, e.g. `/scratch` at NCI
- Date based restart pruning: `payu` now supports pandas style date/frequency syntax to specify what restarts should be retained

## Notes

### Experiment UUIDs

`payu` now automatically generates unique experiment UUIDs. These are [UUIDv4](https://en.wikipedia.org/wiki/Universally_unique_identifier) format, which is typically represented as a 128bit hexadecimal number, e.g.

```auto
550e8400-e29b-41d4-a716-446655440000

```

They are guaranteed to be unique within any reasonable computation effort, and so can be confidently used to identify and track experiments. UUIDs are not human friendly, and are designed to be used by software, but the first 8 digits of the experiment UUID is used to uniquely name experiment laboratory `archive` and `work` directories.

### Branches

`git` branches are now explicitly supported by `payu`, and form a crucial part of the updated workflow. This means a single `control` directory (which is a `git` repository) can contain multiple independent experiments, and it is possible to switch between experiments, though only one experiment can be active and running at any one time.

See the [payu tutorial for more information on branches](https://forum.access-hive.org.au/t/access-om2-payu-tutorial/1750#branches-4).

### Experiment naming

An experiment name is used to identify the experiment inside the `work` and `archive` sub-directories inside the _laboratory_.

The experiment name historically would default to the name of the _control directory_. This is still supported for experiments with pre-existing archived outputs. To support git branches and ensure uniqueness in shared archives, the new default behaviour is to add the branch name and a short version of the experiment UUID to the name of the _control directory_ when creating experiment names.

See the [payu tutorial for more detail](https://forum.access-hive.org.au/t/access-om2-payu-tutorial/1750#experiment-naming-6).

### Syncing

`payu` now supports syncing of an experiment archive to another filesystem, either local or remote. There are a number of [configuration options](https://payu.readthedocs.io/en/latest/config.html#postprocessing) to customise what is sync’ed and when. See the [payu tutorial for more detail](https://forum.access-hive.org.au/t/access-om2-payu-tutorial/1750#branches-4).

### Restart pruning

`payu` now supports specifying which restarts to retain using date-based frequencies. This allows restarts pruning based on time units. For example setting

```yaml
restart_freq: 5YS

```

will only save the first restart of every fifth year, with the rest deleted.

See the [payu tutorial for more detail](https://forum.access-hive.org.au/t/access-om2-payu-tutorial/1750#restart-pruning-9).

### How to access payu

All ACCESS-NRI models and critical supporting software such as `payu` is located in `/g/data/vk83` on NCI. It is necessary to be a member of the [`vk83`](https://my.nci.org.au/mancini/project/vk83) project to use ACCESS-NRI supported versions of `payu`. See NCI Documentation for more information about [how to join a project](https://opus.nci.org.au/display/Help/How+to+connect+to+a+project)

To access `payu` version 1.1:

```auto
module use /g/data/vk83/modules
module load payu/1.1

```

`payu` is installed on `gadi` using an [automated deployment process](https://github.com/ACCESS-NRI/payu-condaenv). This ensures a consistent software environment and the design allows for multiple versions of `payu` to be maintained, so if there are changes which are incompatible with your experiment you can use an older compatible `payu` version.

### Credits

All `payu` development was done by @jo-basevi. Deployment by @TommyGatti with assistance from @jo-basevi and @harshula. Thanks!

---

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [12 April 2024 04:49 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/5 "2024-04-12T04:49:15Z")

</div>

# 12/04/2024

`payu` has been updated to v1.1.3

> **[Release Updates to Packaging Logic, Documentation and Bug Fixes · payu-org/payu](https://github.com/payu-org/payu/releases/tag/1.1.3)**
>
> What's Changed
> 
> Update to packaging tools and logic by @dougiesquire in #403
> Fix PyPI upload by @dougiesquire in #407
> Fix readthedocs configuration by @aidanheerdegen in #408
> Update tag\_prefix in \_...

## Updates

- Bug fixes/improvements for the new branching capabilities ([#423](https://github.com/payu-org/payu/pull/423), [#425](https://github.com/payu-org/payu/pull/425), [#435](https://github.com/payu-org/payu/pull/435))
- Added capability to deal with multiple restarts for ACCESS-OM3 experiments ([#432](https://github.com/payu-org/payu/pull/432))
- Prevent UM restart files being archived to output [#415](https://github.com/payu-org/payu/pull/415)
- Packaging, CI and documentation updates. `payu` now correctly reports it’s own version ([#403](https://github.com/payu-org/payu/pull/403), [#407](https://github.com/payu-org/payu/pull/407), [#408](https://github.com/payu-org/payu/pull/408), [#410](https://github.com/payu-org/payu/pull/410), [#411](https://github.com/payu-org/payu/pull/411))
- Metadata handling updates ([#434](https://github.com/payu-org/payu/pull/434), [#427](https://github.com/payu-org/payu/pull/427))

## Notes

### How to access payu

All ACCESS-NRI models and critical supporting software such as `payu` is located in `/g/data/vk83` on NCI. It is necessary to be a member of the [`vk83`](https://my.nci.org.au/mancini/project/vk83) project to use ACCESS-NRI supported versions of `payu`. See NCI Documentation for more information about [how to join a project](https://opus.nci.org.au/display/Help/How+to+connect+to+a+project)

To access `payu` version 1.1.3:

```auto
module use /g/data/vk83/modules
module load payu/1.1.3

```

`payu` is installed on `gadi` using an [automated deployment process](https://github.com/ACCESS-NRI/payu-condaenv). This ensures a consistent software environment and the design allows for multiple versions of `payu` to be maintained, so if there are changes which are incompatible with your experiment you can use an older compatible `payu` version.

This new version [has been deployed to `vk83`](https://github.com/ACCESS-NRI/payu-condaenv/releases/tag/1.1.3) and is now the default version. It contains a number of bug fixes and improvements and is the recommended version to use. The previously deployed version (1.1) is [deprecated and will be removed](https://github.com/payu-org/payu/issues/429) when the next version of `payu` is deployed, or within 3 months, whichever is sooner.

### Credits

`payu` development was primarily by @jo-basevi. Other contributions from @dougiesquire, @MartinDix and @Aidan. Deployment by @TommyGatti, @dougiesquire, @jo-basevi and @harshula. Thanks!

---

<div class="post-metadata">

**Author:** ![jo-basevi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/jo-basevi/32/1438_2.png) [@jo-basevi](https://forum.access-hive.org.au/u/jo-basevi)\
**Post date:** [27 August 2024 23:12 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/6 "2024-08-27T23:12:48Z")

</div>

# 23/08/2024

🚀 **Release:** 1.1.5

[https://github.com/payu-org/payu/releases/tag/1.1.5](https://github.com/payu-org/payu/releases/tag/1.1.5)

## Getting Started:

This version of payu can now be accessed on `gadi`:

```auto
module use /g/data/vk83/modules
module load payu/1.1.5

```

## Updates

✅ Get module executables from paths added by loaded environment modules ([#439](https://github.com/payu-org/payu/pull/439), [#482](https://github.com/payu-org/payu/pull/482))  
✅ Allow generic tracers with `cesm_cmeps` driver ([#433](https://github.com/payu-org/payu/pull/433))  
✅ Remove unnecessary UM config files ([#455](https://github.com/payu-org/payu/pull/455))  
✅ Replace UM um\_env.py configuration file with yaml file ([#459](https://github.com/payu-org/payu/pull/459))  
✅ User scripts and postscript updates to error handling, environment variables and shell features ([#452](https://github.com/payu-org/payu/pull/452), [#467](https://github.com/payu-org/payu/pull/467), [#445](https://github.com/payu-org/payu/pull/445))  
✅ Adding date based pruning for ACCESS-ESM1.5 ([#465](https://github.com/payu-org/payu/pull/465))  
✅ Changes to manifest logic ([#475](https://github.com/payu-org/payu/pull/475))  
✅ Enable seperate ice\_history.nml & cice\_in.nml settings ([#483](https://github.com/payu-org/payu/pull/483))  
✅ Replace CICE start date calculations ([#484](https://github.com/payu-org/payu/pull/484))  
✅ Add command line flag to disable metadata generation and related commits ([447](https://github.com/payu-org/payu/pull/447))

For a full list of pull requests that includes additional bug-fixes, see the [Payu Github 1.1.5 release](https://github.com/payu-org/payu/releases/tag/1.1.5).

## Notes

### Python version 3.10

Released payu environments modules from `payu/1.1.5` onwards are based on python version 3.10.

### UM um\_env.py files

UM `um_env.py` configuration files are no longer supported and should be replaced with `um_env.yaml` files. Existing `um_env.py` files can be converted to yaml files, using this [script](https://github.com/ACCESS-NRI/esm1.5-scripts/blob/main/config-files/UM/um_env_to_yaml.py).

### Loading model executables using model modules

Payu can now find model executables by searching paths added to `$PATH` by model environment modules. This simplifies the `config.yaml`, as only the name of the executable is required and changing model versions is simpler and less error prone. For example,

```yaml
# Modules for loading model executables
modules:
  use:
      - /g/data/vk83/modules
  load:
      - access-esm1p5/2024.05.0

...
- name: ocean
   model: mom
   exe: fms_ACCESS-CM.x

```

Previously, the executable had to be specified as a full path. For executables built by spack, these included a hash in the path, and were long and complicated. For example:

```yaml
    exe: /g/data/vk83/apps/spack/0.22/restricted/ukmo/release/linux-rocky8-x86_64_v4/intel-19.0.3.199/mom5-git.access-esm1.5_2024.06.20_access-esm1.5-wxxrc3ivrjz76yx565ddkuuiwoqpalko/bin/fms_ACCESS-CM.x

```

Loaded modules in `config.yaml` must be unique to ensure the correct model executables is used. This means modules must be specified with a version, and modules of the same name and version can not be found in multiple module directories.

### Updates to user processing scripts

Payu now exports some current run information to environment variables so they can be accessed in post-processing scripts:

- `PAYU_CURRENT_RUN` - The current run number, e.g. 0 for first run, 1 for the second run
- `PAYU_ARCHIVE_DIR` - Full path to the archive directory - this contains all the outputs and restarts subdirectories
- `PAYU_CURRENT_OUTPUT_DIR`, `PAYU_CURRENT_RESTART_DIR` - Full path to the current output and restart directories, e.g. for first run, it would be `/path/to/archive/output000` and `path/to/archive/restart000`

[Userscript and postscript commands calls](https://payu.readthedocs.io/en/latest/config.html#postprocessing) can now also include shell-specific values such as file re-directions, pipes and environment variables (which are expanded). So it’s now possible to run commands such as:

```yaml
runscript:
    setup: echo "some_data" > input.txt
    archive: some_script.sh $PAYU_CURRENT_OUTPUT_DIR

```

If users scrips exit with an error payu run execution halts. If this is not desirable, error handling will need to be added to post-processing scripts. Previously only warnings were issued if a user scripts exited with an error.

### Changes to manifest logic

Payu manifests store information of files in the `work` directory of an experiment. They are used to track changes to files over an experiment for experiment provenance, and ensure an experiment run can be reproduced. There are three manifests types: executable files, input and restart files (`exe.yaml`, `input.yaml` and `restart.yaml` respectively).

The logic for updating manifests and enforcing reproducibility has been greatly simplified:

- Stored manifests from previous runs are used as the source of truth for full (`md5`) hashes, for all manifest types. Previously this was the case for only input manifests.
- Fast change sensitive hashes, by default [`binhash`](https://github.com/ACCESS-NRI/yamanifest/blob/7f9aaaddc2d31ebe1cd1b9d92ab2df349e35ba82/yamanifest/hashing.py#L48-L65), are calculated at each `payu setup`. If a fast hash matches the value in the stored manifest the full hash from the stored manifest is used. This avoids re-calculating slow `md5` hashes where possible.
- All changes to `config.yaml` are now correctly detected. For example a different executable path or new input file paths.
- `scaninputs` option has been removed. This allowed existing file paths to change but not scan for new inputs. This means only paths configured in `config.yaml` or found through searching input and restart directories, are added to the `work` directory.

#### Enforcing reproducibility

Setting `reproduce` to `true` in `config.yaml` or via a command-line option will check and make sure files have not changed since the previous run. It is possible to set `reproduce` for each manifest type separately.

When `reproduce` for a manifest type is set to true, payu will refuse to run if:

- Full hash changes: calculated `md5` hashes differ to full hash in stored manifest
- New files: if files are found in the `work` directory that were not in the stored manifest
- Missing files: files in the stored manifest are not present in the `work` directory

If a full path to a file has changed or a fast hash has been changed, but there’s a match with the stored full hash (so it is effectively the same file), the manifest will be updated.

Changes to `config.yaml` are now correctly picked up. For example a different executable path or new input file paths. Previously specifying reproducibility would only add paths in the manifests to the work directory and raise errors if those files were modified.

For more information on manifests, see the payu documentation for [configuring your experiment](https://payu.readthedocs.io/en/latest/config.html#manifests) and [manifests content and tracking](https://payu.readthedocs.io/en/latest/manifests.html).

### Disabling metadata + UUID generation and commits

To update manifests without auto-updating metadata:

```auto
payu setup --metadata-off 

```

The `--metadata-off/-m` command line flag was added to make it more convenient to update released configurations which do not include a UUID.

Previously the only way to disable generating a new UUID and updated `metadata.yaml` file and related git commits was via `config.yaml`:

```yaml
metadata:
    enable: false

```

This option is only available with the `payu setup` and `payu sweep` commands. Disabling metadata for `payu run` still needs to be done via `config.yaml`.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Development was by @jo-basevi, @spencerwong, @dougiesquire, @anton, @Aidan and @TommyGatti.

---

<div class="post-metadata">

**Author:** ![jo-basevi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/jo-basevi/32/1438_2.png) [@jo-basevi](https://forum.access-hive.org.au/u/jo-basevi)\
**Post date:** [4 February 2025 22:27 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/7 "2025-02-04T22:27:04Z")

</div>

# 03/02/2025

🚀 **Release:** 1.1.6

[https://github.com/payu-org/payu/releases/tag/1.1.6](https://github.com/payu-org/payu/releases/tag/1.1.6)

## Getting Started

This version of payu can now be accessed on `gadi`:

```shell
module use /g/data/vk83/modules
module load payu/1.1.6

```

## Updates

✅ Fix default shortpath project bug ([#504](https://github.com/payu-org/payu/pull/504))  
✅ Add minimum payu version `config.yaml` option ([#511](https://github.com/payu-org/payu/pull/511))  
✅ Support symbolic links in input files ([#518](https://github.com/payu-org/payu/pull/518))  
✅ Add command line option for `payu clone` to start from commit hash/tag ([#515](https://github.com/payu-org/payu/pull/515))  
✅ Support relative restart paths with `payu clone` command ([#516](https://github.com/payu-org/payu/pull/516))  
✅ ACCESS-OM3: Add checks of input/output parallel processor layout to cmeps driver used for ACCESS-OM3 ([#496](https://github.com/payu-org/payu/pull/496))  
✅ CABLE: Add staged driver for CABLE configurations ([#461](https://github.com/payu-org/payu/pull/461))  
✅ CICE5: Don’t require cice\_in.nml namelist in restarts - this fixes errors when starting from ACCESS-OM2 restarts ([#507](https://github.com/payu-org/payu/pull/507))  
✅ CICE5: Fix restart pointers ([#535](https://github.com/payu-org/payu/pull/535))  
✅ CICE4: Add log file compression ([#532](https://github.com/payu-org/payu/pull/532) and [#542](https://github.com/payu-org/payu/pull/542))  
✅ CICE4: Check restart file dates ([#539](https://github.com/payu-org/payu/pull/539))

For a full list of pull requests that includes additional bug fixes, see the [Payu Github 1.1.6 release](https://github.com/payu-org/payu/releases/tag/1.1.6).

## Notes

### Containerised conda environment

The conda environment for `payu/1.1.6` now runs inside a singularity container. The deployment is based on the work done by CLEX CMS on [Containerised Conda environments](https://github.com/coecms/cms-conda-singularity). This dramatically reduces the inode usage of each conda environment as the environment is compressed into a SquashFS file that is added as an overlay to a singularity container at runtime.

If there are problems with running commands once the payu module is loaded, please create an issue on the [`ACCESS-NRI/model-release-condaenv` repository](https://github.com/ACCESS-NRI/model-release-condaenv/issues).

### Model log compression option for CICE4

CICE4 model log files can take up a significant amount of storage. By default payu now compresses some log files into a tarball during archival.

No changes are necessary to enable this option, but it can be configured in `config.yaml` under the `archive` section, e.g.

```yaml
archive:
  compress_logs: false # Default is true

```

Currently log file compression is only implemented for the CICE4 model.

### Added Beta CABLE driver

Added a driver for running multi-stage CABLE configurations. Supports arbitrary spin-up configurations through the `staged_cable` driver. See the [cable-offline-configs](https://github.com/CABLE-LSM/cable-offline-configs) for instructions on using the driver. Driver internals are still being iterated, but user experience should remain consistent.

### Payu clone/checkout updates

With the `payu clone` command, to create a new git branch starting from a specific git tag or commit, use `-s/--start-point` flag:

```shell
payu clone -b <NEW_BRANCH> -s <COMMIT_HASH|TAG> <REPOSITORY_URL> <EXPERIMENT_NAME>

```

When checking out a new branch starting from a restart directory, a relative path can now be used, e.g.:

```shell
payu checkout -b expt -r archive/restart002

```

Previously the full path to the restart directory was required.

### Payu minimum version

As some configurations might require features in later versions of payu to run, there’s an `config.yaml` option to set a minimum version of payu:

```yaml
payu_minimum_version: 1.1.6

```

This will run a check at the start of `payu setup` to compare this version with the currently running payu version. Note that this check will only run with payu versions `>=1.1.6` so it will be more useful in the future once there are more versions of payu available that support this feature.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Development was by @jo-basevi, @anton, @spencerwong and @lachlanswhyborn.

---

<div class="post-metadata">

**Author:** ![jo-basevi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/jo-basevi/32/1438_2.png) [@jo-basevi](https://forum.access-hive.org.au/u/jo-basevi)\
**Post date:** [23 July 2025 05:04 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/8 "2025-07-23T05:04:36Z")

</div>

# 23/07/2025

🚀 **Release:** 1.1.7

[https://github.com/payu-org/payu/releases/tag/1.1.7](https://github.com/payu-org/payu/releases/tag/1.1.7)

## Getting Started

This version of payu can now be accessed on `gadi`:

```shell
module use /g/data/vk83/modules
module load payu/1.1.7

```

## Updates

✅ UM: Add date-based restart pruning. This supports date-based restart pruning for ESM1.5 and ESM1.6 AMIP configurations as previously it was only supported if there was a MOM subcomponent ([#553](https://github.com/payu-org/payu/pull/553))  
✅ ACCESS-OM3: Remove overriding user settings for restart output frequency in model driver ([#556](https://github.com/payu-org/payu/pull/556))  
✅ ACCESS-OM3: Add date-based restart pruning ([#562](https://github.com/payu-org/payu/pull/562))  
✅ ACCESS-OM3: Commit MOM output documentation into the `docs` folder of the configuration ([#567](https://github.com/payu-org/payu/pull/567))  
✅ ACCESS-ESM1.6: Add new model driver to run the ACCESS-ESM1.6 configurations that are under development ([#563](https://github.com/payu-org/payu/pull/563))  
✅ ACCESS-ESM1.6: Add cable namelists as optional configuration files ([#570](https://github.com/payu-org/payu/pull/570))  
✅ ACCESS-ESM1.6: Incorporate CICE5 into driver ([#585](https://github.com/payu-org/payu/pull/585))  
✅ ACCESS-ESM1.6: Don’t write start date to CICE5 namelist ([#596](https://github.com/payu-org/payu/pull/596))  
✅ ACCESS-ESM1.6: Refactor parsing restart date-times and test methods to be consistent between ESM1.6 components ([#599](https://github.com/payu-org/payu/pull/599))  
✅ Deprecate MATM model driver ([#573](https://github.com/payu-org/payu/pull/573))  
✅ Add more error messaging when cloning from a tag/commit [#575](https://github.com/payu-org/payu/pull/575)  
✅ Add `exe_prefix` configuration option to documentation ([#582](https://github.com/payu-org/payu/pull/582))  
✅ Fix bug that would run archive user scripts after the collation job was submitted ([#584](https://github.com/payu-org/payu/pull/584))  
✅ Allow `repeat` config option to be used with previous `restart` defined in `config.yaml` ([#580](https://github.com/payu-org/payu/pull/580))  
✅ Always map-by `node` when `npernode` is set in `config.yaml` ([#594](https://github.com/payu-org/payu/pull/594))  
✅ Allow starting from a restart in `config.yaml` with a non-zero counter which is used for naming output directories ([#604](https://github.com/payu-org/payu/pull/604)).  
✅ Prepend a launcher script to the Payu generated PBS commands, if it exists ([#607](https://github.com/payu-org/payu/pull/607))  
✅ List loaded environment modules in error logs after all modules are loaded ([#608](https://github.com/payu-org/payu/pull/608))

For a full list of pull requests that includes additional bug fixes, see the [Payu Github 1.1.7 release](https://github.com/payu-org/payu/releases/tag/1.1.7).

## Notes

### Automatically committing MOM6 Parameter output files for ACCESS-OM3

MOM output documentation, e.g. `MOM_parameter_doc.*` files, are committed back into the `docs/` folder of the configuration. This is run when model outputs are archived by payu and the commit message is `payu archive: documentation of MOM6 run-time configuration`. If `runlog` is set to `False`, this will also disable these commits.

### Customising the MPI run command

The `exe_prefix` option adds a string immediately before the model executable in the MPI run command. This can be useful for configuring profilers or valgrind. For example, given the following in `config.yaml`:

```yaml
name: atmosphere
model: um
exe: um_hg3.exe
exe_prefix: aps -r <aps_output_dir>

```

The generated run command would then look like `mpirun <mpi_flags> aps -r <aps_output_dir> <path/to/um_hg3.exe>`.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Development was by @spencerwong, @anton, @jo-basevi and @manodeep  
Reviewers included @dougiesquire, @cbull, @Aidan, @TommyGatti and @tmcadam  
Documentation update by @Benoit

---

<div class="post-metadata">

**Author:** ![jo-basevi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/jo-basevi/32/1438_2.png) [@jo-basevi](https://forum.access-hive.org.au/u/jo-basevi)\
**Post date:** [5 November 2025 04:50 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/9 "2025-11-05T04:50:01Z")

</div>

# 05/11/2025

🚀 **Release:** 1.2.0

[https://github.com/payu-org/payu/releases/tag/1.2.0](https://github.com/payu-org/payu/releases/tag/1.2.0)

## Getting Started

This version of payu can now be accessed on `gadi`:

```shell
module use /g/data/vk83/modules
module load payu/1.2.0

```

## Updates

✅ ACCESS-OM3: Autodetect nuopc.runconfig for restart-only collate ([#621](https://github.com/payu-org/payu/pull/621))  
✅ ACCESS-OM3: Support archiving processor-split restart files ([#601](https://github.com/payu-org/payu/pull/601))  
✅ ACCESS-ESM1.6: Create symlinks rather than copy CICE5 input files ([#635](https://github.com/payu-org/payu/pull/635))  
✅ CICE: Tidy up handling history\_dir and restart\_dir ([#618](https://github.com/payu-org/payu/pull/618))  
✅ ROMS: Add new model driver for [ROMS](https://www.myroms.org) model ([#624](https://github.com/payu-org/payu/pull/624) [#627](https://github.com/payu-org/payu/pull/627) [#630](https://github.com/payu-org/payu/pull/630))  
✅ Raise exception when repository is in a detached HEAD state ([#626](https://github.com/payu-org/payu/pull/626))  
✅ Add telemetry for payu runs and payu status command ([#558](https://github.com/payu-org/payu/pull/558))

For a full list of pull requests, see the [Payu Github 1.2.0 release](https://github.com/payu-org/payu/releases/tag/1.2.0).

## Notes

### Telemetry for payu runs

Telemetry is now enabled for payu runs. This collects experiment metadata, run state, approximate resource usage, and payu manifests. This data is collected by ACCESS-NRI for resource planning, reporting aggregated statistics to funding bodies, and to support plans for improved experiment provenance. For more information on the tracking and reporting ACCESS-NRI does, see [ACCESS-NRI Services Privacy Notice](https://reporting.access-nri-store.cloud.edu.au/privacy).

### Monitoring payu runs with `payu status`

There is a new `payu status` command to display information about payu runs. By default, this displays a summary of the latest run which includes the latest stage of the payu run, filepaths to the PBS standard output/error files, and exit statuses of model run MPI commands and the overall payu run. For more information on various command line options, see the [payu status documentation](https://payu.readthedocs.io/en/1.2.0/usage.html#monitoring-payu-jobs) or run `payu status --help`.

The information parsed by `payu status` is from JSON files that are stored under the `payu_jobs` sub-directory in the `archive` directory, and are updated by payu at various stages of a payu run (e.g. when jobs are queued, during setup and post model runs). Note that these job files replace the `job.yaml` files that were previously created in archive output directories.

Currently, the `payu status` only supports `payu run` jobs not `collate`, `sync` or `postscript` jobs.

### New ROMS model driver

There is now a new model driver added for the [ROMS](https://www.myroms.org/wiki/Regional_Ocean_Modeling_System_(ROMS)) hydrodynamic model. An example `upwelling` configuration can be found in the [ACCESS-Community-Hub/roms-config](https://github.com/ACCESS-Community-Hub/roms-configs/tree/release-upwelling%2Btestcase) repository.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Development was by @Aidan, @anton, @jo-basevi, @minghangli, @spencerwong and @pcouto

Reviewers included @dougiesquire, @helen and @tmcadam

---

<div class="post-metadata">

**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\
**Post date:** [16 January 2026 04:17 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/10 "2026-01-16T04:17:19Z")

</div>

# 16/01/2026

🚀 **Release:** 1.2.1

[https://github.com/payu-org/payu/releases/tag/1.2.1](https://github.com/payu-org/payu/releases/tag/1.2.1)

## Getting Started

This version of payu can now be accessed on `gadi`:

```shell
module use /g/data/vk83/modules
module load payu/1.2.1

```

## Updates

✅ Payu python executable added to PATH ([#491](https://github.com/payu-org/payu/pull/491))  
✅ Support divergent mppnccombine versions([#647](https://github.com/payu-org/payu/pull/647))  
✅ Sync: Default exclude\_uncollated to false ([#649](https://github.com/payu-org/payu/pull/649))  
✅ CICE: Only copy cice\_in to restart when using CICE4 ([#644](https://github.com/payu-org/payu/pull/644))

For a full list of pull requests, see the [Payu Github 1.2.1 release](https://github.com/payu-org/payu/releases/tag/1.2.1).

## Notes

### Python executable in PATH

This change allows [user scripts to run with the same python interpreter as the payu process](https://github.com/payu-org/payu/issues/490).

### Set mppnccombine flags

The mppnccombine program collates tiled MOM model outputs as a post-processing step. There are now versions of this program in use that have different command line options which [caused errors](https://forum.access-hive.org.au/t/cannot-find-mppncombine-after-compiling-access-om2-wombatlite/5477). This is fixed in this release by detecting the version of mppnccombine and using the appropriate command line options.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Development was by @jo-basevi, @dougiesquire, @spencerwong

Reviewers included @jo-basevi,@Aidan, @anton

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [21 April 2026 04:41 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/11 "2026-04-21T04:41:40Z")

</div>

# 21/04/2026

🚀 **Release:** 1.3.0

[https://github.com/payu-org/payu/releases/tag/1.3.0](https://github.com/payu-org/payu/releases/tag/1.3.0)

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.0

```

## Updates

✅ PBS job names for user `postscript` are now standardised to `payu_postscript`. `payu sweep` also cleans up postscript log files ( [#682](https://github.com/payu-org/payu/pull/682)).  
✅ Adds `base_path` as a fallback when `path` is not set in sync. The full sync destination becomes `base_path/experiment_name/`( [#670](https://github.com/payu-org/payu/pull/670)).  
✅ Hidden files and directories are now excluded in manifests by default. To include them, please set `ignore` to an empty list `[]` in `config.yaml` ( [#677](https://github.com/payu-org/payu/pull/677)).  
✅ An interactive mode for `payu clone` setup ([#703](https://github.com/payu-org/payu/pull/703)).  
✅ Display current/total PBS queue time in `payu status --update` ([#686](https://github.com/payu-org/payu/pull/686)).  
✅ Experiment time (e.g., 1900-01-03T00:00:00) tracking in `payu status`( [#702](https://github.com/payu-org/payu/pull/702)).  
✅ Cleaner warning message with stacktrace set to false by default. To see full details, please use flag `--stacktrace`([#713](https://github.com/payu-org/payu/pull/713)).  
✅ A new `payu support` command to display your computing environment and machine configuration ([#710](https://github.com/payu-org/payu/pull/710)).

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.0 release](https://github.com/payu-org/payu/releases/tag/1.3.0).

## Notes

### 1. Payu clone interactive mode

You now can run

```sh
payu clone

```

to launch an interactive wizard that guides you through the cloning process. The wizard:

- provides live searching of remote branch names
- autocompletes existing directory names
- adapts follow-up questions based on your previous inputs.

A flowchart of all interactive prompts is available on [ReadTheDoc](https://payu.readthedocs.io/en/latest/usage.html#:~:text=Payu%20clone%20interactive,is%20as%20follows:).

**Example:**  
Previously, to clone an **ACCESS-ESM1.5** configuration from branch `release-preindustrial+concentrations`, into a local directory (`preindustrial_concen`) and name the local branch as `expt1`, we need

```shell
payu clone -b expt1 -B release-preindustrial+concentrations https://github.com/ACCESS-NRI/access-esm1.5-configs preindustrial_concen

```

Now you can achieve the same result using the interactive `payu clone`:  
 ![payu_clone_demo](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/e/eedbf916bc72fb6c8d1c13d84e23a008effb3790.gif)

### 2. A more informative `payu status`

Now we can check how long a payu PBS job has been in queue by running:

```sh
payu status --update

```

For running PBS jobs, a total queued time is displayed.  
Meanwhile, we can also track the progress of a payu run with

```sh
payu status

```

This displays the current experiment time for running jobs and model finish time for archived jobs. This feature is currently supported for:

- ACCESS-OM2
- ACCESS-OM3
- ACCESS-ESM1.5
- ACCESS-ESM1.6
- MOM6

### 3. Easier support and debugging

The new support command

```sh
payu support

```

provides useful diagnostic information, including:

- Payu version and installation path.
- Python version and the full system path where packages are loaded from.
- Loaded modules (e.g., PBS).
- Machine information, such as the Operating System and CPU architecture.

This information will be helpful in debugging environment issues or providing necessary details when reporting an issue.

### 4. Cleaner warning messages

In the latest payu, warning messages are displayed in a cleaner format with no stack trace by default. To see the full details, please use `--stacktrace` flag.

**Example:**

```bash
$ payu status
[WARNING] Example of warning messages.

$ payu status --stacktrace
[WARNING] /home/$USER/payu/subcommands/status_cmd.py:35: UserWarning: Example of warning messages.
  warnings.warn("Example of warning messages.")

```

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments were by @Qianhui_Chen, @minghangli, @jasmeen_kaur.

Reviewers included @jo-basevi, @Aidan, @spencerwong, @anton, @manodeep, @lachlanswhyborn.

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [21 May 2026 22:53 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/12 "2026-05-21T22:53:29Z")

</div>

# 22/05/2026

🚀 **Release:** 1.3.1

[https://github.com/payu-org/payu/releases/tag/1.3.1](https://github.com/payu-org/payu/releases/tag/1.3.1)

> ⚠  
> There is [a bug in this release affecting collation of ocean model output for run 0](https://github.com/payu-org/payu/issues/850).
> 
> Affected models: ACCESS-OM2, ACCESS-OM3
> 
> Recommended solution: use [payu version 1.3.6](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/16)  
> ⚠

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.1

```

## Updates

- Update the documentation on `payu status` by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#734](https://github.com/payu-org/payu/pull/734)
- ACCESS-OM3: Handling optional input files by [@anton-seaice](https://github.com/anton-seaice) in [#729](https://github.com/payu-org/payu/pull/729)
- Payu clone prompts: color palette and view flowchart by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#719](https://github.com/payu-org/payu/pull/719)
- Create collate job file; log mapping into job file by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#722](https://github.com/payu-org/payu/pull/722)

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.1 release](https://github.com/payu-org/payu/releases/tag/1.3.1).

## Notes

### 1. A clearer instruction of `payu status`

The documentation for `payu status` has been updated with instructions on how to continuously monitor jobs. To refresh status automatically, you can use `watch`, for example:

```shell
watch -n 30 payu status

```

refreshes every 30 seconds. Alternatively, a simple loop can be used:

```shell
while true; do payu status; sleep 30; done

```

This keeps a history of status in the terminal and is useful for tracking job progress over time. Note that `payu status` reads information from job files rather than calling `qstat` each time it runs, allowing refreshes with minimal impact on the scheduler.

### 2. Optional auxiliary input directory support for ACCESS-OM3

Payu now automatically copies an optional directory called

```auto
auxilary_input

```

from the control directory into the work directory during setup.  
This is intended for extra input files that are not included via the standard `input`: directive but are instead tracked and versioned by `git` (see [issue #727](https://github.com/payu-org/payu/issues/727) for details). Ideally these extra input files would be small text files, not large binary files. The `auxiliary_input` directory is not intended to replace specifying inputs in `config.yaml`, but to make experiment setup more flexible and still reproducible.

### 3. Improvements on `payu clone`

Following community feedback, the colour palette in `payu clone` adapts user’s terminal theme for better readability.

Now we can view the flowchart in payu by entering a question mark (`?`) during the prompts, in addition to the [documentation](https://payu.readthedocs.io/en/latest/usage.html#:~:text=Interactive%20mode%20will%20prompt%20for%20all%20required%20inputs.%20The%20workflow%20of%20the%20interactive%20mode%20is%20as%20follows).

### 4. Tracking collation process and file mapping

`payu status` now displays both **run** and **collation** job information.  
For example:

```shell
========================================
Run: 0
  ------------- Run Info -------------
  Job ID: 10000.gadi-pbs
  Run ID: aebsc
  Stage: archive
  Total Queue Time: 0h 0m 41s
  Model Finish Time: 1950-02-01T00:00:00
  ::
  Job File: /xxx/.../1deg_jra_ryf-setupTest-hash/payu_jobs/0/run/10000.gadi-pbs.json
  ------------- Collate Info -------------
  Job ID: 20000.gadi-pbs
  Stage: exited
  Total Queue Time: 0h 0m 30s
  Exit Status: 0 (Success)
  Output Log: /home/xxx/1deg_jra55_ry_c.o20000
  Error Log: /home/xxx/1deg_jra_ryf/1deg_jra55_ry_c.e20000
  Job File: /scratch/$PROJECT/$USER/access-om2/archive/1deg_jra_ryf-setupTest-uuid/payu_jobs/0/collate/20000.gadi-pbs.json
========================================

```

Collate job files include a `collate_mapping` dictionary, recording the mapping between uncollated tiles and their corresponding collated files. This is important information for provenance and reproducibility.

For example:

```auto
"collate_mapping": {
    "restart012": {
        "hash_collated_file_1": ["hash_uncollated_tiles_1a", ...],
        "hash_collated_file_2": ["hash_uncollated_tiles_2a", ...], ...
             }
 }

```

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments was by @Qianhui_Chen, @jo-basevi, @anton.

Reviewers included @jo-basevi, @Aidan, @dougiesquire, @anton, @atteggiani.

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [28 May 2026 23:10 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/13 "2026-05-28T23:10:06Z")

</div>

# 29/05/2026

🚀 **Release:** 1.3.2

[https://github.com/payu-org/payu/releases/tag/1.3.2](https://github.com/payu-org/payu/releases/tag/1.3.2)

> ⚠  
> There is [a bug in this release affecting collation of ocean model output for run 0](https://github.com/payu-org/payu/issues/850).
> 
> Affected models: ACCESS-OM2, ACCESS-OM3
> 
> Recommended solution: use [payu version 1.3.6](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/16)  
> ⚠

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.2

```

## Updates

- cesm\_cmeps driver: fix typo auxilary\_input → auxiliary\_input by [@dougiesquire](https://github.com/dougiesquire) in [#747](https://github.com/payu-org/payu/pull/747)
- Fix missing warning in PBS. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#748](https://github.com/payu-org/payu/pull/748)

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.2 release](https://github.com/payu-org/payu/releases/tag/1.3.2).

## Notes

### 1. Fix the typo `auxilary_input` → `auxiliary_input`

In payu 1.3.1, we introduced optional auxiliary input directory support for ACCESS-OM3. Due to a typo, it was mistakenly written as `auxilary_input` with an `i` missing. In payu 1.3.2, this has been corrected to `auxiliary_input` , and payu will now automatically copy the directory as expected.

### 2. Capture warning in PBS

In payu 1.3.0, we introduce a cleaner format of warning message, with no stacktrace shown by default. However, warnings were not being captured in PBS jobs. In payu 1.3.2, warnings are now also captured in PBS `std.err` output.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments was by @Qianhui_Chen, @dougiesquire.

Reviewers included @jo-basevi, @Aidan, @dougiesquire, @anton.

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [20 July 2026 06:11 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/14 "2026-07-20T06:11:06Z")

</div>

# 20/07/2026

🚀 **Release:** 1.3.3 & 1.3.4

Payu 1.3.3 introduced new features and usability improvements. Shortly after its release, an upstream change in the `filelock` package required a small compatibility fix, which is included in payu 1.3.4. We therefore recommend that all users load **payu/1.3.4**.

[https://github.com/payu-org/payu/releases/tag/1.3.3](https://github.com/payu-org/payu/releases/tag/1.3.3)  
[https://github.com/payu-org/payu/releases/tag/1.3.4](https://github.com/payu-org/payu/releases/tag/1.3.4)

> ⚠  
> There is [a bug in this release affecting collation of ocean model output for run 0](https://github.com/payu-org/payu/issues/850).
> 
> Affected models: ACCESS-OM2, ACCESS-OM3
> 
> Recommended solution: use [payu version 1.3.6](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/16)  
> ⚠

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.4

```

## Updates

- Add option to specify experiment name prefix in `config.yaml`. by @Qianhui_Chen in [#751](https://github.com/payu-org/payu/pull/751)
- Check user access to project before submitting PBS job. by @Qianhui_Chen in [#753](https://github.com/payu-org/payu/pull/753)
- Update status to `running` when `expt.collate()` is running by @Qianhui_Chen in [#758](https://github.com/payu-org/payu/pull/758)
- Create archive symlink in `payu setup`; show experiment paths in `payu status --paths` by @Qianhui_Chen in [#768](https://github.com/payu-org/payu/pull/768)
- Update payu telemetry by @atteggiani in [#726](https://github.com/payu-org/payu/pull/726)
- Configure warning formatting and log-level by @Qianhui_Chen in [#750](https://github.com/payu-org/payu/pull/750)
- Set parent experiment to source branch/hash UUID when `-p BASE` is set in `payu checkout` by @Qianhui_Chen in [#759](https://github.com/payu-org/payu/pull/759)
- Add payu\_jobs as protected in sync. by @Qianhui_Chen in [#790](https://github.com/payu-org/payu/pull/790)
- 673 error handling cleanup by @jasmeen_kaur in [#755](https://github.com/payu-org/payu/pull/755)
- Add `sync:base_path` to PBS storage flag if `sync:path` is not set. by @Qianhui_Chen in [#801](https://github.com/payu-org/payu/pull/801)
- Raise error when archive not found unless a `--new-uuid` flag is specified. by @Qianhui_Chen in [#795](https://github.com/payu-org/payu/pull/795)
- Separate auto-generated and manual fields in `metadata.yaml` by @Qianhui_Chen in [#781](https://github.com/payu-org/payu/pull/781)

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.4 release](https://github.com/payu-org/payu/releases/tag/1.3.4).

## Notes

### 1. ⚠ Change of behavior ⚠: Confirmation required before creating a new UUID

Previously, payu would automatically generate a new experiment UUID whenever the archive directory could not be found. This default behaviour could be risky. For example, if you changed the short path after running a multi-run experiment, payu would silently create a new UUID and restart the experiment from `Run 0` instead of continuing from the existing restart.

Now, payu instead raises an error, unless the user explicitly confirms that a new UUID should be created by supplying the `--new-uuid` flag to `payu setup`, `payu run`, or `payu checkout`.

This confirmation is **not** required for `payu clone` or `payu checkout -b`, where creating a new experiment is the expected behaviour.

### 2. Improved error messages formatting and logging

From payu 1.3.3, payu-specific exceptions (for example, `PayuConfigError` and `PayuRuntimeError`) are displayed as formatted log messages without stack trace by default. Details of logging are controlled with the `--log-level` option. To display the full stack trace for these exceptions, please use the `--stacktrace` flag. Unexpected exceptions (such as `ValueError`) continue to display the full stack trace.

### 3. Check project access before PBS job submission

Before submitting a PBS job, payu now verifies that the user has access to the nominated project and the projects associated with the required files such as inputs, executables and restarts. This avoids waiting in the queue only to find out the job failed. If access is blocked, payu reports the problematic project(s) in the error message.

### 4. Easier access to experiment paths

The new `payu status --paths` command displays all important experiment directories, for example:

```sh
========================================
Experiment Paths:
  Experiment UUID: oo11000
  Experiment Name: mom6-777eeee
  Control Directory: /home/$USER/mom6
                      (Where model configuration is stored)
  Work Directory: /scratch/$PROJECT/$USER/mom6/work/mom6-777eeee
                      (Temporary directory for experiment runs)
  Archive Directory: /scratch/$PROJECT/$USER/mom6/archive/mom6-777eeee
                      (Where all experiment outputs are stored)
  Sync Destination: /path/to/sync
                      (Remote directory to sync outputs to)
========================================

```

For scripting and other machine-readable workflows, please use:

```sh
payu status --paths --json

```

For easier navigation, `payu setup` now creates symbolic links to both the work and archive directories in the control directory, if they do not already exist.

### 5. Shortcut for assigning a parent experiment in `payu checkout -b`

When a new branch (`New_B`) is created from a specific Git reference (`git_ref_A`):

```sh
payu checkout -b New_B git_ref_A

```

it is now possible to use `BASE` to automatically look up the UUID associated with `git_ref_A` and assign it as the parent experiment in `metadata.yaml`:

```sh
payu checkout -b New_B -p BASE git_ref_A

```

Having this information stored in the metadata improves experiment provenance by establishing clear links between experiments.

Previously, assigning parent experiment in `metadata.yaml` required manually specifying the parent UUID:

```sh
payu checkout -b New_B -p <parent_uuid> git_ref_A

```

This change saves users the effort to look up the UUID, and is less error prone.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments was by @Qianhui_Chen, @jasmeen_kaur, @atteggiani.

Reviewers included @jo-basevi, @Aidan.

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [26 August 2026 00:25 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/15 "2026-08-26T00:25:05Z")

</div>

# 26/08/2026

🚀 **Release:** 1.3.5

[https://github.com/payu-org/payu/releases/tag/1.3.5](https://github.com/payu-org/payu/releases/tag/1.3.5)

> ⚠  
> There is [a bug in this release affecting collation of ocean model output for run 0](https://github.com/payu-org/payu/issues/850).
> 
> Affected models: ACCESS-OM2, ACCESS-OM3
> 
> Recommended solution: use [payu version 1.3.6](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/16)  
> ⚠

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.5

```

## Updates

- Support relative restart path. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#803](https://github.com/payu-org/payu/pull/803)
- Telemetry: add file volume into job file. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#788](https://github.com/payu-org/payu/pull/788)
- Update new-uuid error message to be more clear. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#826](https://github.com/payu-org/payu/pull/826)
- Add parent branch off time in metadata by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#811](https://github.com/payu-org/payu/pull/811)
- Replace PBS submission layer with HPCpy. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#802](https://github.com/payu-org/payu/pull/802)
- Ask for shortpath and restart path in payu clone. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#814](https://github.com/payu-org/payu/pull/814)
- After `payu sweep --hard`, `payu setup` works without `--new-uuid` by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#833](https://github.com/payu-org/payu/pull/833)
- Skip asking for branch\_or\_commit if branch is specified in URL. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#838](https://github.com/payu-org/payu/pull/838)

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.5 release](https://github.com/payu-org/payu/releases/tag/1.3.5).

## Notes

### 1. Support relative path for restarts in `config.yaml`

From payu 1.3.5, the restart directory can be provided as a relative path in `config.yaml`. Payu will resolve the relative path to an absolute path and find the files.

### 2. `payu clone` supports changing project and branch-supplied URL

Based on community feedback, we add a new prompt in `payu clone` interactive mode:

```sh
>> Do you want to override the shortpath? (Default is '/xxx/yy')
    yes
    no

```

Selecting `yes` allows user to override the shortpath and specify a desired project.

`payu clone` interactive mode also supports branch-supplied URL (e.g., [https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/release-piControl](https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/release-piControl)), which will be convenient for beginners. Branch-supplied URLs are only supported in interactive mode and are not supported by non-interactive `payu clone`.

### 3. Parent experiment time and commit hash are added to metadata

When checking out a new branch, payu now records the following parent information in metadata.yaml:

- `parent_experiment`: The UUID of the parent experiment, if this experiment branches from another.
- `parent_experiment_branch_time`: The model time corresponding to the restart path used to create this branch, if restart path is provided.
- `parent_experiment_branch_commit`: The Git commit hash of the source branch when the new branch was created.

Please see [payu documentation](https://payu.readthedocs.io/en/latest/usage.html#:~:text=Some%20common%20metadata%20fields%20are%20shown%20in%20the%20table%20below%2C%20with%20their%20definitions.) for details on each field in `metadata.yaml`.

### 4. After `payu sweep --hard`, `payu setup` no longer requires `--new-uuid` flag

Since payu 1.3.4, `payu setup` requires explicit confirmation before generating a new archive and new UUID, using the `--new-uuid` flag.

From payu 1.3.5, `payu sweep --hard` sets the UUID to `uuid-swept-by-hard-sweep`. When payu detects this UUID, `payu setup` will not require the `--new-uuid` flag before generating a new archive directory and new UUID. This is based on the assumption that users want to start a new experiment after calling `payu sweep --hard`.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments by @Qianhui_Chen, @minghangli, @Aidan.

Reviewers included @jo-basevi, @Aidan, @jasmeen_kaur, @ben.

---

<div class="post-metadata">

**Author:** ![Qianhui\_Chen](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qianhui_chen/32/4372_2.png) [@Qianhui\_Chen](https://forum.access-hive.org.au/u/Qianhui_Chen)\
**Post date:** [9 September 2026 06:32 UTC](https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098/16 "2026-09-09T06:32:02Z")

</div>

# 09/09/2026

🚀 **Release:** 1.3.6

[https://github.com/payu-org/payu/releases/tag/1.3.6](https://github.com/payu-org/payu/releases/tag/1.3.6)

> ⚠
> 
> There is a [bug in this release for ACCESS-ESM1.5 models](https://github.com/payu-org/payu/issues/870) that returns the error
> 
> ```bash
> [ERROR] ‘Experiment’ object has no attribute ‘repeat_run’
> 
> ```
> 
> Work-around is to use another payu version for ACCESS-ESM1.5 models  
> ⚠

## Getting Started

This version of payu is now available on `gadi`:

```sh
module use /g/data/vk83/modules
module load payu/1.3.6

```

## Updates

## What’s Changed

- Add runlog-off, repeat and reproduce-off CLI flag in payu run by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#835](https://github.com/payu-org/payu/pull/835)
- Add job dependency: run \> collate \> postscript \> sync. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#832](https://github.com/payu-org/payu/pull/832)
- Write minimal provenance information to restarts by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#849](https://github.com/payu-org/payu/pull/849)
- Check for pre-existing restarts if repeat is True. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#848](https://github.com/payu-org/payu/pull/848)
- Fix handling of init\_run=0 in collate workflow by [@minghangli-uni](https://github.com/minghangli-uni) in [#851](https://github.com/payu-org/payu/pull/851)
- Handle empty-string timestamp, as well as other invalid string and None. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#857](https://github.com/payu-org/payu/pull/857)
- Add validation of parent uuid and check against the designated restart. by [@Qian-HuiChen](https://github.com/Qian-HuiChen) in [#854](https://github.com/payu-org/payu/pull/854)

For a full list of pull requests that includes additional bug fixes, please see [Payu Github 1.3.6 release](https://github.com/payu-org/payu/releases/tag/1.3.6).

## Notes

### 1. Bug fix: payu was collating `output001` instead of `output000` for run 0.

For each run `N`, payu is expected to collate `output00(N-1)`. However, @minghangli spotted that for run 0, payu was attempting to collate `output001` instead of `output000`. This resulted in a `FileNotFoundError`.

This bug was introduced in payu 1.3.1 and now is fixed in payu 1.3.6. It only affects run 0 and does not impact any other runs.

### 2. Add job dependencies to run → collate → postscript → sync workflow

Previously, payu submitted the postscript and sync jobs at the same time. This meant that the sync job could start before the postscript job had finished. For this reason, previous payu doesn’t sync the last run if postscript is enabled.

With payu 1.3.6, job dependencies are added to ensure that each stage only starts after the previous stage has exited successfully. The sync job can now safely sync the last run, ensuring that the postscript has completed before syncing.

All stages can be monitored via `payu status`, a feature requested by the community. For example:

```shell
$ payu status
========================================
Run: 37
  ------------- Run Info -------------
  Job ID: 1700.gadi-pbs
  Run ID: 989xxxxx3
  Stage: archive
  Total Queue Time: 0h 1m 57s
  Model Finish Time: 1903-12-01T00:00:00
  Model Exit Code: 0 (Success)
  Exit Status: 0 (Success)
  Output Log: /home/189/xxx
  Error Log: /home/189/xxx
  Job File: /scratch/tm70/xxx/run/1700.gadi-pbs.json
  ------------- Collate Info -------------
  Job ID: 1710.gadi-pbs
  Job Dependencies: 1700.gadi-pbs
  Stage: exited
  Total Queue Time: 0h 1m 12s
  Exit Status: 0 (Success)
  Output Log: /home/189/qsss
  Error Log: /home/189/ssss
  Job File: /scratch/tm70/xxx/collate/1705.gadi-pbs.json
  ------------- Postscript Info -------------
  Job ID: 1706.gadi-pbs
  Job Dependencies: 1705.gadi-pbs
  Stage: queued/running
  Job File: /scratch/tm70/xxx/postscript/1706.gadi-pbs.json
  ------------- Sync Info -------------
  Job ID: 1707.gadi-pbs
  Job Dependencies: 1706.gadi-pbs
  Stage: queued
  Job File: /scratch/tm70/xxx/sync/1707.gadi-pbs.json
========================================

```

All post-processing jobs (`collate`, `postscript`, and `sync`) can be disabled in `config.yaml`. If enabled, they will be submitted all together after the model run.

To remove killed jobs from the `payu status` output run:

```auto
payu status --update

```

## 3. Write minimal provenance information into restart directory

Payu now records minimal provenance information in the restart directory as `restart_metadata.yaml`.

```spoiler
name: mom6
payu_current_run: 3
experiment_uuid: xxx
payu_run_id: yyy0000
model_finish_time: '1901-03-22T00:00:00'
parent_experiment: <parent_experiment_uuid_if_exists_in_metadata>

```

This provides important experiment provenance information if the restart directory is moved out of the original archive directory.

For any information that cannot be retrieved (e.g., when runlog or metadata is disabled), the value will be recorded as ` __FAILED_TO_RETRIEVE__ `.

## 4. Validate the parent uuid format before adding it to metadata

In payu 1.3.6, the parent uuid is validated to ensure that it is in UUID4 format before being added to `metadata.yaml`, e.g.,

```sh
payu checkout -b new_branch ctrl_branch -p FAKE_UUID

```

will raise an error because `FAKE_UUID` does not match UUID4 format.

## 5. Bug fix: empty-string timestamp causing errors

As reported by the community and ACCESS-NRI model developers, the cached PBS node file may contain an empty timestamp (`""`), which can cause payu to error out.

This bug has been fixed in payu 1.3.6. When an empty-string timestamp is found, payu will generate a new cached `pbsnode.json` file.

## Support

Replies to this topic are disabled.

If you have specific questions about this release [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

If you have questions about #payu create a topic in a category that best matches the model you are using, or in the #Technical category and tag it with #payu. If you require assistance [follow the guidelines for requesting help from ACCESS-NRI](https://forum.access-hive.org.au/t/access-help-and-support/908).

## Credits

Developments by @Qianhui_Chen, @minghangli.

Reviewers included @jo-basevi, @Aidan, @spencerwong.