Payu not recognising archive directory

Technical
,
1

Hi! I’m having the same issue as this post, but the solutions there are not working for me.

I have been running with the new payu for a while, but this is the first time that payu does not recognise my archive directory where I have the restarts I want to continue my experiment from..

My working directory is:

/home/561/jn8053/payu/mom6/panant-ross/panant-01-zstar-ACCESSyr2-spinup-events

My archive is at:

/scratch/x77/jn8053/mom6/archive/panant-01-zstar-ACCESSyr2-spinup-events

And the branch I’m on is called panant-01-zstar-ACCESSyr2-spinup-events.

Payu is not finding the archive and creating this directory: `/scratch/x77/jn8053/mom6/archive/panant-01-zstar-ACCESSyr2-spinup-events-panant-01-zstar-ACCESSyr2-spinup-events-55118948/`

Setting a symlink manually works - but I don’t think that’s how payu is intended to work?

2

Also, payu statusdoesn’t work for me either (I read it here)

[jn8053@gadi-login-05 panant-01-zstar-ACCESSyr2-spinup-events]$ payu status
usage: payu [-h] [--version] {archive,branch,build,checkout,clone,collate,ghsetup,init,list,profile,push,run,setup,sweep,sync} ...
payu: error: argument {archive,branch,build,checkout,clone,collate,ghsetup,init,list,profile,push,run,setup,sweep,sync}: invalid choice: 'status' (choose from 'archive', 'branch', 'build', 'checkout', 'clone', 'collate', 'ghsetup', 'init', 'list', 'profile', 'push', 'run', 'setup', 'sweep', 'sync')
4

Hi @JuliaN

Can you paste the output of the following commands into a reply:

which payu

module list

and when in your experiment control directory

git branch -a

Thanks.

5

Thanks Aidan!

[jn8053@gadi-login-06 panant-01-zstar-ACCESSyr2-spinup-events]$ which payu
/g/data/vk83/apps/conda_scripts/payu-1.1.7.d/bin/payu

[jn8053@gadi-login-06 panant-01-zstar-ACCESSyr2-spinup-events]$ module list
Currently Loaded Modulefiles:
 1) singularity   2) payu/1.1.7(default) 

[jn8053@gadi-login-06 panant-01-zstar-ACCESSyr2-spinup-events]$ git branch -a
  panan-01
* panant-01-zstar-ACCESSyr2-spinup-events
  remotes/origin/HEAD -> origin/panan-01
  remotes/origin/master
  remotes/origin/panan-01
6

Immediate fix

payu generates the archive directory by concatenating the control directory name, the branch name and a short version of the experiment UUID.

This is explained in the payu documentation.

Your archive directory name is the same as your control directory name, so I assume this experiment was originally created with an older version of payu. Is that correct?

To be backwards compatible with the previous behaviour payu uses some logic to try and do the right thing:

To preserve backwards compatibility, if there’s a pre-existing archive under the control directory name, this will remain the experiment name (e.g. my_expt in the above example). Similarly, if the experiment value is configured (see Configuring your experiment), this will be used for the experiment name.

So in this instance I would recommend you add

experiment: panant-01-zstar-ACCESSyr2-spinup-events

to your config.yaml and payu should correctly find the archive directory and link to it.

General advice

I would not recommend naming a branch the same as the name of the control directory. It’s redundant.

Think of the control directory as the top level of a namespace hierarchy, and the branch name as the next level down.

For example, if your top level experiment directory is panant-01-zstar, your branch could be ACCESSyr2-spinup-events, and maybe another branch could be ACCESSyr3-spinup-events.

Or your top level experiment directory might be panant-01-zstar-ACCESS-spinup with branches yr2 and yr3.

:warning: I am not suggesting you change your control directory name, I am just giving examples for the future.

The traditional way to do this would be to make separate control directories for every experiment. You can do this still. If you did, I’d recommend just using main branches because the branch name won’t have any meaning, and payu doesn’t try and add it to the archive name.

payu docs

The docs you linked to default to the latest commit on the master branch, which is not version 1.1.7 that you are using. You can select the version of the docs using the selector in the bottom right corner.

The status command was added in this PR

We are planning on tagging another version and creating a release soon. Do you need that functionality immediately?

1 Like
7

Thank you Aidan! All great info. It’s more clear now.

Re status, I thought the docs would correspond the latest version posted in this thread (and it also says payu 1.1.7 at the top of the docs page) so I guess that without browsing the PR’s in the github I wouldn’t have known that the 1.1.7 in the docs is not the same as the 1.1.7 in the Hive. I don’t need it, but it struck me as strange that it doesn’t work. Reassuring to know its because it hasn’t been added yet.

8

Yes it can be a little difficult to tell, but the docs do default to the most recent release (1.1.7) which is shown in the top left of the screen:

Screenshot 2025-10-28 at 11.08.07 am
Screenshot 2025-10-28 at 11.08.07 am307×100 4.52 KB

and if you select “latest” it looks a little different:

Screenshot 2025-10-28 at 11.08.19 am

Where the version (1.1.7+50.g46fb4ac) indicates there are 50 commits on top pf the 1.1.7 release.

I’ve marked my reply as a solution. If that isn’t correct you can deselect it. I’ll close this topic as resolved at the end of the day unless you have any more questions.

1 Like
10

FYI payu/1.2.0 has just been released

and this has support for payu status.