I have an OM2 experiment and the config files are here: /home/561/qo9901/access-om2/1deg_esm1p5_hist_atm.
The config was originally created by following the payu clone instructions here and then making 20 billion changes so that ESM forcing can be used. I tried to push a copy to my personal github, but in the process I must have disconnected my exp config from the original archive directory, with the restarts later in the experiment.
The archive folder I want to reach is: /scratch/if69/qo9901/access-om2/archive/1deg_esm1p5_hist_atm-expt-b6610589. I added a symlink to this folder in the experiment config file so the restart file can be read and the experiment can be carried on from where I left off, but whenever I do payu sweep or payu setup, it makes a new experiment uuid and archive dir on scratch so I cannot continue the original experiment.
Any help on fixing this would be greatly appreciated, thank you in advance!
Sorry for the delayed reply. Could you confirm this directory /scratch/if69/qo9901/access-om2/archive/1deg_esm1p5_hist_atm-expt-b6610589 exists? I dont have permisson to that folder.
If the data is available, you can replace the experiment_uuid in metadata.yaml with b6610589-c5bd-4d1d-9939-8be93a3d5a76. Then create a symbolic link with,
Should I be changing the metadata.yaml file and make the symbolic link before or after the payu sweep step? I thought I tried both but it kept making a new experiment uuid.
@minghangli I did what you suggested after the payu sweep, so when I ran the model the uuid and archive was directed correctly. However, when I ran the model and checked the metadata.yaml in my exp directory again, the uuid has now changed. It looks like the uuid will be overridden when I do payu run?
As suggested by @Aidan (recreated your issue), the problem is that your working directory and branch name don’t match the name of your archive directory.
In this case, your working directory is 1deg_esm1p5_hist_atm, branch is main,archive is 1deg_esm1p5_hist_atm-expt-b6610589 (directory = 1deg_esm1p5_hist_atm, branch = expt, uuid = b6610589)
To align everything:
First create or switch to the correct branch:
payu checkout -b expt
Then update the experiment_uuid in your metadata.yaml to b6610589-c5bd-4d1d-9939-8be93a3d5a76.
Don’t need to create the archive symlink manually cuz payu will handle that for you.
Thanks minghang. Long story short, I got it to run, but I ended up staying on the main branch so it links with my github and renaming my archive directory. I therefore also had to create the symlink manually.
New archive dir name: 1deg_esm1p5_hist_atm-b6610589
The main branch was created when I tried to share the config to github. If there is a better way to keep track of config changes (e.g. forking from the NRI release at the very start), that would be super helpful.
but I ended up staying on the main branch so it links with my github and renaming my archive directory. I therefore also had to create the symlink manually.
It looks like you changed the names and checked out branches using git checkout instead of payu checkout. That’s likely why you had to create the symlink manually?
If there is a better way to keep track of config changes (e.g. forking from the NRI release at the very start), that would be super helpful.
I’d recommend sticking with payu commands for such - for example, Usage — payu 1.1.7+46.ga662f10 documentation in the Payu docs gives a good overview of how to manage metadata and experiment names cleanly.
