I have an ESM1.6 pre-industrial simulation running. One of the years has failed to convert the atmospheric data to netcdf. I’m running payu from /g/data/p66/rml599/picontrol/land-update and it is output022 that has failed to connvert. I’ve tried payu collate but got the following error message
Loading access-esm1p6/pr129-3
Loading requirement: cice5/access-esm1.6-2025.07.001-atis5ep
mom5/2025.05.000-hvqc4og um7/access-esm1.6-2025.06.000-cevll72
Traceback (most recent call last):
File "/g/data/vk83/prerelease/apps/base_conda/envs/payu-dev-20250923T072120Z-d026d1e/bin/payu-collate", line 7, in <module>
sys.exit(runscript())
File "/g/data/vk83/prerelease/apps/base_conda/envs/payu-dev-20250923T072120Z-d026d1e/lib/python3.10/site-packages/payu/subcommands/collate_cmd.py", line 111, in runscript
expt.collate()
File "/g/data/vk83/prerelease/apps/base_conda/envs/payu-dev-20250923T072120Z-d026d1e/lib/python3.10/site-packages/payu/experiment.py", line 973, in collate
model.collate()
File "/g/data/vk83/prerelease/apps/base_conda/envs/payu-dev-20250923T072120Z-d026d1e/lib/python3.10/site-packages/payu/models/fms.py", line 255, in collate
fms_collate(self)
File "/g/data/vk83/prerelease/apps/base_conda/envs/payu-dev-20250923T072120Z-d026d1e/lib/python3.10/site-packages/payu/models/fms.py", line 96, in fms_collate
assert mppnc_path, 'No mppnccombine program found'
AssertionError: No mppnccombine program found
Also yesterday I was trying to run some short tests swapping UM executable or restart file. I was running payu on /scratch and had the executable or restart on /g/data and payu was reporting that it coudn’t find the executable/restart. When I copied the executable or restart to /scratch, payu ran OK.
Hi @RachelLaw. The atmospheric netcdf conversion for ESM1.6 is not done as part of a Payu collate step. Instead, there is a postscript step that runs the script ./scripts/NetCDF-conversion/UM_conversion_job.sh. You can see how this is invoked by Payu by looking at the last line of the config.yaml.
@dougiesquire - thanks. This has worked for me (after I got the output directory wrong a couple of times) and noting no space before PROJECT.
I have previously used payu collate to do the conversion so don’t know if something has changed.
Thanks @RachelLaw. I’ve corrected my original post to remove the accidental space.
Regarding the second part of your question about changing the exe/restarts, my initial guess without more information is that the storage mount points for the exe/restart locations are not being set in the Payu job. Payu tries hard to set these automatically for you but, depending on how you’re doing things, they may not be being set. Could you try explicitly telling Payu to set the storage mount points? In the config.yaml add
This works because payu looks at paths in the manifests and in config.yaml to check for storage flags it needs to add. If you just do payu run it does the setup phase in a PBS job, which requires the correct storage flags to already be set. So it is a “chicken and egg” problem.
By running setup first on a login node, where a user has access to all their /g/data projects, payu can find everything it needs and update manifests, which can inform the next invocation of payu run.
It is also necessary to do payu sweep after payu setup, as payu will refuse to run if their is an extant work directory.
Hi @RachelLaw. Have the responses here answered your questions satisfactorily? If so, could you please mark one of the responses as the solution (by clicking on the tick at the bottom of the post) and I’ll close the topic
It is also necessary to do