Payu collate on output that's been moved

Technical Infrastructure
, ,
1

Hi All,

I’ve been tidying up some data from an ACCESS-ESM1-5 run that was initially output onto /scratch/x77, but that I moved off scratch before fixing up some of the collation fails. In order to rerun payu collate, I found I had to move the output back onto the scratch directory where it was originally run, which feels like it shouldn’t be the case.

Should payu be able to handle this better? All I want is for it to tidy up the giant pile of temporary files in the ocean output and convert my um files to netcdf.

(in more detail, my workflow was/is:

  • run model, output onto scratch (a handful of collations failed)
  • move output onto gdata
  • realise collations failed
  • move output back onto scratch
  • run payu collate -i <x>
  • move output from scratch onto gdata again
  • )

I tried using -d and the directory name of where it’s currently stored, but I didn’t have any success or useful error messages, just a warning about not being able to find a restart

2

Hi Jemma,

Thank you for raising this issue!

I tried to reproduce the issue by moving output018 and restart017 out of my archive directory. Then I ran:

payu collate -i 18 -d /new/path/to/output018/ocean

This successfully collated my ocean .nc files (also see the payu documentation).

I also tried this instead:

payu collate -i 18 -d /new/path/to/output018

However, this didn’t collate any of my tile files inside output018/ocean.

My suspicion is that the ocean subdirectory needs to be specified explicitly so that payu can locate the tiles when the outputXXX is outside archive.

Would this be helpful in your case?

3

It hadn’t occurred to me to try and use -i and -d together in the same command, so that might have been my problem. Either way, if it worked for you then it sounds like a me problem rather than a universal problem, and if I’ve got a workaround then it’s not so important spending time to fix. Thanks for taking the time to replicate this :), and good to know that I’d need to specify the /ocean bit not just output folder in general.

In my specific case, I’ve worked around the problem by just moving everything back to scratch to collate, and then I’ll move it back again once the collation finished.

Out of curiosity, did you move your output018 etc to a new project or a different folder on the same project? We were wondering if the storage flags were causing me problems, but it didn’t seem to resolve when adding those storage options to the config.yaml.

4

Hi Jemma,

The -i flag shouldn’t affect the collation process itself. I added it so that the collate job file is stored in the expected directory. But yes, specifying the ocean/ layer should help payu find the .nc tiles.

Regarding the storage, good point!

I first tested this by moving my output018 from /scratch/tm70/$USER/access-om2/archive/ctrl to /scratch/tm70/$USER/ForumTest/. Collation succeeded with:

payu collate -i 18 -d /scratch/tm70/$USER/ForumTest/output018/ocean

Then I moved output018 to /g/data/tm70/$USER/ForumTest/, I had to add:

storage:
    gdata:
        - tm70

before running payu collate -i 18 -d /g/data/tm70/$USER/ForumTest/output018/ocean. This also succeeded.

This section in payu documentation may be helpful for configuring the storage.