Splitting storage/compute projects in rose/cylc

Technical
1

Hi All

I’m in a situation where my compute grant is in one project on Gadi, but the storage grant is in another. The suite I’m working with uses the $PROJECT environment variable to pick the project under which jobs will be qsub’d and also to create the symlinks into /scratch for the work and share directories. Right now, I’m starting the suite held and recreating those symlinks manually into the correct /scratch directory. Is there a better way to do this?

Dale

2

Never mind, I found a couple of ways to do this. For those interested:

  1. Add the following to rose-suite.conf outside of any other sections (e.g. [jinja2:suite.rc]):
root-dir{share}=gadi*=/scratch/<storage proj>/$USER
root-dir{share/cycle}=gadi*=/scratch/<storage proj>/$USER
root-dir{work}=gadi*=/scratch/<storage proj>/$USER

And set $PROJECT on accessdev to the compute project before running rose suite-run

Or 2)
Set $PROJECT on accessdev to the compute project before running rose suite-run
In ~/.config/gadi-login.conf, Change PROJECT to the storage project.

1 Like
3

FYI in case it is useful for anyone reading in payu this is done by setting shortpath in config.yaml

shortpath: /scratch/$DATA_PROJECT

I also recommend setting the setgid bit on any output directories so that the files have the same group as that of the containing directory, namely $DATA_PROJECT.