Hi Dave,
Awesome to hear about the grant and your interest in getting OM2 over to NeSI!
before I dive in and start trying to do it myself, I wanted to check in and see if ACCESS-NRI has any capacity / interest to provide advice on this?
AFAIK, this is not something we officially support at the present time and the release team are low on people-time at the moment. Still, as you’ve seen with Leonardo progress was possible with some perseverance as it has also been on Setonix (Pawsey).
Also, NeSI uses SLURM rather than PBS… again I can dive in and try to configure this myself (I did something similar with a much older version of payu), but any capacity to pick your brains would be helpful.
A little work has been done on this in the past. Here’s a comment from @jo-basevi:
This is Aidans PR from a couple years ago to get something working on pawsey- Porting to pawsey by aidanheerdegen · Pull Request #326 · payu-org/payu · GitHub - related to this issue. I think that is more likely likely to work than my draft PR which is only just using the slurm client from hpcpy so far.
@ChrisC28 and @john_reilly also have a hacked version of payu from a while back that has limited functionality (I’m told) on Pawsey, given that system also uses Slurm that might help. HPCpy is probably the better route to go long-term as it’s intended to be scheduler agnostic.
As it happens, myself @helen @harshula have been working on similar things (OM3) over on Setonix recently. So below are a few things that might help you get started/have a clearer sense of what’s involved. I’d expect the general workflow to be similar-ish for OM2/OM3.
I’ve got a mega-issue here that collates the various issues. My latest problem is likely related to not setting up the run folder correctly but before that I had some build issues. You can see my very scrappy notes here (these arn’t meant to be used by anyone else just yet!).
Perhaps the most useful thing to look at to start is the build changes that were made for Setonix. In the Setonix case, we’re lucky that spack is already installed on the system. You might have to install it, if it’s not there already.
Chris