Ok so after some fiddling about, I have hit some new issues. I think I have managed to get past a problem with -xCORE-AVX2 vectorisation by manually editing the cmake of MOM5 to remove this problematic flag. (Which causes problems on the AMD infrastructure.)
But now when I try to run it, I get the following errors:
--------------------------------------------------------------------------
PMI2_Init failed to intialize. Return code: 14
--------------------------------------------------------------------------
--------------------------------------------------------------------------
The application appears to have been direct launched using "srun",
but OMPI was not built with SLURM's PMI support and therefore cannot
execute. There are several options for building PMI support under
SLURM, depending upon the SLURM version you are using:
version 16.05 or later: you can use SLURM's PMIx support. This
requires that you configure and build SLURM --with-pmix.
Versions earlier than 16.05: you must use either SLURM's PMI-1 or
PMI-2 support. SLURM builds PMI-1 by default, or you can manually
install PMI-2. You must then build Open MPI using --with-pmi pointing
to the SLURM PMI library location.
Please configure as appropriate and try again.
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[c028.hpc.nesi.org.nz:923286] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
[proxy:0@c027.hpc.nesi.org.nz] HYD_pmcd_pmip_control_cmd_cb (proxy/pmip_cb.c:487): assert (!closed) failed
[proxy:0@c027.hpc.nesi.org.nz] HYDT_dmxu_poll_wait_for_event (lib/tools/demux/demux_poll.c:76): callback returned error status
[proxy:0@c027.hpc.nesi.org.nz] main (proxy/pmip.c:122): demux engine error waiting for event
srun: error: c027: task 0: Exited with exit code 7
srun: Terminating StepId=4538385.0
[mpiexec@c027.hpc.nesi.org.nz] HYDT_bscu_wait_for_completion (lib/tools/bootstrap/utils/bscu_wait.c:109): one of the processes terminated badly; aborting
[mpiexec@c027.hpc.nesi.org.nz] HYDT_bsci_wait_for_completion (lib/tools/bootstrap/src/bsci_wait.c:21): launcher returned error waiting for completion
[mpiexec@c027.hpc.nesi.org.nz] HYD_pmci_wait_for_completion (mpiexec/pmiserv_pmci.c:196): launcher returned error waiting for completion
[mpiexec@c027.hpc.nesi.org.nz] main (mpiexec/mpiexec.c:260): process manager error waiting for completion
~