Introduction
This guide explains the steps to build ACCESS-OM3 2025.01.0 (0.4.0) from source using Spack, with a specific focus on modifying and rebuilding WAVEWATCH III (WW3). If you need to make local modifications to WW3—such as tuning wave physics, or implementing new parameterizations—this guide will walk you through the process of building ACCESS-OM3 with your changes applied to WW3.
Note: These instructions are only for users modifying the ACCESS-OM3’s WW3 source code. If you want to use a prebuilt version, refer to the documentation here
Prerequisites
Before building ACCESS-OM3 with modifications to WW3, you need to set up Spack on your system. Follow these steps documented here to install and configure Spack on Gadi.
Setting Up a Spack Env">Setting Up a Spack Development Environment for ACCESS-OM3 with WW3
The following steps assume you are already inside the Spack installation directory:
cd /g/data/$PROJECT/$USER/spack/0.22
1. Create a Development Directory for Spack
First, create a new directory to manage your Spack development environment:
mkdir spack-dev
cd spack-dev/
2. Clone the ACCESS-OM3 Spack Configuration Repository
Clone the ACCESS-OM3 repository that contains Spack configuration files:
git clone git@github.com:ACCESS-NRI/ACCESS-OM3.git
Note: This repository contains the Spack environment configuration for ACCESS-OM3, not the actual model source code.
3. Create a Separate Directory for WW3 Development
Since we are focusing on modifying and rebuilding WW3, create a dedicated directory for WW3 development:
mkdir ww3-dev
4. Create and Activate a Spack Environment
Use the spack.yaml file from the cloned ACCESS-OM3 repository to create a new Spack environment:
spack env create -d . ../ACCESS-OM3/spack.yaml
spack env activate .
Modifying and Building ACCESS-OM3 with a Custom WW3 Fork
After setting up the Spack environment, follow these steps to modify and build ACCESS-OM3 with your own WW3 fork.
1. Clone the ACCESS-OM3 Model Repository
Next, clone the ACCESS-OM3 model source code (which includes MOM6, CICE6, and WW3) into your working directory:
git clone git@github.com:COSIMA/access-om3.git access-om3-nuopc
cd access-om3-nuopc/
2. Update the WW3 Component to Use Your Fork
-
Create a fork of the official WW3 repository:
- Go to https://github.com/ESCOMP/WW3 and create a fork under your GitHub account.
-
Edit
.gitmodules
to point to your fork instead of the default WW3 repository:vi .gitmodules
Inside
.gitmodules
, locate the WW3 submodule entry and update theurl
to point to your fork, e.g.:[submodule "components/WW3"] path = components/WW3 url = git@github.com:<your-github-username>/WW3.git
-
Save and close the file.
3. Initialize and Update Submodules
After modifying .gitmodules
, update the WW3 submodule:
git submodule update --init --recursive
4. Register ACCESS-OM3 in Spack for Development
Now, register the access-om3-nuopc source code in Spack:
cd ../
spack develop -p ./access-om3-nuopc/ access-om3-nuopc@git.0.4.0
spack develop access-om3@git.2025.01.0
5. Concretize and Install Dependencies
Force a fresh Spack concretization (dependency resolution) and install everything:
spack concretize -f --fresh
Run Spack Install
Execute the following command to build and install all dependencies:
spack install
Note: The installation process will take a long time, especially on the first run, as Spack compiles all required components from source, including MOM6, CICE6, and WW3.
Rebuilding ACCESS-OM3 After Modifying WW3
Once you have successfully installed ACCESS-OM3, you may need to rebuild it after making changes to WW3. Follow these steps to recompile and update your modified version:
1. Enable Spack
If you have opened a new terminal session or logged in again, first enable Spack:
cd /g/data/$PROJECT/$USER/spack/0.22
. spack-config/spack-enable.bash
2. Activate the Spack Environment
Reactivate your Spack development environment before making modifications:
spack env activate ww3-dev
3. Modify WW3 Code
Navigate to your ACCESS-OM3 source directory and make the necessary modifications to the WW3 component:
cd spack-dev/ww3-dev/access-om3-nuopc/WW3/WW3
Once done, save the changes and commit them (optional):
git add .
git commit -m "Relevant commit message that describes the changes made."
4. Rebuild ACCESS-OM3 with the Updated WW3
Run Spack install to recompile the model with your changes:
spack install
Note: This step will only recompile components that were modified (i.e., WW3) and not rebuild everything from scratch.
Run the Model with the Updated WW3 Executable
Once you have made changes to the WW3 codebase and rebuilt ACCESS-OM3, you will need to configure the model to use the updated executable. Here’s how to proceed:
1. Load the Latest Version of Payu
module use /g/data/vk83/prerelease/modules && module load payu
2. Clone the access-om3-wav-configs
Repository
Clone the appropriate branch from the access-om3-wav-configs
repository. For example:
For the dev-1deg_jra55do_ryf
branch:
payu clone --branch dev-1deg_jra55do_ryf git@github.com:ACCESS-NRI/access-om3-wav-configs.git wav_config_ryf
For the dev-1deg_jra55do_iaf
branch:
payu clone --branch dev-1deg_jra55do_iaf git@github.com:ACCESS-NRI/access-om3-wav-configs.git wav_config_iaf
3. Update the Configuration
Modify the config.yaml
file to update the executable path and remove the default module:
- Find the
exe:
line and replace it with the locally built executable
- The executable is usually located in:
/g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-om3-nuopc-***/bin/access-om3-MOM6-CICE6-WW3
- Update
config.yaml
in thewav_config_*
directory accordingly:
exe: /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-om3-nuopc-***/bin/access-om3-MOM6-CICE6-WW3
- Remove the
access-om3/2024.09.0
module
- In the
modules:
section, remove the following line:
- access-om3/2024.09.0
Regenerating Input Files for WW3
When modifying the WW3 code, you need to update the executable and ensure that the wave model uses the correct grid and restart files. After building WW3 with Spack, the next step is to regenerate key input parameter files (mod_def.ww3
) and restart files (restart.ww3
) using the updated executable.
This process involves:
- Running
ww3_grid
to generate a newmod_def.ww3
(grid input parameters). - Running
ww3_strt
to generate a newrestart.ww3
(restart conditions). - Updating
config.yaml
to use these new files.
Steps to Regenerate and Update WW3 Grid & Restart Files
1. Navigate to the WW3 Pre-Processing Directory
Move to the WW3_PreProc
directory inside the wave configuration folder:
cd /g/data/$PROJECT/$USER/wav_config_*/WW3_PreProc
2. Generate the Grid Input Parameter File (mod_def.ww3
)
Run the ww3_grid
executable to create mod_def.ww3
:
/g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-om3-nuopc-***/bin/ww3_grid
This will generate the mod_def.ww3
file in the WW3_PreProc
directory.
3. Generate the Restart File (restart.ww3
)
Run the ww3_strt
executable to create restart.ww3
:
/g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-om3-nuopc-***/bin/ww3_strt
This will generate the restart.ww3
file in the WW3_PreProc
directory.
4. Update config.yaml
to Use the New Files
Modify config.yaml
so the model picks up the newly generated files. Locate the following lines:
- /g/data/vk83/configurations/inputs/access-om3/ww3/initial_conditions/global.1deg/2024.04.18/restart.ww3
- /g/data/vk83/configurations/inputs/access-om3/ww3/mod_def/global.1deg/2024.04.18/mod_def.ww3
And replace them with:
- /g/data/$PROJECT/$USER/wav_config_*/WW3_PreProc/restart.ww3
- /g/data/$PROJECT/$USER/wav_config_*/WW3_PreProc/mod_def.ww3
5. Verify and Proceed
- Ensure
mod_def.ww3
andrestart.ww3
exist inWW3_PreProc
. - Confirm
config.yaml
reflects the updated paths. - You can now proceed with running the model using:
payu sweep && payu run
You’ve successfully updated the WW3 executables and regenerated necessary input files for ACCESS-OM3. Ensure the correct executable and files are referenced in config.yaml
before running the model with payu. This process keeps WW3 aligned with your modifications.