COSIMA Hackathon - G1 training

COSIMA Workshops
, , , ,
3

COSIMA hackathon v5.0 - G1 training

This is material to support training for the G1 group at the COSIMA hackathon v5.0, 8th Sept 2025. It is an update to the G1 training from last year’s COSIMA hackathon.

It is aimed at people who are new to COSIMA recipes. After completing this training, users will be able to use and modify COSIMA recipe notebooks using the NCI Australian Research Environment (ARE).

Prerequisites

Please make sure you’ve completed the steps in this topic post prior to the hackathon.

1. Logging in to the ARE

The Australian Research Environment is a web-based graphical interface for performing your computational research on Gadi, provided by NCI. There are lots of existing resources on how to use the NCI ARE. The instructions on the ACCESS-Hive and the NCI ARE User Guide include lots of useful information.

In this training, we’ll use the ARE to run both terminal commands and a JupyterLab instance on Gadi nodes. The terminal commands run below could also be run from your own terminal after SSHing into Gadi.

To log in to the ARE, go to https://are.nci.org.au/ and enter your NCI username and password.

2. Set up Github authentication

In order to interact with Github from Gadi we need to authenticate to Github. Last year, we manually set up SSH keys to do this. This year we’ll use the Github command-line tool gh to greatly simplify the process.

Steps:

  1. From your ARE dashboard, select the “Gadi Terminal” app. This will open a terminal with SSH connection to a Gadi login node.
  2. In the terminal window enter:
    $ module use /g/data/vk83/modules
$ module load gh
$ gh auth login
    You will be prompted with a few questions. Answer as follows:
    ? What account do you want to log into? GitHub.com
? What is your preferred protocol for Git operations on this host? HTTPS
? Authenticate Git with your GitHub credentials? Y
? How would you like to authenticate GitHub CLI? Login with a web browser
! First copy your one-time code: ABCD-1234
Press Enter to open github.com in your browser... 
! Failed opening a web browser at https://github.com/login/device
  exec: "xdg-open,x-www-browser,www-browser,wslview": executable file not found in $PATH
  Please try entering the URL in your browser manually
✓ Authentication complete.
    As you can see, gh will fail to automatically open the web browser from Gadi so we’ll do this manually. In a new browser tab, go to https://github.com/login/device and follow the prompts. You will be asked for a verification code, which is printed to your terminal (e.g. ABCD-1234 above). Then click “Authorize GitHub”.
  3. That’s it. You do not need to do this again on Gadi.

3. Cloning COSIMA recipes

COSIMA recipes is a collection of Python notebooks for analysing and plotting ocean model output.

COSIMA recipes is a repository on Github. In order to use the recipe notebooks on the ARE we need to create a copy of COSIMA recipes on Gadi. In git terminology, this is called “cloning” the repository.

Steps:

  1. From your ARE terminal window, navigate to where you would like to clone the COSIMA recipes. E.g.:

    $ mkdir -p /scratch/ik11/$USER/cosima_hackathon_2025
$ cd /scratch/ik11/$USER/cosima_hackathon_2025

    Note, files in /scratch will be automatically removed when they are older than 100 days (see details here). Use a project directory on /g/data/ for something more permanent.

  2. Clone the COSIMA recipes Github repo:

    $ git clone https://github.com/COSIMA/cosima-recipes.git

    You will now have a directory called cosima-recipes containing all of the COSIMA recipes.

4. Starting an ARE JupyterLab session

JupyterLab is a web-based interactive development environment for Python, R and Julia notebooks. We’ll use JupyterLab to run and modify the COSIMA recipes Python notebooks.

Steps:

  1. From your ARE dashboard, select the “JupyterLab” app and enter the following configuration settings (The fields marked with * are found in “Advanced Options”):

    Field Entry
    Walltime 2 hours
    Queue normalbw
    Compute Size Medium
    Project Choose one
    Storage gdata/xp65+gdata/ik11+gdata/cj50
    Module directories* /g/data/xp65/public/modules
    Modules* conda/analysis3

    Note, the storage section must contain all the /g/data projects you need access to. Above is only a minimum set needed for most COSIMA recipes. Note also that we are using the shared analysis3 conda environment from the xp65 project. This contains all the Python packages we need to run COSIMA recipes.

  2. Click the “Launch” button. You should see something like this:

    Screenshot 2024-06-27 at 4.22.07 PM
    Screenshot 2024-06-27 at 4.22.07 PM1868×672 49.2 KB

    You may have to wait a few minutes for your queued job to start running. When your JupyterLab session is running, you should see something like this:

    Screenshot 2024-06-27 at 4.25.46 PM
    Screenshot 2024-06-27 at 4.25.46 PM1874×774 64.1 KB

    Click the “Open JupyterLab” button to open your JupyterLab session in a new window.

5. Navigate to a recipe notebook and open it in JupyterLab

We’ll be running the COSIMA_Cookbook_Tutorial.ipynb notebook. Click on the folder icon at the top left of your JupyterLab session and user the interface to navigate to your clone of the COSIMA recipes.

Within the Cooking-Lessons-101-Tutorials directory, double click COSIMA_Cookbook_Tutorial.ipynb to open the notebook.

You can execute cells in the notebook using the arrow button in the control panel along the top of the notebook or using <shift>+<enter>

5. Self-led tutorials

That concludes the guided segment of the G1 training. For the rest of the Hackathon, it’s recommended that you either join a group working on an issue, or go through the following COSIMA recipes tutorials in the Cooking-Lessons-101-Tutorials directory. We’ll be around to help with questions!

  1. ACCESS-NRI_Intake_Catalog.ipynb
  2. Making_Maps_with_Cartopy.ipynb

You may also find these additional resources/tutorials useful: