# Tennessee Leeuwenburg: Auto-encoder training sessions **URL:** \ **Category:** Atmosphere/ML 2026 Training/Discussion Sessions\ **Created:** [12 August 2026 04:46 UTC](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788 "2026-08-12T04:46:02Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![Tennessee](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/tennessee/32/2245_2.png) [@Tennessee](https://forum.access-hive.org.au/u/Tennessee)\ **Post date:** [16 August 2026 23:39 UTC](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2 "2026-08-16T23:39:25Z")
> **ML Workshop 2026 Technical Setup Instructions for Autoencoder Training Sessions** ### Introduction: This post covers the technical setup requirements for the hands-on training session on: **Autoencoders: Part 2** | 2026-08-20T03:30:00Z→2026-08-20T05:00:00Z. We will go through the rest of these setup instructions together at the end of the Autoencoders: Part 1 training session, so it’s not essential to complete these steps 3-4 in advance. However, if you’d like to maximise your hands-on time during the workshop, working through these steps beforehand will give you a head start. * * * ### Table of Contents: > ‼ **The first two steps must be completed prior to the training sessions**. 1. [**Sign up for an NCI account**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-1-sign-up-for-nci-account-4) 2. [**Join up to the relevant NCI projects**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-2-join-nci-projects-5) * * * > 📝 **Setup instructions: We’ll go together at the end of Day 1 Training** 1. [**Create your working directories and copy the training notebooks**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-3-create-your-working-directories-and-copy-the-training-notebooks-7) 2. [**Set up a JupyterLab session in NCI ARE**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-4-set-up-jupyterlab-in-nci-are-8) 3. [**Test your setup**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-5-test-your-setup-9) > 🚀 **On the day: Start the session 30 mins prior** 1. [**Launch JupyterLab early on the day**](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788/2#p-23841-h-6-launch-jupyterlab-early-on-the-day-10) Preferably 30 minutes prior * * * ### Required setup before the training session * * * #### 1. Sign up for NCI account If you don’t already have an NCI account, [sign up here](https://my.nci.org.au/mancini/signup/0). When creating your account, you can nominate project `nf33` as your main project. > **New to NCI ARE?** > The practical exercises will use the [NCI Australian Research Environment (ARE)](https://docs.access-hive.org.au/getting_started/are/). Basic familiarity with the ARE environment is recommended, but we’ll also step through the setup together during the training. * * * #### 2. Join NCI projects Please request membership of the following NCI projects. - [dk92](https://my.nci.org.au/mancini/project/dk92/join): Data science environment, used to access the PyEarthTools Python modules. - [nf33](https://my.nci.org.au/mancini/project/nf33/join): ACCESS-NRI training environment, used for compute and setting up your working directory. - [rv74](https://my.nci.org.au/mancini/project/rv74/join): Himawari 8 data used in the Auto-encoder examples. - [rt52](https://my.nci.org.au/mancini/project/rt52/join): Required to fully participate in the workshop. * * * - Optional [wb00](https://my.nci.org.au/mancini/project/wb00/join): For accessing ERA5 on-disk data without needing to download it. - Optional [rq0](https://my.nci.org.au/mancini/project/rq0/join): For radar-based precipitation data. > **Note** : `rt52` project was not listed in the original setup email, but is required to fully participate in the workshop. Please make sure you request access to this project as well. * * * ### Setup instructions: We’ll go through at the end of the Part 1 training * * * #### 3. Create your working directories and copy the training notebooks Once your NCI account and project access are ready, log in to Gadi from a terminal: ```auto ssh gadi ``` Move to the `nf33` project directory: ```auto cd /g/data/nf33 ``` Create a directory using **your own NCI username** , then move into it: ```auto mkdir -p cd ``` For example, if your NCI username were `ab1234`: ```auto mkdir -p ab1234 cd ab1234 ``` You can check your current location with: ```auto pwd ``` It should look similar to: ```auto /g/data/nf33/ ``` Now create the directories you’ll need for the workshop: ```auto mkdir -p userbase62 notebooks ``` Your directory should now contain: ```auto userbase62/ notebooks/ ``` Now, we are done with the creation of working directories! Let’s copy over the **training notebooks**. **Copy the training notebooks:** Move into your notebooks directory: ```auto cd notebooks ``` Copy the workshop notebooks from the shared training directory: ```auto cp -r /g/data/nf33/autoencoder_notebooks/* . ``` Check that the files have copied successfully: ```auto ls -l ``` You should see files including the workshop notebooks and a `tutorial` directory. > **Example Output for `ls -l`:** > > ```auto > [ab1234@gadi-login-08 notebooks\]$ ls -l > > total 696 > > rw-r----- 1 ab1234 nf33 523602 Aug 12 17:04 DataAccessForWorkshopsNCI.ipynb > rw-r----- 1 ab1234 nf33 184259 Aug 12 17:04 TemporalWindowDemo.ipynb > drwxr-sr-x 2 ab1234 nf33 4096 Aug 12 17.04 tutorial > > ``` * * * #### 4. Set up JupyterLab in NCI ARE Open [NCI ARE](https://are.nci.org.au/) and select **JupyterLab**. **Main JupyterLab settings:** Recommended fields are as follows: | Setting | Value | | --- | --- | | **Walltime** | `3` hours | | **Queue** | `gpuvolta` | | **Compute Size** | `1gpu` | | **Project** | `nf33` | | **Storage** | See below | For **Storage** , add the project directories required by the notebooks: ```auto gdata/dk92+gdata/nf33+gdata/wb00+gdata/rv74+gdata/rq0+gdata/rt52 ``` > **Main JupyterLab settings** > > ![Screenshot 2026-08-18 at 09.07.27](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/e/ee248d2a9c95005fd32f0b1e45d39e1320d357c3.png) **Advanced Settings:** Tick **Show advanced settings**. For **Module directories** , enter: ```auto /g/data/dk92/apps/Modules/modulefiles ``` For **Modules** , enter: ```auto pet/0.6.2 ``` > **Advanced Settings:** > > ![image004](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/7/731f5a9367de68e2f7a411718da3810ffdfffaed.png) Further down the advanced settings, find **Environment variables**. Enter: ```auto PYTHONUSERBASE="/g/data/nf33//userbase62/" ``` Replace `` with your actual NCI username. For example: ```auto PYTHONUSERBASE="/g/data/nf33/abc123/userbase62/" ``` Set **Jobfs size** to: ```auto 100GB ``` > **Advanced Settings \> Environment Variables:** > > ![image005](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/f/f2d237ccf978d310f63d238fef014827d3b09a32.png) > **Tip:** Once you’ve confirmed everything is working, you can use **Save settings** in ARE so you don’t need to enter all of the configuration again on the day. 🚀 When you’re ready, select **Launch**. It may take a few minutes for your JupyterLab session to start. * * * #### 5. Test your setup Once the JupyterLab session has started, click Open JupyterLab button under My Interactive Sessions tab. > Click Open JupyterLab > > ![Screenshot 2026-08-18 at 09.40.55](https://us1.discourse-cdn.com/flex020/uploads/access1/original/2X/3/35b40318dea2fddacdbd5ae7c8745ce8cbe53217.png) Use the file browser to navigate to: ```auto /g/data/nf33//notebooks ``` Open and run the cells for: ```auto DataAccessForWorkshopsNCI.ipynb ``` This gives us a chance to check that the environment and data access are all working before moving on to the main exercises. * * * #### 6. Launch JupyterLab early on the day Before we move into the hands-on component, we’ll make sure everyone’s JupyterLab session is running. ARE jobs can sometimes spend a little time waiting in the queue, so we’ll aim to launch the sessions around **30 minutes** before we need them. Once your session starts, navigate to: ```auto /g/data/nf33//notebooks/tutorial ``` and open: ```auto IntroductionForWorkshops.ipynb ``` From there, we’re ready to start the exercises. ✅ --- _[View the full topic](https://forum.access-hive.org.au/t/tennessee-leeuwenburg-auto-encoder-training-sessions/6788)._