VSCode Extension to run ARE session

Hello everyone.

Since many NCI users have VSCode as their main text editor, I thought it would be useful having a VSCode extension to start and connect to an ARE session.

This to have a better workflow rather than having to start an ARE session separately, get the ssh config parameters and then connect through VSCode (using the remote-ssh extension).

At the moment, I don’t think there is any extension that allows such direct connection, but I was wondering if anyone had already set up such a thing ( Maybe @Scott?).

Would love to know more and in case work on that if other people are interested in having such a workflow.

Cheers
Davide

There’s no api for starting a session, so you’d need to scrape the web interface to do that or work out what the web interface submits to the pbs queue.

If a session is already running you can get the connection info from the ~/ondemand directory. https://git.nci.org.au/saw562/tools/-/blob/master/ood sketches out how this can be done. I’ve been putting this together mainly for easier vnc access but ./ood ssh should ssh into the most recent session

At the moment I’m already getting infos through the ~/ondemand directory, and everything works.

I was mainly looking for a more “straightforward” way to run the session directly from gadi (by submitting a PBS job) and, ultimately, to do this using a VSCode API.

Do you know where can I get more information on what the web interface submits to the pbs queue to start an ARE session?

qstat -f on a running ARE job should show everything used to start the queue jobs, I don’t know how ondemand works internally though

Not what you wanted, but related, some OOD services provide VS Code in a browser

https://www.emsl.pnnl.gov/MSC/UserGuide/ood/vscode.html

https://hpc.nmsu.edu/discovery/ondemand/vscode/

It appears this is one of OODs supported apps

https://osc.github.io/ood-documentation/latest/install-ihpc-apps.html

and this is the app they are running:

Nice. I’d imagine a VS Code service would be appreciated by a broad range of NCI users. Perhaps worth dropping a question to NCI Help @atteggiani?

@Aidan I had read about the possibility to have OOD services in VSCode browser. I have never used VSCode browser so maybe I’ll check it out.
However, I think most people would still prefer using their local VSCode so I’m gonna look more into this and maybe follow @dougiesquire’s suggestion and contact NCI directly.
I’ll keep you posted.

Just wondering if you ever got this to work @atteggiani? My use case, mostly, is that it would be nice to use VSCode to edit notebooks on an ARE session.

Hi @cbull,

I didn’t follow it through with NCI.
Even though is a bit “clunky”, you can still connect to an ARE session via VSCode by using the Remote - SSH VSCode extension.

The other information needed is the hostname of the computing node you are connected to through ARE.

In general, I would suggest the following workflow to make things a bit simpler and quicker:

1. Install the Remote - SSH VSCode extension

You can do this manually, or from a local terminal you can run:

code --install-extension ms-vscode-remote.remote-ssh

This step only needs to be carried out once.

2. Add SSH configuration for compute node hosts

Add the following (by changing <your-gadi-username> with your Gadi username) to your ~/.ssh/config file (before the Host * if present):

Match exec "echo %n | grep -q '^gadi-cpu-'"
    ProxyJump <your-gadi-username>@gadi.nci.org.au
    User <your-gadi-username>
    ForwardX11 true
    ForwardX11Trusted yes

This will enable to connect to a compute node (hostname always in the form gadi-cpu-...) via SSH (by automatically using a proxyjump).
This step only needs to be carried out once.

3. Start an ARE session

Start an ARE session (VDI or Jupyterlab).
Once the session starts, your ARE Dashboard will tell you the name of your compute node Host:


Copy the compute node Host (for simplicity, you can click on it and then in the new tab copy the last portion of the URL).

4. Use the compute node Host as the hostname for VSCode remote - SSH

You can paste the compute host manually in VSCode’s Remote - SSH: Connect to Host.... command, when asked “Select configured SSH host or enter user@host”.

Alternatively, you can run the following command on a local terminal:

code --file-uri vscode-remote://ssh-remote+<compute-node-host>

For the example in the image above, the command to run would be:

code --file-uri vscode-remote://ssh-remote+gadi-cpu-bdw-0031.gadi.nci.org.au

This opens a new “blank” VSCode session connected to the compute node.

Extra

1 ex. Connect to a specific folder (automatically creating a workspace)

You can also connect to a specific folder on Gadi and automatically create a VSCode workspace using the --folder-uri option in the command above:

code --folder-uri vscode-remote://ssh-remote+<compute-node-host><path/to/folder/on/gadi>

For example, with the comput host of the image above, to connect to the /scratch/public folder, run:

code --folder-uri vscode-remote://ssh-remote+gadi-cpu-bdw-0031.gadi.nci.org.au/scratch/public

2 ex. Automate the connection with a script/alias

You can also write a bash function or alias (in your local ~/.bashrc), to automate the connection.
For example, if you want to connect to ARE though VSCode using arecode <compute-node-host>, you can add the following to your local ~/.bashrc file:

alias arecode="code --file-uri vscode-remote://ssh-remote+${1}"

Then (in a new bash session, or after sourcing the ~/.bashrc), to connect to the ARE session of the example image above, you could run:

arecode gadi-cpu-bdw-0031.gadi.nci.org.au

Hope this helps simplify workflows!
Davide

I am guessing we also need to add an entry for Match exec "echo %n | grep -q '^gadi-gpu-'" for ARE runs with the GPU node?

We’ve had success using the VScode CLI and remote tunnel extension to run from compute nodes.

From an ARE session (jupyter or VDI should work, you need ARE for the external internet access) get the vscode x64 CLI from Download Visual Studio Code - Free AI Code Editor for Mac, Linux, Windows

Log in using your Github account in jupyter/VDI terminal with

code tunnel user login

then start vscode with

code tunnel

You can then use the remote tunnels extension to connect from Vscode desktop (use the same github account you logged in with) or code tunnel also gives you a URL you can connect to in a web browser.

If you load an environment before running code tunnel vscode should be running in that environment. If you start ARE on a gpu node you’ll have access to the gpus in vscode for ML development.

Hi @atteggiani ! I was wondering if this is the current recommended way of using vscode with an ARE sessions?

Following these steps, I have been able to open the session. However (a) it doesn’t have internet access and (b) I can’t open the conda environments.

Maybe I’m missing something extra?

If you connect to an ARE session using VScode you won’t have an internet connection. It’s annoying but I don’t see NCI changing that soon.

To find the conda/analysis3 environments, I use these instructions (assuming you added the corresponding modules in the session advanced settings):

  1. Open the command palette (Ctrl+Shift+P or Cmd+Shift+P on Mac) and type “Python: Select Interpreter”.
  2. You will see a list of available interpreters, including those from your conda environments. Most probably the analysis3 won’t be on the list so you will need to add it manually by typing the path to the interpreter in the command palette. So, select “Enter interpreter path…” and copy the path. The path to the analysis3 conda environment will be /g/data/xp65/public/apps/med_conda_scripts/analysis3-XX.XX.d/bin but replace XX.XX with the corresponding version of the environment you want to use. For example, 25.06.
  3. This will now be available in the jupyter kernel.

It works for me, but I know others had problems with ARE now seeing the jupyter kernel.

Great, thank you Paola! I seem to be able now to find the environment jaja! Maybe I just had a typo before..