After having a bunch of problems with ARE sessions I decided to stop using them. But because I still need to spin up a job in a node to run analysis code interactively, I created this small bash tool.
The command
job start
submits a job to the pbs queue that just sits there doing nothing. But then you can ssh into the node and work. It also has a cute monitor system that tells you if the job is running, how much time you’ve got left and quick copy-paste commands do interact with it.
It works well with vscode and its forks and also with jupyter notebooks.
The script supports creating “profiles”, so you can create your own “big”, “small”, “bigmem” or whatever other profile you want, then you spin up your job with job start <name of profile>.
I tried to make installation as straightforward a possible. You need to clone my repo and then run the install script.
Let me know if anyone finds this useful and what other features would you like implemented. Right now the tool only supports one running job that is “global”, but I was thinking that maybe it would be better to use a job per project.
I think being able to quickly spin up a job on a computing node and being able to connect to it through VSCode is very useful for many people in the community.
I created a similar script years ago following this (fairly old) discussion as I was getting frustrated with ARE, and @abhaasgoyal might also have done something similar based of my script.
What are the thoughts about creating a VSCode extension (some old details in the link above) to spin up a computing job and connect to it?
We might need to coordinate with NCI too on this, but I would be happy to help.
We talked about this a bit, but I think the idea of having a standard “API” that other tools can hook could be a good solution. Then, anyone can create VSCode extensions or local bash commands, or whatever (does jupyter notebook have extensions?) using those standard endpoints.
One of the reasons I like the script running on gadi is that I don’t need to worry about supporting multiple OS and IDEs.
I don’t know exactly what that would look like. Maybe it’s a standard module that you can load and all the interaction is done with ssh commands.
I think the idea of a standard API is very good. As you mentioned, the API could be used by any IDE extension, or as a CLI from a terminal.
I would say the first approach here is to get in contact with NCI to understand whether there is scope for them to collaborate and see how we can help with that, because I think this is something that should be managed (or at least approved) by NCI in the first place.
