Command line utility: pbs-workbench; your own personal ARE

Technical Research Software Community
5

A good start to compile a list of all these custom solutions and requests to show that the community really needs this.

  1. pbs-workbench (this tool)
  2. The previous center of excellence for extreme weather used to have this tool.
  3. Discussion here, including manual steps.
  4. Requests from researchers:
    1. Working with Jupyter notebooks on gadi/ARE via VS Code .
    2. https://forum.access-hive.org.au/t/opening-the-are-jupyterlab-session-on-vs-code/22

I believe @sam.green creates an interactive job and sshs into it. We could gather “testimonials” of other researchers’ custom solutions.

Secondly , describe exactly what is missing from the ARE workflow. Some ideas are:

  1. IDEs limited to RStudio and jupyter notebooks.
  2. Virtual desktop is very slow compared with native/local GUIs.
  3. Harcoded modules loaded that cannot be changed after spinup (that is true for rstudio and jupyter, not sure about the VDI).
    1. If in the process of data analysis you realise you need a new module, you need to close everything, add the modules and spin the whole thing up again.
    2. This makes it hard to run the same code interactively in ARE and via terminal or in jobs in the queue.
  4. Relies on external service (what happens if you ARE is down?)

Third, specify what “endpoints” would we need.

  1. Start a workbench with particular resources/profile for a particular ammount of time
  2. List running workbenches.
  3. Get information from a workbench:
    1. Resources
    2. time spent
    3. time left
    4. node address
    5. SU cost?
    6. … Something else?
  4. Stop a workbench
  5. Save “profiles” for particular configurations (similar to ARE ability to save settings)
  6. Modify existing profiles.
  7. …Something else?

For troubleshooting, it might be useful to get easy access to the logs.

1 Like