Python and Fortran languages servers in VS Code not working after using cylc/rose/persistent-sessions

Dear community

I am seeking help about python and Fortran languages servers in VS Code (no syntax highlighting after using persistent sessions two weeks ago). I already posted two issues on pylance and remote-ssh github pages. After discussion with a pylance developer, we think the problem may be with my use of persistent-session while running ACCESS-rAM3 using cylc/rose.

The main problem is that I cannot have syntax highlighting for python and fortran files from two weeks ago as shown in the attached figure (I can have syntax highlighting on another remote host, but not gadi with another computer:). That was the time when I started to use persistent-session/cylc/rose to run ACCESS-rAM3. I am not really sure why this occurs, but after many failed attempts (Reinstall VS Code/all extensions/python conda environment, Remove remote .vscode and .vscode-server folders, Reset all python-extension-related settings, and everything I can find on google), I started to think this is a problem with X11 forwarding or localhost that is widely used while running ACCESS-rAM3. If you have met similar problems, or have any idea about underlying mechanisms of the persistent session, please let me know.

Hi @qinggangg , Thanks for reaching out to ACCESS-NRI. Your specific question is outside the topics that ACCESS-NRI officially supports. However, there may be a community member who can help you with your question.

Hi @harshula Thank you, I fully understand.

It would be greatly appreciated if anyone could provide some information about the use of localhost/X11 forwarding by persistent-session/rose/cylc.

Could you clarify why you think the persistent session is involved? Are you trying to run vscode on the persistent session?

As I understand it there’s a special compute node on Gadi that runs everyone’s persistent session, with some DNS magic to set up the ssh.

Hi Scott. Thank you for having a look. I am not running VS Code on persistent session. I only run ACCESS-rAM3 on persistent session before, and now I killed all persistent-session and deleted the folder ~/persistent-session. However, the problem with language servers in VS code persists.

I thought it may be the problem with localhost, which is used by rose/cylc while running simulations and also VS Code while starting a remote session. Is persistent session also using localhost? Does rose/cylc modify any thing about localhost permanently (e.g. memory usage/limit)?

No that is unrelated. ‘localhost’ is an alias that refers to the node that a process is running on. For vscode running on a login node, localhost will be that login node. For cylc running on the persistent session, localhost will be the persistent session node.

Thank you. That is good for me to know.

I got the following email from the NCI Helpdesk. In short, Gadi kill pylance processes on VSCode without any warning.

Would you have any suggestions for a workaround? I do like connecting to Gadi using VS Code and coding in Python on that for data analysis. Could I log into a computing node directly?

VSCode’s remote development tools are designed for when the remote machine is e.g. your own cloud instance and not a shared system like Gadi’s login nodes or when running on top of shared filesystems (e.g. NFS or Lustre). A lot of HPC sites have had issues with them, and Gadi is no exception. Instead of just outright banning VSCode, we instead have some automated system monitors that restrict or pause the corresponding processes.

In this case, the PyLance processes are one of those that is auto-detected and paused. This is because it can generate a very high load both on the node itself and on the Lustre filesystems as it almost constantly walks through the tree.

In general, we recommend developing your code locally and then uploading it to Gadi when you want to run it. That way you get the best of both worlds: a much more responsive development environment (since it’s local) but the large-scale compute resources of Gadi for the actual computations.

Hi @qinggangg,
You can connect to a compute node using VSCode.

I think the easiest way is to add a Match section in your ssh config file to check for a specified Host with the pattern gadi-cpu- (as all the Gadi compute nodes start with this pattern):

Match exec "echo %n | grep -q '^gadi-cpu-'"
    ProxyJump USERNAME@gadi.nci.org.au
    User USERNAME

You can also add other ssh keys/options as needed (for example ForwardX11 true).

This way, if you have a job running on a compute node (e.g. gadi-cpu-compute-example-node), you can connect to it by specifying its name:

ssh gadi-cpu-compute-example-node

From VSCode you can connect to it using the remote-ssh extension.

Alternatively, if you want to launch a remote VSCode session from your local terminal you can also run:

code --file-uri "vscode-remote://ssh-remote+gadi-cpu-compute-example-node"

(We dind’t specify any file path so no file gets open, but the remote VSCode session gets created)

or if you want to open a specific remote folder:

code --folder-uri "vscode-remote://ssh-remote+gadi-cpu-compute-example-node//PATH/TO/FOLDER/ON/GADI"

Hope that helps.
Cheers
Davide

1 Like

Hi Davide

Thank you. That is super helpful. I followed your instruction in another thread, and the python and fortran language servers finally get to work now:)

Have you used Modern Fortran in VS Code for fortran code syntax highlighting/hover? While the fortran language server is working with fortls in my conda environment, it outputs multiple error message like Module "***" not found in project. I am viewing ACCESS-rAM3 UM source code, which is suggestd not to be posted in this Forum as it is not open source.

I think the problem is with the first error message “Can’t open module file ‘***.mod’ for reading at (1): No such file or directory”. If I mannually open a .F90 file, a corresponding .mod file will be generated and the error message will disappear. But since I donot open most of the .F90 file, there are too many warning messages. Can we point Modern Fortran to these .F90 files rather than .mod files?

Regards, Qinggang

Hi @qinggangg,

Happy to know you managed to get the Python and Fortran language servers to work.

I don’t have much experience with Fortran in general, and I have never used Modern Fortran, so I cannot be of help there sorry.

Davide

Thank you @atteggiani, that is already very helpful.