Local Module Not Found

Technical
,
1

Hi, I need some help with a python technical issue.

I have a working directory and set up an env.sh file listing local modules that I’m using within this directory:

#!/bin/bash

# Define some basic environmental variables before launching the suite

# Load the analysis3 conda environment
module purge
module use /g/data/xp65/public/modules
module load conda/analysis3

# Root directory for this repo
export ROOT=/home/565/${USER}/trop-wx-sys-predict       # Change this to match where you clone this repo

# Append modules to our python path
export MODULES=${ROOT}/budget
export PYTHONPATH=${MODULES}:${PYTHONPATH}

export MODULES=${ROOT}/filter_mode
export PYTHONPATH=${MODULES}:${PYTHONPATH}

export MODULES=${ROOT}/track_cell
export PYTHONPATH=${MODULES}:${PYTHONPATH}

export MODULES=${ROOT}/submit_job
export PYTHONPATH=${MODULES}:${PYTHONPATH}

It’s been working without any problem for almost 2 years now, but today I’m encountering an issue when I try to run a python script. I did source env.sh then python3 this_is_a_file_name.py, and I got the following error message:

Traceback (most recent call last):
  File "/home/565/mr4682/trop-wx-sys-predict/this_is_a_file_name.py", line 7, in <module>
    import submit_job
  File "/g/data/xp65/admin/analysis3/sitecustomize.py", line 72, in tracking_import
    mod = _real_import(name, globals, locals, fromlist, level)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'submit_job'

I believe the module directories are already in my PYTHONPATH:

echo $PYTHONPATH

/home/565/mr4682/trop-wx-sys-predict/submit_job:/home/565/mr4682/trop-wx-sys-predict/track_cell:/home/565/mr4682/trop-wx-sys-predict/filter_mode:/home/565/mr4682/trop-wx-sys-predict/budget:/g/data/xp65/admin/analysis3:/home/565/mr4682/trop-wx-sys-predict/submit_job:/home/565/mr4682/trop-wx-sys-predict/track_cell:/home/565/mr4682/trop-wx-sys-predict/filter_mode:/home/565/mr4682/trop-wx-sys-predict/budget

However, when I tried importing in a python console, it worked:

python3
Python 3.11.13 | packaged by conda-forge | (main, Jun  4 2025, 14:48:23) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Ctrl click to launch VS Code Native REPL
>>> import budget
>>> import submit_job
>>>

I have tried using different versions of conda/analysis3, but none has worked so far.

Any help and advice to this issue would be greatly appreciated.

Thank you

2

Is there a file /home/565/mr4682/trop-wx-sys-predict/submit_job/__init__.py?

Otherwise there appear to be current connection issues with Gadi, might be a temporary disk outage

3

Yes, I have the __init__.pyin submit_job module directory. It contains this:

__name__ = "submit_job"
__all__ = ["submit", "check"]

from . import submit
from . import check
4

Only other thing I can think of is make sure sys.path is reflecting what you’ve set in $PYTHONPATH

5

Oh shouldn’t PYTHONPATH=$ROOT? You want it to be the parent directory of the module, not the module directory itself

6

Any reason not to use a virtual environment?

There is an example of how this might be done “on top” of a conda/analysis3 environment in the CLEX CMS docs

7

The reason for this is most likely that you started the python console from your home directory where the local module is.
Python also looks for files in the current directory to find modules to import, so in that case the module is imported correctly because it’s read from the current directory.
If you try running the console from a different directory, I think it will fail.

I think the simple fix is what @Scott suggested

Oh shouldn’t PYTHONPATH=$ROOT ? You want it to be the parent directory of the module, not the module directory itself

8

HI all, thank you for your kind replies.

I tried @Scott’s suggestion, and now it works!

The reason I didn’t set $PYTHONPATH=$ROOT is because I just followed this tutorial when I set up the directory. Surprisingly, it always worked until today.

Thank you!