Virtual environment built on top of xp65 conda fails with segmentation fault during module import

Technical
,
1

Hello all.

The WCRP global hackathon in May 2025 uses data stored in the healpix format :

The required python libraries to load this data are healpy and ezygems.

I followed standard procedures to build a virtual environment on top of xp65 incorporating these two libraries :

$ python -m venv hackathon_env --system-site-packages
$ source hackathon_env/bin/activate
$ pip install healpy
$ pip install easygems

The environment is located in

/g/data/nf33/public/hackathon-2025/venvs/hackathon_env/

I also built a jupyter kernel using

python -m ipykernel install --prefix /g/data/nf33/public/hackathon-2025/venvs/hackathon_env --name hackathon_env --display-name "hackathon_kernel"

The kernel works inside an ARE notebook, but the python interpreter will throw a segmentation fault when importing easygems

$ python
Python 3.11.11 | packaged by conda-forge | (main, Mar  3 2025, 20:43:55) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import healpy as hp
>>> import easygems.healpix as egh
/g/data/nf33/public/hackathon-2025/venvs/hackathon_env/bin/python: line 165: 2249617 Segmentation fault      "$SINGULARITY_BINARY_PATH" -s exec --bind "${bind_str}" ${overlay_args} "${CONTAINER_PATH}" "${cmd_to_run[@]}"

Evidently this is an issue related to the singularity container.

Any suggested work-arounds?

FWIW a virtual environment built using hh5 analysis3 can import easygems via the python interpreter and a Jupyter kernel.

2

Hi Paul,

which version of the xp65 conda environment are you using? (Apologies if it’s in your post, recovering from a nasty cold & might have missed it)

Around the time that the analysis3-25.04 was released, we started experiencing segmentation faults which appear to be related to the netcdf-python library. We’ve been working to fix this, but haven’t managed to find a fix yet.

Assuming you are using analysis3-25.04, I’d be willing to bet your segmentation faults are related. In this instance, could you try downgrading to analysis3-25.02 and see if that stops the issues? I’ve just tried in the analysis3-25.02 environment & didn’t get any issues on import, so fingers crossed it should solve the problem.

eg. I sucessfully ran:

module use /g/data/xp65/public/modules && module load conda/analysis3-25.02
Loading conda/analysis3-25.02
  Loading requirement: singularity
[ct1163@gadi-login-02 paul_easygems]$ python -m venv venv && source venv/bin/activate
(venv) [ct1163@gadi-login-02 paul_easygems]$ python -m pip install easygems healpy
Collecting easygems
  Using cached easygems-0.0.14-py3-none-any.whl.metadata (1.2 kB)
Collecting healpy
  Using cached healpy-1.18.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.1 kB)
...
Using cached certifi-2025.4.26-py3-none-any.whl (159 kB)
Installing collected packages: pytz, tzdata, six, PyYAML, pyshp, pyparsing, pillow, packaging, numpy, kiwisolver, fonttools, cycler, certifi, astropy-iers-data, shapely, scipy, python-dateutil, pyproj, pyerfa, healpix, contourpy, pandas, matplotlib, astropy, xarray, healpy, cartopy, cf_xarray, easygems
Successfully installed PyYAML-6.0.2 astropy-7.0.1 astropy-iers-data-0.2025.4.28.0.37.27 cartopy-0.24.1 certifi-2025.4.26 cf_xarray-0.10.5 contourpy-1.3.2 cycler-0.12.1 easygems-0.0.14 fonttools-4.57.0 healpix-2024.2 healpy-1.18.1 kiwisolver-1.4.8 matplotlib-3.10.1 numpy-2.2.5 packaging-25.0 pandas-2.2.3 pillow-11.2.1 pyerfa-2.0.1.5 pyparsing-3.2.3 pyproj-3.7.1 pyshp-2.3.1 python-dateutil-2.9.0.post0 pytz-2025.2 scipy-1.15.2 shapely-2.1.0 six-1.17.0 tzdata-2025.2 xarray-2025.4.0

[notice] A new release of pip is available: 24.0 -> 25.1
[notice] To update, run: pip3.11 install --upgrade pip
(venv) [ct1163@gadi-login-02 paul_easygems]$ python

Python 3.11.11 | packaged by conda-forge | (main, Dec  5 2024, 14:17:24) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import healpy as hp
>>> import easygems.healpix as egh

N.B: Unfortunately when segfaults occur, error messages can get very misleading - I assume the error message you get related to the container is something to do with the process dying/being restarted.

3

Thanks Charles, I’ll give that a go.

I used the default version which appears to point me to analysis3-25.03.d.

I’ll replicate your commands above.

4

Hi Paul,

That’s good to know that the segmentation faults appear as of 25.03 - should be useful information tracking the root cause of the issue down.

Cheers, Charles

5

It all seems to working fine Charles. Thanks for the quick fix. The hackathon team greatly appreciate it :smiley:

1 Like