Shapely fails to import in analysis3-25.04: undefined symbol GEOSGeom_getExtent_r

Technical Infrastructure
,
1

Hi team,

This environment was working yesterday, but today I’m seeing this error in analysis3-25.04:

ImportError: shapely/lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: GEOSGeom_getExtent_r

Traceback:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 3
      1 import sys, os#, pygmt
      2 sys.path.insert(0, '/home/581/da1339/AFIM/src/AFIM/src')
----> 3 from sea_ice_processor          import SeaIceProcessor
      4 from sea_ice_plotter            import SeaIcePlotter
      5 from grounded_iceberg_processor import GroundedIcebergProcessor

File ~/AFIM/src/AFIM/src/sea_ice_processor.py:5
      3 import pandas as pd
      4 import numpy  as np
----> 5 import xesmf  as xe
      6 from pathlib      import Path
      7 from datetime     import datetime, timedelta

File /g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/xesmf/__init__.py:3
      1 # flake8: noqa
----> 3 from . import data, util
      4 from .frontend import Regridder, SpatialAverager
      6 try:

File /g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/xesmf/util.py:5
      3 import numpy as np
      4 import xarray as xr
----> 5 from shapely.geometry import MultiPolygon, Polygon
      7 try:
      8     import esmpy as ESMF

File /g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/shapely/__init__.py:1
----> 1 from shapely.lib import GEOSException  # NOQA
      2 from shapely.lib import Geometry  # NOQA
      3 from shapely.lib import geos_version, geos_version_string  # NOQA

ImportError: /g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/shapely/lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: GEOSGeom_getExtent_r

This appears to be a mismatch between the shapely Python package (which expects GEOS 3.11+) and the GEOS C library on the system.

Can someone please check whether the linked libgeos_c.so in this environment is out of sync with the installed version of shapely?

Thanks!

2

I believe that solution might be as simple as:

conda install shapely=1.8

or maybe downgrade both

conda install shapely=1.8 geos=3.10
3

Believe the problem is effecting multiple libraries, not just shapely.

/g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/xarray/backends/plugins.py:110: RuntimeWarning: Engine 'rasterio' loading failed:
/g/data/xp65/public/apps/med_conda/envs/analysis3-25.04/lib/python3.11/site-packages/rasterio/../../.././libspatialite.so.8: undefined symbol: GEOSMakeValidParams_create_r
  external_backend_entrypoints = backends_dict_from_pkg(entrypoints_unique)

That reported when doing this:

P_kmt_mod = "/g/data/gv90/da1339/grounded_icebergs/MyThin_db/kmt_mod_thinned-0p75_v1p50.nc"
P_kmt_org = "/g/data/gv90/da1339/grounded_icebergs/MyThin_db/KMT_original_reformatted_with_coordinates.nc"
ds_mod    = xr.open_dataset(P_kmt_mod)
ds_org    = xr.open_dataset(P_kmt_org)
kmt_mod   = ds_mod['kmt'].values
kmt_org   = ds_org['kmt'].values
lat       = ds_mod['lat']
lon       = ds_mod['lon']
print("MOD KMT LAND CELL COUNT:", np.sum(kmt_mod == 1))
print("ORG KMT LAND CELL COUNT:", np.sum(kmt_org == 1))
print("Total mismatches:", np.sum(kmt_mod != kmt_org))
print("Unique values in KMT MODIFIED:", np.unique(kmt_mod))
print("Unique values in KMT ORIGINAL:", np.unique(kmt_org))
assert kmt_mod.shape == kmt_org.shape == lat.shape == lon.shape
mask_diff = (kmt_org == 1) & (kmt_mod != 1)
lat_diff  = lat.where(mask_diff)
lon_diff  = lon.where(mask_diff)
mask_flat = ~np.isnan(lat_diff)
lat_diff  = lat_diff.values[mask_flat]
lon_diff  = lon_diff.values[mask_flat]
print(f"🔍 Found {lat_diff.size} grid cells where 1s were removed")

That is, after commenting out import xesmf

4

Thanks for the report @dpath2o.

This is in scope for ACCESS-NRI support. I will find someone who can assist you with this.

6

Hi @dpath2o,

The analysis3-25.04 environment is frozen. I have just tried with analysis3-25.05 which has shapely v2.0 and geos 3.12. It looks like it is importing properly. Can you try for me please and let us know?

8

Thanks @rbeucher . Yes, import xesmf is working now for conda/analysis3 kernel in /g/data/xp65/public/modules. Sorry but I don’t have specific options for analysis3-25.05 when I start my ARE session using /g/data/xp65/public/modules. With hh5, I did have a range of kernels to choose from. I’m probably not doing something to enable this feature in ARE with xp65. Do you have a suggestion? (I realise that this is separate to this thread. So I understand if you can’t answer that here).

10

@dpath2o

Since a solution was found for your initial question, I will close this topic.

If you still have problems finding kernels, please check if there are already related topics in the forum, otherwise feel free to open a new topic about that.

Thank you.