From analysis3 26.07 onwards, xarray hits "Importerror" on compute nodes

Technical Infrastructure
,
1

From 26.07 onwards, a command like this:

import xarray
xarray.open_dataset("/g/data/gb02/ab8992/rom3-inputdirs/sots5km/hgrid.nc")

throws this error:

/g/data/xp65/public/apps/med_conda/envs/analysis3-26.08/lib/python3.12/site-packages/argopy/utils/lists.py:38: UserWarning: An error occurred while loading the ERDDAP data fetcher, it will not be available !
<class 'requests.exceptions.ConnectionError'>
HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /IrishMarineInstitute/awesome-erddap/master/erddaps.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x14644da53fe0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

when you’re running python on a compute node with no internet access. I’m not sure exactly what’s changed, but it seems like argopy is trying to fetch some kind of external data and this triggers every time you try to use xarray.

It’s annoying, but also causes a slowdown to your compute jobs since xarray doesn’t appear to start running until after it’s hit the max HTTP retries

I have no idea what the fix is but I’d suggest that the default behaviour of packages on analysis3 shouldn’t be to try fetch libraries from the internet

2

I’d seen this a few times and thought it was just an irritation rather than causing a meaningful delay.

Shouldn’t be too hard to chop this behaviour out of the environment, hopefully be sorted later today

3

This turns out to be an import time side effect in the argopy package.

I’m in the process of submitting a fix there, so this might take (quite) a bit longer than expected.

Fix is in, we just need to clear up a xarray version pin and then get a new minor release of argopy out.