# Warnings in xp65 conda env

**URL:** <https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133>\
**Category:** Ocean\
**Tags:** help\
**Created:** [27 August 2025 01:35 UTC](https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133 "2025-08-27T01:35:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![polinash](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/polinash/32/3258_2.png) [@polinash](https://forum.access-hive.org.au/u/polinash)\
**Post date:** [27 August 2025 01:35 UTC](https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133/1 "2025-08-27T01:35:59Z")

</div>

I am adapting my past scripts for new conda env (transition hh5→xp65) and noticed that my jobs sometimes get thousands line of warnings. I’ll copy a few most common ones and would like to get advice how to avoid them:

`/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/core.py:259: FutureWarning: When grouping with a length-1 list-like, you will need to pass a length-1 tuple to get_group in a future version of pandas. Pass (name,) instead of name to silence this warning.`  
`records = grouped.get_group(internal_key).to_dict(orient=‘records’)`  
`/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/source.py:280: ConcatenationWarning: Attempting to concatenate datasets without valid dimension coordinates: retaining only first dataset. Request valid dimension coordinate to silence this warning.`  
`warnings.warn(`

Also, it got a hundred of the same warning for one specific line: `/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/source.py:82: FutureWarning: In a future version, xarray will not decode timedelta values based on the presence of a timedelta-like units attribute by default. Instead it will rely on the presence of a timedelta64 dtype attribute, which is now xarray’s default way of encoding timedelta64 values. To continue decoding timedeltas based on the presence of a timedelta-like units attribute, users will need to explicitly opt-in by passing True or CFTimedeltaCoder(decode_via_units=True) to decode_timedelta. To silence this warning, set decode_timedelta to True, False, or a ‘CFTimedeltaCoder’ instance.`  
`ds = xr.open_dataset(url, **xarray_open_kwargs)`

My guess that the warnings are related to loading variables from intake catalog. Here’s how I load them:

```python
        variable = 'salt'
        tracer = catalog[iaf_cycle3].search(
            variable=variable,
            frequency='1mon'
        ).to_dask()
    

```

---

<div class="post-metadata">

**Author:** ![polinash](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/polinash/32/3258_2.png) [@polinash](https://forum.access-hive.org.au/u/polinash)\
**Post date:** [27 August 2025 02:36 UTC](https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133/2 "2025-08-27T02:36:35Z")

</div>

After giving all these warnings, the job finished unsuccessfully with the following error:

```auto
2025-08-27 12:29:09,548 - distributed.worker - ERROR - Failed to communicate with scheduler during heartbeat.Traceback (most recent call last):File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 226, in readframes_nosplit_nbytes_bin = await stream.read_bytes(fmt_size)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^tornado.iostream.StreamClosedError: Stream is closed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/worker.py”, line 1267, in heartbeatresponse = await retry_operation(^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/utils_comm.py”, line 416, in retry_operationreturn await retry(^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/utils_comm.py”, line 395, in retryreturn await coro()^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/core.py”, line 1259, in send_recv_from_rpcreturn await send_recv(comm=comm, op=key, **kwargs)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/core.py”, line 1018, in send_recvresponse = await comm.read(deserializers=deserializers)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 237, in readconvert_stream_closed_error(self, e)File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 137, in convert_stream_closed_errorraise CommClosedError(f"in {obj}: {exc}") from excdistributed.comm.core.CommClosedError: in <TCP (closed) ConnectionPool.heartbeat_worker local=tcp://127.0.0.1:38990 remote=tcp://127.0.0.1:35201>: Stream is closed

```

---

<div class="post-metadata">

**Author:** ![CharlesTurner](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/charlesturner/32/2518_2.png) [@CharlesTurner](https://forum.access-hive.org.au/u/CharlesTurner)\
**Post date:** [27 August 2025 03:01 UTC](https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133/3 "2025-08-27T03:01:55Z")

</div>

Hi Polina,

Thanks for posting this - these are all coming out of intake-esm, and one in particular I’ve been meaning to fix for a while, but couldn’t remember how to recreate it 😅.

1. `/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/core.py:259: FutureWarning: When grouping with a length-1 list-like, you will need to pass a length-1 tuple to get_group in a future version of pandas. Pass (name,) instead of name to silence this warning.`  
`records = grouped.get_group(internal_key).to_dict(orient=‘records’)`

Very much on my todo list, I’ll fix it now but it’ll be a few weeks before the fix is in the xp65 environment. Unfortunately it can’t be suppressed right now. I’ll update this comment when it is fixed.

1. `/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/source.py:280: ConcatenationWarning: Attempting to concatenate datasets without valid dimension coordinates: retaining only first dataset. Request valid dimension coordinate to silence this warning.`  
`warnings.warn(`

I think that this should only happen when you load coordinate variables. I think you can work around it with something like:

```python
# Generates warnings
esm_datastore = esm_datastore.search(variable='geolon_t').to_dask() 

```

```python
esm_datastore = esm_datastore.search(variable='geolon_t')
esm_datastore = esm_datastore.search(path = esm_datastore.unique().path[0]).to_dask()
# Does not generate warning

```

This effectively limits us to the first result, only opens a single file, and so shouldn’t generate the warning.

1. `/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/intake_esm/source.py:82: FutureWarning: In a future version, xarray will not decode timedelta values based on the presence of a timedelta-like units attribute by default. Instead it will rely on the presence of a timedelta64 dtype attribute, which is now xarray’s default way of encoding timedelta64 values. To continue decoding timedeltas based on the presence of a timedelta-like units attribute, users will need to explicitly opt-in by passing True or CFTimedeltaCoder(decode_via_units=True) to decode_timedelta. To silence this warning, set decode_timedelta to True, False, or a ‘CFTimedeltaCoder’ instance.`  
`ds = xr.open_dataset(url, **xarray_open_kwargs)`

I think this is related to a change in defaults in xarray (annoyingly, we fixed a closely related one a couple of months ago). To get around it, you can do:

```python
variable = 'salt'
        tracer = catalog[iaf_cycle3].search(
            variable=variable,
            frequency='1mon'
        ).to_dask(
    xarray_open_kwargs = {
        "decode_timedelta" : True, # or False, or a 'CFTimedeltaCoder' instance 
)

```

as with 1, this can be fixed in intake-esm, so I’ll do that. Not sure how quick it’ll be though as I don’t fully understand the implications of changing the default for other users yet.

Cheers!

---

<div class="post-metadata">

**Author:** ![CharlesTurner](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/charlesturner/32/2518_2.png) [@CharlesTurner](https://forum.access-hive.org.au/u/CharlesTurner)\
**Post date:** [27 August 2025 03:02 UTC](https://forum.access-hive.org.au/t/warnings-in-xp65-conda-env/5133/4 "2025-08-27T03:02:59Z")

</div>

> [@polinash](#):
>
> ```auto
> 2025-08-27 12:29:09,548 - distributed.worker - ERROR - Failed to communicate with scheduler during heartbeat.Traceback (most recent call last):File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 226, in readframes_nosplit_nbytes_bin = await stream.read_bytes(fmt_size)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^tornado.iostream.StreamClosedError: Stream is closed
> The above exception was the direct cause of the following exception:
> Traceback (most recent call last):File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/worker.py”, line 1267, in heartbeatresponse = await retry_operation(^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/utils_comm.py”, line 416, in retry_operationreturn await retry(^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/utils_comm.py”, line 395, in retryreturn await coro()^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/core.py”, line 1259, in send_recv_from_rpcreturn await send_recv(comm=comm, op=key, **kwargs)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/core.py”, line 1018, in send_recvresponse = await comm.read(deserializers=deserializers)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 237, in readconvert_stream_closed_error(self, e)File “/g/data/xp65/public/apps/med_conda/envs/analysis3-25.07/lib/python3.11/site-packages/distributed/comm/tcp.py”, line 137, in convert_stream_closed_errorraise CommClosedError(f"in {obj}: {exc}") from excdistributed.comm.core.CommClosedError: in <TCP (closed) ConnectionPool.heartbeat_worker local=tcp://127.0.0.1:38990 remote=tcp://127.0.0.1:35201>: Stream is closed
> 
> ```

I think this is just a dask communication error - can you confirm if it fail with the same error if you retry the script verbatim?

EDIT: I’ve just tried the code snippet you passed me, and I get a segmentation fault. This is related to another, much more complicated problem, but can be avoided by passing `threaded=False` in the `to_dask()` call: eg.

```python
variable = 'salt'
        tracer = catalog[iaf_cycle3].search(
            variable=variable,
            frequency='1mon'
        ).to_dask(
    threaded=False,
    xarray_open_kwargs = {
        "decode_timedelta" : True, # or False, or a 'CFTimedeltaCoder' instance 
)

```

If running the script again fails in the same way, could you let me know if this fixes it (N.B If so, this is a really handy test case that I know I can reliably trigger a segfault with, so I’d be really grateful to know the result either way).