``invalid utf-8 sequence`` error

[polinash (Polina Sholeninova)]

I started getting error when simply loading ht in all my notebooks. Don’t I see something?
There are others who stumbled upon the same error but none of us figured it out (the error isn’t restricted to ht).

Code:

iaf_cycle3 = '01deg_jra55v140_iaf_cycle3'
catalog = intake.cat.access_nri
var='ht'
ht = catalog[iaf_cycle3].search(
    variable=var
).to_dask(
    
    threaded=False,
    xarray_open_kwargs = {            
    "decode_timedelta" : True}
)
ht = ht['ht']

Out of nowhere error:

KeyError: '01deg_jra55v140_iaf_cycle3'

During handling of the above exception, another exception occurred:

ComputeError                              Traceback (most recent call last)
Cell In[9], line 2
      1 var='ht'
----> 2 ht = catalog[iaf_cycle3].search(
      3     variable=var
      4 ).to_dask(
      5     
      6     threaded=False,
      7     xarray_open_kwargs = {            
      8     "decode_timedelta" : True}
      9 )
     10 ht = ht['ht']
     12 ht = ht.sel(yt_ocean=lat_slice)
...
   2333 # Only for testing purposes
   2334 callback = _kwargs.get("post_opt_callback", callback)
-> 2335 return wrap_df(ldf.collect(engine, callback))

ComputeError: invalid utf-8 sequence 

@CharlesTurner would you know what’s going on?

[CharlesTurner]

@polinash

KeyError: '01deg_jra55v140_iaf_cycle3'

During handling of the above exception, another exception occurred:

ComputeError                              Traceback (most recent call last)
Cell In[9], line 2
      1 var='ht'
----> 2 ht = catalog[iaf_cycle3].search(
      3     variable=var
      4 ).to_dask(
      5     
      6     threaded=False,
      7     xarray_open_kwargs = {            
      8     "decode_timedelta" : True}
      9 )
     10 ht = ht['ht']
     12 ht = ht.sel(yt_ocean=lat_slice)
...
   2333 # Only for testing purposes
   2334 callback = _kwargs.get("post_opt_callback", callback)
-> 2335 return wrap_df(ldf.collect(engine, callback))

ComputeError: invalid utf-8 sequence

This looks like a variable name has gotten misformatted in a catalog to me. Looking into it now.

[CharlesTurner]

Weird… Runs fine for me. @polinash can you confirm the notebook kernel you’re using?

Could you also let me know the output of

$ ls -l ~/.local/lib/python3.11/site-packages/

This is with conda/analysis3-25.09 (ie. plain conda/analysis3).

[polinash (Polina Sholeninova)]

I’m using 25.08.

@aekiss what variable/kernel did you load/use?

edit: I can confirm that switching to 25.09 fixes the problem

[aekiss (Andrew Kiss)]

I was using 25.08, so hopefully 25.09 will fix it. I’ll try that next time I run it.

Here’s my script with the error fyi access-om3-paper-1/notebooks/salt-vs-depth-time.ipynb at cef4dc4f022cd798f79fde760fcbfc1ffa25dfd1 · ACCESS-Community-Hub/access-om3-paper-1 · GitHub

[CharlesTurner]

Sorry this is a bit delayed - was in a meeting.

I can reproduce the same error with analysis3-25.08. Upgrading to analysis3-25.09 should fix the issue.

In late August, polars pushed some breaking changes. I think I was ill that week, so the updates to account for them unfortunately didn’t make it into that environment in time. I think 25.07 or lower should also work too, I think it’s just the 25.08 environment with the issue.