Intake warning

I’m using the latest version of intake in xp65 for the first time. Can someone help me silence this warning when using the .search() function?

/g/data/xp65/public/apps/med_conda/envs/analysis3-25.02/lib/python3.11/site-packages/intake_esm/source.py:82: FutureWarning: In a future version of xarray decode_timedelta will default to False rather than None. To silence this warning, set decode_timedelta to True, False, or a 'CFTimedeltaCoder' instance.

Hi @JuliaN
A simple way to filter that would be to add:

import warnings
warnings.simplefilter('ignore', FutureWarning)

at the top of your script, or before you use the intake catalog command that spawns the warning.
If you want to ignore that only for certain lines, you can instead do:

with warnings.catch_warnings():
    warnings.simplefilter("ignore", FutureWarning)
    ... # Lines that would generate a FutureWarning

Cheers
Davide

Hi Davide! Yes, I already have the warnings filter in my script but it does nothing :frowning:

Hmm not sure why this is not working, there might be a few reasons though.

Can you please post the lines of code that you are running and that spawn the warning?

Sure:

import intake
import xarray as xr

catalog = intake.cat.access_nri
exp = "01deg_jra55v140_iaf_cycle3"
data = catalog[exp].search(variable = 'tx_trans_int_z', frequency = '1mon').to_dask()

My ignorant 2 cents is that this has to be silenced/fixed in the intake source (?) code.

Which xp65 environment are you using?
Using conda/analysis3-24.11 the code you referenced fails:

Failed to load dataset with key='ocean_2d_tx_trans_int_z_1_monthly_mean_ym_XXXX_XX.1mon'

I am using 25.02 as per 21st Feb 2025 - Building and Using Intake Datastores

Hmm, I am sorry but also with conda/analysis3-25.02 the code you referenced above is still failing for me:

File "/g/data/xp65/public/apps/med_conda/envs/analysis3-25.02/lib/python3.11/site-packages/intake_esm/source.py", line 292, in _open_dataset
    raise ESMDataSourceError(
intake_esm.source.ESMDataSourceError: Failed to load dataset with key='ocean_2d_tx_trans_int_z_1_monthly_mean_ym_XXXX_XX.1mon'
                 You can use `cat['ocean_2d_tx_trans_int_z_1_monthly_mean_ym_XXXX_XX.1mon'].df` to inspect the assets/files for this key.

I am not sure I will be able to help you if I cannot reproduce the warning, sorry.

It might be because I don’t have access to cj50.
I requested it in case.

1 Like

ok, no worries! I wonder if maybe you are missing access to projects.

1 Like

The error here is not of the most friendly!
As soon as I’ll get approved I’ll try again :smiley:

Ok I got approved at a speedy time.

I can confirm the following:

import intake
import xarray as xr
import warnings

catalog = intake.cat.access_nri
exp = "01deg_jra55v140_iaf_cycle3"
with warnings.catch_warnings():
    warnings.simplefilter("ignore", FutureWarning)
    data = catalog[exp].search(variable = 'tx_trans_int_z', frequency = '1mon').to_dask()

doesn’t spawn any warning for me.

Wow, so strange!

1 Like

This is so weird.
I’m not using ARE, and from the terminal it works correctly (note the second time if I issue the command without warning filter it spawns the warnings)

Not really sure what’s going on here, sorry

Right, I think it is because I had a dask client open:

from dask.distributed import Client
client = Client(threads_per_worker=1)

without the client the warning suppression works! Although I would like to keep using dask..

1 Like

Ah!
Yes dask clients have their own way to manage warnings.
You can see a related approach to remove warnings with the dask client enabled here. Maybe a related approach would work in your case too.

Ok, awesome ! I’ll check it out tomorrow. Sorry for taking up your time :roll_eyes: should have added the client at the beginning

1 Like

It’s all good!
This forum exists exactly for these issues :slight_smile:

Haha sorry, I’ve tried that solution and it didn’t work either :frowning: This is the minimal working example:

import intake
import xarray as xr

import warnings # ignore these warnings
warnings.filterwarnings("ignore", category = FutureWarning)
warnings.filterwarnings("ignore", category = UserWarning)
warnings.filterwarnings("ignore", category = RuntimeWarning)

from dask.distributed.diagnostics.plugin import WorkerPlugin
class CaptureWarningsPlugin(WorkerPlugin):
    def setup(self,worker):
        logging.captureWarnings(True)
    def teardown(self,worker):
        logging.captureWarnings(False)
        
from dask.distributed import Client
client = Client(threads_per_worker=1)
client.register_worker_plugin(CaptureWarningsPlugin())

catalog = intake.cat.access_nri
exp = "01deg_jra55v140_iaf_cycle3"

data = catalog[exp].search(variable = 'tx_trans_int_z', frequency = '1mon').to_dask()

Hi @JuliaN,

This warning

FutureWarning: In a future version of xarray decode_timedelta will default to False rather than None. To silence this warning, set decode_timedelta to True, False, or a 'CFTimedeltaCoder' instance.

comes from xarray, and is basically due to them changing a default from None to False. You can silence it with

data = catalog[exp].search(
    variable='tx_trans_int_z', frequency='1mon'
).to_dask(
    xarray_open_kwargs = {"decode_timedelta" : False}
)

This won’t change any behaviour & will silence the warnings.


Some more bits & pieces of relevant info

You are (very nearly) correct - it needs fixing in intake-ESM - we’ve actually already merged the fix, but we haven’t updated the environment with the fix yet.

Regarding silencing warnings when we have threads on the go, I’m yet to find a good way to do this.

Setting

warnings.simplefilter("ignore")
os.environ["PYTHONWARNINGS"] = "ignore"

before you initialise any threads (ie. open a Dask Client) seems to work, but I think I’ve found it a little unreliable - it needs to go right at the start of the notebook. You can make it reliably crush the warnings by setting it in the environment variables section of an ARE Session:

However, I wouldn’t recommend doing this - it’ll turn off all warnings for the ARE Session, which might include important ones.

In general, warnings are important information that things aren’t behaving as they should (although not badly enough to warrant crashing the program), and turning them off leaves you open to subtle bugs.

EDIT:
I just reread this & I just want to be explicit: I’m not anti capturing warnings in specific instances, as suggested by @atteggiani (eg. with warnings.catch_warnings).

However, disabling all warnings for the entire python session is a really bad idea, in my opinion. If it does come to disabling warnings like this, I’d suggest using PYTHONWARNINGS="once" instead, which should suppress subsequent warnings of the same kind. This is a bit safer.

5 Likes