Hi All. For those interested, I think I’ve found the ‘pythonic’ ‘correct’ way to deal with this. So the warning is coming from this function, but the catch is it’s being executed on the dask workers, and by default, dask workers don’t inherit any logging settings from the client, including logging.captureWarnings. Dask does have a method to have the workers forward their logs to the client to deal with, however, there is no in-built way to have logging.captureWarnings(True) run on all of the workers. So to have this happen, a simple worker plugin needs to be created following the instructions here:
from dask.distributed.diagnostics.plugin import WorkerPlugin
class CaptureWarningsPlugin(WorkerPlugin):
def setup(self,worker):
logging.captureWarnings(True)
def teardown(self,worker):
logging.captureWarnings(False)
And then after initialising the client run:
client.register_worker_plugin(CaptureWarningsPlugin())
This causes the setup method to run immediately on all of the workers. From there, if you’re not handling the py.warning logger, you’re done, all warnings.warn output from the dask workers will disappear into the aether. If, however, you are setting up custom logging handlers, you can run:
client.forward_logging()
(See here). This has dask workers send all logs back to the client, which now includes calls from warnings.warn, to be dealt with using whatever logging configuration has already been set up.
As usual with this kind of thing its way overcomplicated and @dougiesquire’s solution is just fine on a case-by-case basis. However, this could be implemented directly in cosima_cookbook to silence and/or control these warnings without changing the dataset chunking.