Dask remove time chunks for Fourier transforms

Yup, its most efficient to use the chunks argument when opening the data, where possible (see here). But this chunking is applied before the combine step in open_mfdataset. In your case, this means chunk sizes in time will be truncated at the length of the time dimension in the files you’re combining.

I think it’s probably a good idea to do all the chunking you can using the chunks argument in open_mfdataset, and only chunk time using the .chunk() method, e.g.,

eslice = xr.open_mfdataset(
    paths,
    chunks={"time": -1, "zi": 1, "xh": 500, etc},
    etc
).chunk(chunks={"time": -1})
4 Likes