COSIMA recipes / on-ramping for new people

[Aidan (Aidan Heerdegen, ACCESS-NRI Release Team Lead)]

@aekiss

Cosima cookbook + Big data

Some (or most) of the notebook examples do not scale well when using higher resolution models (>1/10 of a degree). This should be a necessary improvement moving forward (ACCESS NRI)

Hopefully this update to dask:

Latest version of dask (2022.11.0) could fix many workflow issues

The latest version of dask (2022.11.0) uses a new mode of scheduling by default that can significantly reduce the memory usage in a lot of typical geoscience workflows. The new default tries to address the problem of “root task overproduction” - if you’ve ever inexplicably run out of memory while computing a climatology, for example, you were probably experiencing root task overproduction. Details here: Reducing memory usage in Dask workloads by 80%. It would be great to explore/benchmark any i…

will solve some of these issues.