Cosima cookbook updating needs

COSIMA Workshops
1

We discussed briefly during the breakout sessions possible improvements to the cookbook:

  1. It is currently not scaling well with higher resolution models (>1/10 of a degree). Could this be improved?
  2. Many experiments in the database are not well documented and lack metadata. We should enforce mandatory metadata for the experiments indexed in the database.

Please add to the list any other points for improvement :slight_smile:

2

Good points @JuliaN.

We’ve long aspired to have some test codes to use as a basis for optimisation, but it hasn’t gone very far. Still a worthy goal!

The idea of only indexing data with sufficient metadata has been floated before, and I agree, it is long overdue.

3

Hi @JuliaN ,

Do you have a specific recipe in mind?
I am looking at setting up an environment on Gadi to do that sort of scaling exercise.
I do think that is something that we need to run routinely.
A use case to get us started would be great!

Thanks!

Romain

4

From the top of my head I remember that the overturning stream function example can be quite demanding when using the 0.1 degree output.

https://cosima-recipes.readthedocs.io/en/latest/documented_examples/Zonally_Averaged_Global_Meridional_Overturning_Circulation.html#gallery-documented-examples-zonally-averaged-global-meridional-overturning-circulation-ipynb

@AndyHoggANU might have more in mind?

1 Like
5

We also discussed that we want some sort of automation to ensure that all notebooks run using the latest conda-analysis environment and that no output has moved or deleted, etc.

This is related to Automating jupyterbook running with CI tools - #3 by navidcy

1 Like
6

Yes I second using overturning as a test example. Calculating time series of overturning for a couple of hundred years of the RYF takes a long time and is a metric that is used a lot.

7

This can be done relatively easily with an automated job that updates the kernel version in the file, without even having to run it. It is definitely something worthwhile to automate so that recipes are always correct when run at NCI.

8

OK. I’ll add that one to my To-do list then