ACCESS-NRI Intake Catalog now supports AM3

Atmosphere AM3
1

The ACCESS-NRI Intake Catalog now supports ACCESS-AM3. To turn your AM3 run into a temporary catalog, follow the “I’m a catalog contributor” instructions, using AM3.

Build the datastore using python (either Jupyter or interpreter directly):

from access_nri_intake.experiment import use_datastore
from access_nri_intake.source.builders import AccessAm3Builder
builder = AccessAm3Builder(path="/path/to/AM3/dir/").build()
builder.save(name="my-AM3-run", description="Model output from AM3 run", directory="path/to/desired/location")

Note that there are currently issues we are trying to work out with the CLI build-esm-datastore command, so use the direct python method for creating the datastore for now.

This has been fixed (thanks @CharlesTurner), so you can also create the datastore with the CLI:

build-esm-datastore --builder AccessAm3Builder --expt-dir  "/path/to/AM3/dir/" --cat-dir "/path/to/desired/location"

Then other users can load in the datastore:

import intake

cat = intake.open_esm_datastore(
    "/path/to/desired/location/my-AM3-run.json",
    columns_with_iterables=["variable"] # This is important
)

ACCESS-NRI have not yet decided on the details of the model runs to be included on a permanent basis in the catalog. Details to be decided include:

  • Where on Gadi should the outputs be stored?
  • Which configurations should we store? Should we only store outputs on major version releases? How long do we keep them around?
  • How long, in model time, should model runs be? How do we strike the balance between storage limitations and providing enough output to do meaningful analyses?

The intention is to provide visibility of model data to the community, and an easy way to track the performance of the model as it is being developed. If you have opinions on this, please reach out to us, either via this thread or elsewhere.

3 Likes