ACCESS-NRI Intake catalog: a way to find, load and share data on Gadi

ACCESS-NRI Releases
, , , , ,
1

What is this?

A topic that will be used to announce updates to the ACCESS-NRI Intake catalog at NCI.

What is the ACCESS-NRI Intake catalog?

The ACCESS-NRI Intake catalog provides a simple, community-driven approach to finding, loading & sharing ACCESS & ACCESS-related model data on Gadi. The catalog can be used from within your Python environment to discover & load data that meet your research needs without having to know where the data are or how they fit together. It’s also easy to add your data to the catalog so that others can find & use them.

See the documentation for more information: access-nri-intake-catalog.rtfd.io

What are we releasing?

ACCESS-NRI has released v0.1.0 of the access-nri-intake Python package. This package allows users to find relevant data from the ACCESS-NRI Intake catalog and open them into xarray Datasets for subsequent analysis.

Subsequent releases of the catalog will be posted in this topic.

What’s in the ACCESS-NRI Intake catalog?

Approximately 3 PB of ACCESS-related data are accessible through the catalog. These include the NCI-managed CMIP5 and CMIP6 data collections as well as a large number of non-CMIP ACCESS-OM2, ACCESS-CM2 and ACCESS-ESM1.5 model runs. Requests to add additional ACCESS-related data are encouraged and can be made via Github.

How to get started?

The access-nri-intake Python package is installed in the CLEX CMS “analysis3” conda environment and the ACCESS-NRI “access-med” conda environment. Instructions on how to utilise these conda environments on NCI’s ARE can be found in the ACCESS-NRI Intake documentation. The documentation also includes a quickstart tutorial demonstrating how to use the catalog from your Python environment.

What should I do to be notified of any updates?

You can watch this topic, and be notified by email of every update.

Where do I ask questions?

Replies to this topic are disabled.

We welcome feedback and contributions through the ACCESS-Hive forum. Please create a topic in the Technical category and follow the guidelines for requesting help from ACCESS-NRI should you need it. You can also open an issue on Github.

Useful links

2 Likes
3

04/03/2024

The catalog has been updated to v0.1.1

Updates

  • Updates to the developer documentation (#138 by @anton)
  • Added ACCESS-OM3 Builder for building datastores of ACCESS-OM3 output (#154 by @dougiesquire)
1 Like
4

28/03/2024

The catalog has been updated to v0.1.2

Updates

1 Like
5

07/05/2024

The catalog has been updated to v0.1.3

Updates

2 Likes
6

11/12/2024

access-nri-intake-catalog has been updated to version 1.0.0, and a new catalog has been built (v2024-12-10).

The new package is available on the conda/analysis3 environment on Gadi, available from project xp65.

Updates

  • The access-nri-intake-catalog Python package version (vX.Y.Z) has now been decoupled from the catalog version. Updated catalogs (with a version format vYYYY-MM-DD) can now be created without a new Python package release being required. All previous access-nri-intake-catalog catalog versions can be accessed by supplying a version keyword argument to intake.cat.access_nri. Use access_nri_intake_catalog.data.utils.available_versions to see the available iterations of the catalog.
  • Builders for new data sets (ERA5, CMIP5, CMIP6, MOM6, NARClim, BARPA, Cordex) have been added, and examples of each data type have been added to the new catalog (except for Cordex, which will follow at a later date).
  • The metadata validation script (metadata-validate) now provides a full report, rather than only giving the first validation issue found.
  • User, developer & maintainer documentation has been updated.
4 Likes