Cryosphere Community Datapool & ccdtools: Release Information

ACCESS-NRI Releases
, , , , ,
1

What is this?

A topic that will be used to announce release information and updates for the Cryosphere Community Datapool and its companion Python API ccdtools.

What are the Cryosphere Community Datapool and ccdtools?

The Cryosphere Community Datapool (CCD) is a joint data sharing and discovery project between ACCESS-NRI and the Cryospheric Sciences Working Group (CSWG). The CCD has been designed as an open-access and freely available cryospheric data resource for the Australian scientific community, maintaining a variety of datasets commonly used to parameterise ice sheet models. The CCD is hosted on the National Computational Infrastructure (NCI) Gadi supercomputer within project av17.

ccdtools is the Python interface developed by ACCESS-NRI that provides seamless access to the data located within the CCD. The ccdtools library offers an intuitive interface for discovering, accessing, and analysing diverse data sources including satellite observations, in-situ measurements, and model outputs. The library features a powerful dataset catalog system, enabling users to efficiently retrieve and work with data commonly used for ice sheet modelling, climate analysis, and cryospheric studies.

How is this topic used?

This topic will be the place where release information and updates to both Cryosphere Community Datapool and ccdtools will be announced, with details of what those updates entail.

What should I do to be notified of any updates?

You can watch this topic, and be notified by email of every update.

Where do I ask questions about releases?

Replies to this topic are disabled.

If you have specific questions about this release follow the guidelines for requesting help from ACCESS-NRI.

If you have questions about the cryosphere-datapool or ccdtools please create a topic in the relevant channel within Cryosphere > Cryosphere Community DataPool (CCD) community space. If you require assistance follow the guidelines for requesting help from ACCESS-NRI.

3

Cryosphere Community Datapool and ccdtools release

:rocket: Release: ccdtools v0.2.1

What can you expect to be able to do with this?

A user should be able to

  • install ccdtools into their own Python environment from PyPI, conda or GitHub.
  • discover, search and load datasets from the Cryosphere Community Datapool (CCD) on NCI Gadi directly from a Python workflow, script or notebook, returning Pandas DataFrames, GeoPandas GeoDataFrames or Xarray Datasets as appropriate.

About the Cryosphere Community Datapool (CCD)

The Cryosphere Community Datapool (CCD) is a joint data sharing and discovery project between ACCESS-NRI and the Cryospheric Sciences Working Group (CSWG). The CCD has been designed as an open-access and freely available cryospheric data resource for the Australian scientific community, and houses a variety of datasets commonly used to parameterise ice sheet models. The CCD is hosted on the NCI Gadi supercomputer within project av17.

Datasets are currently grouped into the following categories:

  • Elevation and geometry — digital elevation models and key geometry attributes, including bed topography/bathymetry, ice surface elevation, ice thickness, and ice/ocean masks.
  • Ice velocity — ice velocity parameters, including surface ice velocities and associated error estimates.
  • Geospatial — geospatial information commonly used to discretise ice sheet models, including ice sheet basin boundaries and grounding line locations.
  • Basal forcing — basal forcing fields, such as geothermal heat flux and ice shelf basal melt datasets.
  • Surface forcing — surface forcing fields, such as temperature and surface mass balance.
  • ISMIP6 simulation — results from ISMIP6 model simulations of Antarctica and Greenland.
  • ISMIP6 forcing — forcing parameters, such as surface mass balance, surface temperature, surface wind speeds and geothermal heat flux.

ccdtools is the open-source Python interface to the CCD, developed by ACCESS-NRI. A full list and description of the available datasets, including their Gadi paths and original sources, is maintained on the ACCESS-Hive Docs CCD page.

:loudspeaker: THIS IS A BETA RELEASE :loudspeaker:

This is a beta release while building the data repository, rolling out features and getting feedback from the community.

More information on release stages are available on the Hive Docs:

Release stages - ACCESS-Hive Docs

Feedback

We welcome feedback from all interested users. This is a chance to improve this proposed new release.

Please provide feedback through relevant channels within Cryosphere > Cryosphere Community DataPool (CCD) community space on this forum.

ACCESS-NRI is interested in all feedback, positive or negative, but in particular we would like to know:

  • Are the instructions clear and easy to follow? Is there anything unclear, missing or incorrect?
  • Are there datasets you use regularly that are missing from the CCD?

Prerequisites

The CCD data is hosted on NCI Gadi. To use it you need to:

How to use

Install from PyPI:

pip install access-ccdtools

or with conda, from the accessnri channel:

conda install accessnri::access_ccdtools

Then, from Python for an example dataset:

# 1. Initialize the catalog
catalog = DataCatalog()

# 2. Browse and search for datasets
results = catalog.search('ice velocity')
catalog  # View interactive catalog

# 3. Load data
data = catalog.load_dataset('dataset_name', version='v1', subdataset='sub1')

Features

:white_check_mark: Unified dataset catalog: a centralised catalog of cryosphere-focused datasets maintained in a flexible YAML configuration.
:white_check_mark: Dataset discovery and search: keyword search across dataset names, display names and descriptive tags (e.g. “antarctica”, “ice velocity”, “ice thickness”), plus an interactive catalog view.
:white_check_mark: Version and subdataset management: support for datasets with multiple versions and hierarchical subdatasets, enabling access to specific data variants and historical version comparisons.
:white_check_mark: Flexible data loaders: CSV, GeoPackage, Shapefile, GeoTIFF and NetCDF handled transparently, returning Pandas DataFrames, GeoPandas GeoDataFrames or Xarray Datasets.
:white_check_mark: Resolution and parameter filtering: for datasets with multiple resolutions or static/annual variants, the correct data subset is filtered and loaded automatically.
:white_check_mark: Available from PyPI and conda, with documentation and tutorials.

Common use cases

  • Model initialisation — quickly access ice sheet model input datasets (topography, velocities, boundaries).
  • Scientific analysis — discover and load multi-source cryosphere datasets for comparative studies.
  • Data discovery — explore available datasets and their properties through keyword search.
  • Version control — access specific dataset versions for reproducibility and validation studies.

Limitations

:stop_sign: The CCD data is hosted on NCI Gadi and is only accessible from Gadi (including ARE). ccdtools does not download or mirror data locally.
:stop_sign: Membership of the av17 and cm45 NCI projects is required.
:stop_sign: ccdtools is under active development. The API may change between releases.

Future developments

:construction: Continued expansion of the CCD dataset catalog in response to community requests.
:construction: Further development of the ccdtools API, loaders and tutorials.
:construction: Contributions from the community to expand both dataset offerings and enhance functionality.

To request a new dataset be added to the CCD, or an update to a currently supported dataset, please create an issue in the ccdtools GitHub repository using the New Dataset Request template. For broader discussions around cryospheric data, please start a thread in the Cryosphere Community Datapool category of the ACCESS-Hive Forum.

Credits (Latest developments only)

:star: Development of ccdtools (@lawrenceabird @mtetley @justinh2002)

:star: Curation and deployment of the Cryosphere Community Datapool (@lawrenceabird @mtetley)

:star: Documentation (@lawrenceabird @mtetley)

:star: Dataset suggestions and testing from the Cryospheric Sciences Working Group

Support

Replies to this topic are disabled.

If you have specific questions about this release please add a topic on this forum and tag with help (more information about support for ACCESS-NRI’s released software).

Resources