# Optimizing Climate Model Output File Names and Directory Structures for Efficient Data Access with Intake

**URL:** <https://forum.access-hive.org.au/t/optimizing-climate-model-output-file-names-and-directory-structures-for-efficient-data-access-with-intake/2247>\
**Category:** Atmosphere\
**Tags:** model-evaluation\
**Created:** [23 July 2024 00:31 UTC](https://forum.access-hive.org.au/t/optimizing-climate-model-output-file-names-and-directory-structures-for-efficient-data-access-with-intake/2247 "2024-07-23T00:31:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rbeucher](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/rbeucher/32/11_2.png) [@rbeucher](https://forum.access-hive.org.au/u/rbeucher)\
**Post date:** [23 July 2024 00:31 UTC](https://forum.access-hive.org.au/t/optimizing-climate-model-output-file-names-and-directory-structures-for-efficient-data-access-with-intake/2247/1 "2024-07-23T00:31:29Z")

</div>

The Model Evaluation and Diagnostics team is working on improving how we find and load data on NCI Gadi. We want to start discussions on how we could enhance how the ACCESS model writes data to disk, which will help remove some constraints during the analysis phase. We believe this is crucial for better evaluating model outputs.

see topics:

> [@ACCESS-NRI Intake catalog: a way to find, load and share data on Gadi](https://forum.access-hive.org.au/t/access-nri-intake-catalog-a-way-to-find-load-and-share-data-on-gadi/1659):
>
> What is this? A topic that will be used to announce updates to the ACCESS-NRI Intake catalog at [NCI](http://nci.org.au). What is the ACCESS-NRI Intake catalog? The ACCESS-NRI Intake catalog provides a simple, community-driven approach to finding, loading & sharing ACCESS & ACCESS-related model data on Gadi. The catalog can be used from within your Python environment to discover & load data that meet your research needs without having to know where the data are or how they fit together. It’s also easy to add your d…

### Current Structure and Issues in the p73 Project

We’re encountering an issue with the output of the ACCESS-ESM model, specifically with the atmospheric data. The data is stored as follows:

```auto
./atm/netCDF/:
HI-CN-05.pa-185001_mon.nc
HI-CN-05.pa-185002_mon.nc
HI-CN-05.pa-185003_mon.nc
HI-CN-05.pa-185004_mon.nc
HI-CN-05.pa-185005_mon.nc
HI-CN-05.pa-185006_mon.nc
HI-CN-05.pa-185007_mon.nc
HI-CN-05.pa-185008_mon.nc
HI-CN-05.pa-185009_mon.nc
HI-CN-05.pa-185010_mon.nc

```

To analyze data across the entire historical period, we need to create an XArray DataArray or IRIS cube by piecing together all these files. Each file contains a lot of metadata and all the variables, which makes this process slow and resource-intensive.

### How to Improve

1. **One Variable per File:**

2. **Use Clear and Consistent File Names:**

3. **Time Segments:**

4. **Easy Metadata Extraction:**

5. **Borrow from CMIP Standards:**

### Model Evaluation Tools and CF Compliance

Most model evaluation tools are built for CMIP analysis and expect data to be CF-compliant and, in some cases, CMORised. CMORisation aligns data with CF (Climate and Forecast) conventions and CMIP standards, simplifying analysis and comparison. However, CMORisation is resource-intensive and should be avoided if possible.

@heidi @MartinDix @dougiesquire @RhaegarZeng