Introduction
ACCESS-NRI (@dougiesquire) has developed a “Meta-Intake” catalog of a number of datasets located at NCI.
More information is available on his workshop poster:
New developments
Martin Durant (orcid.org/0009-0007-2516-6831) is working on Intake 2, an ambitious redesign of Intake that would make it significantly more capable and extensible.
He presented some of the work he has already done and his proposed design in a pangeo showcase to get feedback from the community about what they needed, and if, in fact, this was a useful thing to do:
I was particularly interested in the proposed support for pipelines: intake catalogs could be of data, or views into transformations of multiple datasets.
This would be a very useful capability. A catalog could contain an entry into raw model data, as well as a CMORised view of the same data, with altered meta-data, to be compatible with tools that require CMORised inputs, e.g. ESMValtool. It would also be possible to have the same data available at different resolutions targeting specific use-cases, and generate the regridding on the fly. Combine that with proposed support for caching and it would be possible to offer a number of different regridded end-points that could be dynamically generated, with the most used retained in the cache.
This is potentially quite transformative.