Share your experience working with ACCESS models and data

Hi everyone,

I’m working with ACCESS-NRI to collect some ideas for the future direction of our web services. Are there any areas in your work with ACCESS models or data that could benefit from better tooling? Please let us know in this thread.

For example, some feedback we’ve had so far is difficulty in finding and working with input datasets, and lack of access to data in the cloud.

4 Likes

This is something I perceive as as an issue but I am not sure how much people agree. If researchers publish using ACCESS data, it is impractical for them to actually provide the data with the publication (with the exception of the CMIP datasets). So even if they provide their code, its basically impractical for others to replicate their analysis. This is something cloud access could help with (obv noting bandwidth limitations)

Also, there are some science / government organisations that would be interested in ACCESS outputs but are not big NCI users. e.g. cloud access should make it easier to access for researchers using R/Matlab/QGIS on their local machines / a different super-computer / cloud machines.

2 Likes

There are third party providers of “data lake” platforms for exactly our sort of complex, multidimensional data. Earthmover springs to mind.

https://www.eratos.com/products is also another one which has had some local discussion recently.

An in house solution might be better for us, (e.g. a STAC catalogue of zarr assets / files ? )

I’m wary of the big cloud providers, as they have a tendency to use research data as a honey pot to draw users into their paid services.

A public/gov alternative is the Pawsey Acacia object store, where the same data is available internally and also publicly: access is controlled but the data owner.

To me that is the future for ACCESS experiment data and similar. Seamless access internally and externally.

1 Like

To be clear, Earthmover is not a big cloud provider. Their main (only?) product Arraylake is a “data lake platform” that is compatible with all major object storage services, on prem or in the cloud. It provides data cataloging, version control, access control for multidimensional data like ours. I think it looks pretty neat - there’s a demo in this video at 11.18s.

(No I don’t receive any form of payment from Earthmover :grin:)

1 Like

Oh I didn’t mean to imply they are.

That does sound amazing.