Earthmover.io are open sourcing Icechunk, the back-end software for their ArrayLake offering:
This is a seriously impressive bit of software that adds transactions and versioning to zarr datastores.
1 Like
I’m hoping Pawsey HPC engineers are having a serious look for their warm tier object storage cluster, Acacia? And given the hope that NCI will increasingly include on-premises object store capacity that NCI HPC engineers are getting across this to? @Aidan, what is your view on best ways to socialise this across the community?
Find some like-minded folks around here and look into uses cases? I know @anton and @MartinDix are very keen on improving versioning of important data, like model inputs.
Proof of concept demos?
Get someone who is familiar with it to give a presentation?
1 Like
We might get a list together of key people and then ask EarthMover folks to think about tailoring a virtual talk about how to use Icechunk for on-prem object store?
1 Like
1 Like