The CMIP Rapid Evaluation Framework: A Pipeline for Fast, Reproducible Model Evaluation
Romain Beucher
Fast, reproducible evaluation becomes critical as Earth system models grow in complexity, especially with upcoming CMIP7 deadlines and the IPCC’s next assessment cycle. The Rapid Evaluation Framework (REF) is a community-driven platform designed to streamline the assessment of model outputs through an automated pipeline. REF brings together widely used tools already supported by ACCESS-NRI, such as ESMValTool and ILAMB, under one umbrella to simplify model evaluation.
At ACCESS-NRI, the Model Evaluation and Diagnostics (MED) Team contributed to the REF development and co-authored the EGUSphere paper describing its design. We are currently deploying a beta version of REF on Gadi, allowing the modeling community to evaluate ACCESS-ESM1.6 simulations being prepared for the FASTRACK initiative.
REF enables users to submit model outputs, choose diagnostics, and view results through a web interface available on the Australian Research Environment (VDI). It leverages best-available observations and supports rapid feedback loops for model developers. While REF is initially focused on CMIP7, we see great potential to extend its use across other projects and modeling efforts.
This presentation will introduce REF, demonstrate its current deployment, and outline plans for community use and feedback.
Please use this thread for discussion about this talk.