Sean Bryan: Benchcab: a scientific evaluation framework for the CABLE land surface model
Model developers often test the scientific validity of their modifications using ad-hoc scripts/tooling which target the scientific expertise of the model developer. A lot of the evaluation is also done via published results which may overrepresent positive effects on simulations. For the CABLE land surface model, we developed benchcab: a scientific evaluation framework to provide a standardised and systematic evaluation of the model. With benchcab, a user can run various CABLE configurations over a range of flux tower sites and generate the novel land model evaluation used in the PLUMBER2 model inter comparison project. The evaluation compares the land model against a suite out-of-sample empirical models, highlighting the key areas where the land model has the capacity for improvement in its input and output space. The results show a range of analyses, ranging from overall performance across several statistical metrics, all the way to detailed timeseries, and is consistent to allow comparison between model developments.