Community Talk: Sean Bryan (ACCESS-NRI)
Benchcab: a scientfic evaluation framework for the CABLE land surface model
Abstract
Model developers often test the scientific validity of their modifications using ad-hoc scripts/tooling which target the scientific expertise of the model developer. A lot of the evaluation is also done via published results which tend to overrepresent positive effects on simulations. For CABLE, we decided to develop benchcab: a scientific evaluation framework to provide a standardised and systematic evaluation of the model. With benchcab, a developer can run several versions of CABLE across a range of model configurations and then get access to a statistical analysis of the performed simulations via the modelevaluation.org platform. The results show a range of analyses from overall performance across several statistical metrics all the way to detailed timeseries. The analyses look at all aspects of a land surface model and are consistent to allow comparison between developments.
Please use this thread for further discussion on this talk.