Automated Workflows for Management of Model Configurations
About
Model Configurations, released and supported by ACCESS-NRI, are stored within git repositories which allow changes to be automatically tracked.
Configurations are also versioned. Users can know that, if the major version number is the same, the configurations all produce the same answers - they are bitwise compatible. This is important when comparing experiments using these configurations, and when upgrading a configuration to a newer version.
Generic automated testing using GitHub Actions is performed on these configurations to ensure metadata correctness, bit-reproducibility (if required) and efficiency (in future work). These checks are run when
changes to the configuration are proposed (via pull requests), ensuring configurations perform as expected before they are merged.
Furthermore, scheduled self-tests are run on select configurations to verify that they continue to produce exactly the same answers over time.
This configuration management workflow produces high-quality model configurations and allows developers to skip menial tasks. It also provides certainty to users that any changes in the output of an experiment are a result of their choices, not due to an unexpected system or configuration change.
How are you handling sharing settings between different configurations? For instance different CMIP experiments use substantially the same settings with different inputs, would a configuration update need to be applied to each individually or is there a way to add the update simultaneously to all?
Sharing settings between configurations (configuration branches in a config repository such as access-nri/access-om2-configs) would at the moment require cherry-picking the commit that modifies the settings, and adding it to all of the relevant dev-* config branches so we can run our QA and repro tests against those particular configurations. Going to tag @Aidan to make sure that I understand the question correctly.
Happy to discuss new solutions for sharing updates between configurations, as well.
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
4
@micael made a cherry-pick workflow to automate this
This could be extended to add more functionality as use-cases arise.
