`benchcab`: Python-based software for the evaluation of CABLE

ACCESS-NRI Releases
, , , ,
1

This topic is to announce new releases of benchcab. benchcab is a Python-based software for the scientific evaluation of CABLE on NCI.

To receive email notifications when a new release is announced, please watch this topic.

:rocket: Release: 2.0.0

This is our first fully public release of this software. Please do not hesitate to give us feedback on this forum or GitHub so benchcab can become more useful to you.

Get started
Refer to benchcab’s User guide.

Features:

  • :white_check_mark: run several CABLE versions at various flux tower sites with the same compilation* and a standard set of science configurations.
  • :white_check_mark: choice of sets of flux tower sites to allow different levels of testing
  • :white_check_mark: bitwise comparison of CABLE outputs
  • :white_check_mark: CABLE outputs from benchcab is compatible with experiments on modelevaluation.org (workspace benchcab-evaluation)
  • :white_check_mark: compatible with CABLE v3 and v2 (limited testing of v2).
  • :white_check_mark: allow running tests of CABLE’s branches owned by others
  • :white_check_mark: versioned documentation, including how to use modelevaluation.org with benchcab outputs.
  • :white_check_mark: example work directory
  • :white_check_mark: allow running a customised set of science configurations on one branch (e.g. test new feature) or all branches.

*: the compilation may differ between the branches if the CABLE source codes are not both based on CABLE v3.

Limitations:

  • :stop_sign: no support for gridded (spatial) simulations
  • :stop_sign: limited standard set of science configurations. Feedback on the choice of configurations is welcomed. Please see this forum topic to leave some feedback.
  • :stop_sign: no testing with any spin-up. But starting the simulations from a given restart file is possible and easy to do.
  • :stop_sign: no testing with CASA
  • :stop_sign: analysis plots will be updated to allow easier comparison between science configurations and include more variables.
  • :stop_sign: limited to use on NCI’s supercomputer

:warning: Since benchcab is still in active development, future versions might not be backward-compatible

3 Likes
4

2 posts were merged into an existing topic: Analysis plots for benchcab

6

:rocket: Release: 3.0.0

Benchcab v3.0.0 has now been released. Please do not hesitate to give us feedback on this forum or GitHub so benchcab can become more useful to you.

Get started
Refer to benchcab’s User guide.

Features:

  • :white_check_mark: configure PBS job parameters at run time for fluxsite tests
  • :white_check_mark: delete namelist parameters for a specific model version with patch remove
  • :white_check_mark: added support for testing CABLE repositories on GitHub
  • :white_check_mark: general improvements and bug fixes

See the release notes for more details.

:warning: Configurations used with benchcab v2.0.0 need to be updated to be compatible with v3.0.0. The realisations option in the config.yaml file has a new format, see example.

2 Likes
7

:rocket: Release: 4.0.0

Benchcab v4.0.0 has now been released. Please do not hesitate to give us feedback on this forum or GitHub so benchcab can become more useful to you.

Get started
Refer to benchcab’s User guide.

Features:

  • :white_check_mark: Config changes - move experiment key under fluxsite, make project key optional.
  • :white_check_mark: Config addition - support local realisation paths for CABLE checkouts.
  • :white_check_mark: Add Endpoint for spatial workflows using payu (as of now, version >=1.0.30 is supported).
  • :white_check_mark: Using logging module for controlling and formatting standard output.
  • :white_check_mark: Deprecate serial_cable and parallel_cable scripts in older versions of CABLE.
  • :white_check_mark: Add new CLI argument - clean for resetting benchcab workflow stages.
  • :white_check_mark: general improvements in documentation, testing, versioning and CI/CD. Minor bug fixes.

See the release notes for more details.

:warning: Configurations used with benchcab v3.0.0 may need to be updated to be compatible with v4.0.0. For example, moving experiment key under fluxsite.

2 Likes