Disconcertingly, we now have a 2nd example of a non-reproducible run. It’s unclear how often this occurs, as we don’t discover it unless we re-run and check the restarts.
The lesson is: always check the md5 hashes in the manifests in a re-run - they should match those from the original run (the only exception being ocean_barotropic.res.nc.* which always differ - but this doesn’t affect any other restarts so is presumably benign, maybe a datestamp in the file or something).