clairecarouge
(Claire Carouge, ACCESS-NRI Land Modelling Team Lead)
122
It is now Friday, and as announced previously, we are making this discussion a public topic.
As part of the changes, you are now all members of the ESM1.6 developers group. This group has been associated with the esm16-dev-mailing tag so that all members of the group will automatically watch all posts (i.e. receive an email) in topics with this tag.
The tag can only be used by group members, use it with care to avoid spamming too many people.
The group has been created using the list of people coming to the ESM1.6 stand-ups. Feel free to change your watching level using the icon on this topic or to leave the group as necessary.
Let me know if ESM1.6 developers do not receive updates from this topic and should be added to the group.
We are also closing the ESM1.6 land chat to make this topic our main way of communication.
Do we need to include some version information for payu as part of this?
Also (mainly for later use) it would be good to find a way to prompt people to include specific detail around
if this experiment is a branch off an earlier/existing experiment (or a fresh start) and which one.
year of branching (if appropriate)
Later on we’ll be looking to run multiple e.g. historical simulations (for config testing and actual production runs) so keeping track of which spin up/control they start from and from where in those runs will be important. This likely should go in the description/notes sections but probably better to make it part of the template somehow.
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
125
Most of this information, and much of what is listed above, is captured automatically by payu in the git history, as long as branching is done from the correct commit.
Branching from a specific commit is supported by payu:
To create a new git branch starting from a tag or commit, use -s/–start-point flag:
Re data management for spin-up / pre-spin-up runs. For the atmosphere, is there any need to keep the daily output files? This would reduce atmosphere output by at least 50%.
I’d meant to bring up in the meeting – the Spinup presets from ESM1.5 haven’t been ported over to the ESM1.6 configurations yet.
Would these be saving the right variables that should be kept, perhaps with added WOMBAT-lite specific variables? Using these presets would reduce the total output by to ~1/5th of the size.
I think we would need to output, post-process and then delete these files rather than not produce them. Some key metrics - I’m thinking averages of the daily max/min temperatures in particular - are best obtained that way. It really depends on the detail of benchmarking/assessment that is being considered.
There is the more general method of reducing the number of variables produced by tweaking the STASH-C request (especially for daily variables).
I think this also requires updating the UM version. Could someone that knows about the UM developments please chime in on this PR with a version we should use?
@pearseb, I appreciate you couldn’t make it today but but please add experiment for current spin up experiments to this repo’. Brief instructions here and repo here. Get in touch if any issues. (Awesome to see this in use already!)
@RachelLaw@tiloz@ClareCR@clairecarouge; any thoughts on the best way to proceed for a DMP? (Great to see some thoughts above about reducing output size.)
Harshula expressed the need to use a newer compiler and OpenMPI version for future test runs. Namely openmpi: 4.1.5 / ifort: 2021.10.0, see pr. We have discussed this offline but are unaware of any issues regarding the timing of this change. @ESM16-dev; please let @harshula and @cbull know as soon as possible if you are concerns about the timing of this.
For switching compilers + MPI stack, it would be better to switch to oneAPI instead of another classic intel compiler now, and then switching to oneAPI again soon after. The biggest downside is that ESM1.6 does not yet have a spack package recipe that can be compiled by the oneAPI suite; whereas the later version of the classic compiler presumably can be swapped in easily.
I have individually compiled all the components with oneAPI - now it is a matter of fixing up the individual spack recipes for the components.
Hello everyone. I’ve been responsible for upgrading the Prerelease infrastructure across the model deployment repositories, and ACCESS-ESM1.6 is the final repository to be updated - see the relevant pull request here.
I am looking to get this pull request merged, and, when it is, I will need all the pull request branches to be rebased onto the new main, so they can use it!
If you all don’t mind, I can rebase these branches - or you can rebase when you want to.
If you want me to rebase the branches, I can message this chat when:
I’ve rebased the branches - @Jhan and others, you may git pull. If there are errors, you might have to run git reset --hard origin/YOUR_PR_BRANCH_NAME. Please tag me or message me and I can help out if needed.
You may notice that the pull request doesn’t quite look as you left it - the commits have all been moved to the end of the pull request, and the comments to the start - this is just a quirk of GitHubs handling of rebases. Rest assured that your Prereleases are still as you left them, and the commits were used to create them are still in the repository, but just not on the branch.
For some branches, you may note that it states the deployment has failed. This is because I’d done the rebases quite quickly and had accidentally run afoul of GitHubs deployment scheduling, which is something I’m looking into. Do note that if your deployment worked before, it would still work now.
Again, please tag me if you need a hand with your local repos, or have any other questions!
Thanks @cbull … Looking at both this post and the minutes, my advice would be that if there is no one in your teams who has been assigned to do the DMP then I’d recommend you escalate to Andy/Martin or other agencies to see if someone can be assigned. You might need to explain to them what the impact/consequence is of not having a DMP, and the extra work it ends up making for your all. I am sorry that I can’t offer to assist but CMIP7 is not currently on my work plan and I don’t know what ACCESS-NRI has agreed to commit at this time.
However, in case your question was more about the next steps then I’ve had a quick look at the Data Governance Plan doc that Rachel shared and while it needs a review/updating, particularly regarding what each agency is contributing and estimates of effort, I think the details for the Data Management Plan in Appendix A would be a great start for what you should be documenting.
Re the specific question about ideas for “reducing output size”, without knowing who you will be sharing the data with and for what purpose, and when/for how long, I can’t provide much advice. However, perhaps start with developing the “Storage & compute provision & management” field of the DMP. That would be where you set out how you will manage your data outputs including which variables, when they are shared, for how long and in which project. That might help you identify when you will run out of storage and then you can work out options to resolve.
I hope that helps but let me know if there’s something I’ve missed/misunderstood.
clairecarouge
(Claire Carouge, ACCESS-NRI Land Modelling Team Lead)
137
Hi,
Romain sent this message:
this is the link to the observation datasets repository for the CMIP Rapid Evaluation Framework. It would be interesting to know if it covers your needs from evaluating the new ESM outputs. Note that we are currently focusing on Fast track.
clairecarouge
(Claire Carouge, ACCESS-NRI Land Modelling Team Lead)
138
As always, please edit if you see any error or omission.
1 Like
MartinDix
(Martin Dix ACCESS-NRI Associate Director for Model Development)
139
The piControl configuration is currently using a solar constant of 1365.65 W/m^2 which was the CMIP5 value. For the CMIP6 historical experiment we applied an offset between this and the CMIP6 PI value of 1360.75.
The CMIP7 PI value is 1361.62. Using this would be a change in radiative forcing of approx -0.7 W/m^2, approx 1/5 of doubled CO2, so would be expected to give a cooling of 0.5 - 0.8 K.
This will be much larger than any other change from the CMIP7 PI forcing.
Is this likely to be too large a change to be acceptable?
I’ll do a short PI control run to confirm the magnitude of the change.
Is there a consensus on which run would be the best comparison? Still Pearse’s orginal run?
MartinDix
(Martin Dix ACCESS-NRI Associate Director for Model Development)
140
Global mean surface air temperature from all the CMIP6 historical runs suggests some cooling in ESM1.6 from a lower solar constant would be ok.
icon on this topic or to leave the group as necessary.
