Updating land-use change in ACCESS restart files for new tiles

tammasloughran · 20 January 2023 10:45

I’m having a problem with running simulations with land use change in payu.

Payu configurations with land use change call a pre.sh script which calls a python script

coecms/esm-historical/blob/master/scripts/update_cable_vegfrac.py

# Update CABLE vegfrac field and set new tiles to the grid box mean
# Arguments are the old and new dump file and the new vegetation fraction ancillary

import numpy as np, sys, umfile, argparse
from um_fileheaders import *
import netCDF4

parser = argparse.ArgumentParser(description="Update vegetation fractions in dump file")
parser.add_argument('-i', dest='ifile', help='Input UM dump')
parser.add_argument('-o', dest='ofile', help='Output UM dump')
parser.add_argument('-f', dest='fracfile', help='New vegetation fraction (ancillary or netCDF)')
parser.add_argument('-v', '--verbose', dest='verbose', action='store_true',
                    default=False, help='verbose output')
args = parser.parse_args()

ntiles = 17

# Get old vegetation fraction from dump file
f = umfile.UMFile(args.ifile)

This file has been truncated. show original

This script:

Re-scales the new cover fraction so that they add up to 1.
Does nothing more if the old and new cover fractions are the same.
Inserts the new land cover fractions into the restart file.
Initialise variables for any new tiles that are created by the changed land-use with grid averages.
Set variable values to 0 for tiles that transition to 0 cover fraction.
Re-scales most of the 8xx code variables proportionately for the new cover fractions.

2 is basically always untrue due to poor handling of missing values in the old and new fractions.
4 should only be necessary for certain variables such as soil and snow temperature.
5 might also be unnecessary, since CABLE won’t use them anyway.
6 Seems problematic to me, since CABLE already does this in subroutines like newplant() and it’s not something that should be done twice.

Also, it does not explicitly insert the previous year’s land use change into the correct code, but instead relies on 6. to do something similar.

This is quite a bit more than the old ksh scripts for running ACCESS, which as far as I can tell, simply insert the new fractions into the restart file (code 216), and move the old ones to the previous year (code 835). This is ok, since land-use change that creates new tiles never occurs in existing simulations.

I’m doing some simulations with very large land use change, where all crops are turned into tree PFTs. This means, new tree PFT tiles are created and need to be initialised. When looking at the total wood carbon pool of the 4 forest PFTs, there are these strange large drops at the start of each year. This happens only for grid cells where land-use change occurs. I would have expected the wood carbon pool to start at the same point as the simulation without land use change.

The orange line is the large drops in cwood.

The blue line I simply switched off land use change after the first year.

The green line is just a regular pre-industrial simulation.

The red line is with land use change turned on but it runs the update script to insert the regular 1850 land cover maps each year. I was surprised that it is slightly less than the regular pre-industrial simulation. Maybe this has always happened in other simulations but no one noticed because the land use change is usually so small?

Screenshot from 2023-01-18 17-48-59

I managed to mitigate the problem partially by fixing some of the obvious issues above with 2. and 6. But I’m not exactly sure what the cause is otherwise.

Below is the script i used to create the no-crops land use map:

And the land use map on gadi /g/data/p66/tfl561/sensitivity_lu_map/1850_no_humans_CABLE_fraction.nc

MartinDix · 25 January 2023 05:33

This script was originally intended to adjust the CABLE fields at the start of a new run and is not appropriate for updating vegetation within a run.

tammasloughran · 25 January 2023 05:42

Thanks, Martin.
Is this the right one then?

github.com

coecms/esm-historical/blob/master/scripts/update_landuse.py

#!/g/data/hh5/public/apps/nci_scripts/python-analysis3 
# Copyright 2020 Scott Wales
# author: Scott Wales <scott.wales@unimelb.edu.au>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import mule
import xarray
import os
import shutil

This file has been truncated. show original

MartinDix · 25 January 2023 06:02

This updates current and previous years land use from cableCMIP6_LC_1850-2015.nc which is presumably a concatenation of the individual years from /g/data/p66/txz599/data/luc_hist_maps/.

Script form runs at end of the year and uses

ncfile_prev="cableCMIP6_LC_${endyear}.nc"
ncfile="cableCMIP6_LC_${nextyear}.nc"

The new script gets the year number from the restart file which would be same as nextyear. When does it run?
and sets field 216 from ncfile and 835 from ncfile_prev.

tammasloughran · 29 January 2023 23:33

I’m stupid. It helps to use the correct cover fractions when plotting global totals. The pre-industrial line was using the cover fractions from my no crop run, so they were too high also. Correcting that brings them very close to each other.
Screenshot from 2023-01-30 10-26-03

Nevertheless, it remains that the existing update_cable_vegfrac.py is not quite suitable for simulations with large land-use change. My version of it that fixes the above issues is here, in case anyone is interested in the future:

github.com

tammasloughran/esm-esm-piNoCrops/blob/master/scripts/update_cable_vegfrac.py

#!/usr/bin/env python
# Update CABLE vegfrac field and set new tiles to the grid box mean.
# Arguments are the old and new dump file and the new vegetation fraction ancillary.
# Note: the variable name vegfrac does not refer to fractions of only vegetated types, but rather
# tile fractions of any type.
# Author: Martin Dix, Tammas Loughran
import argparse
import sys

import netCDF4
import numpy as np

import umfile
from um_fileheaders import *

NTILES = 17
VEGFRAC_CODE = 216
PREV_VEGFRAC_CODE = 835
MASK_CODE = 30

This file has been truncated. show original

Thanks for the help Martin.

clairecarouge · 30 January 2023 04:53

Is this updated script valid for all land-use change scenarios or is it only limited to large changes? I am wondering if it needs to be updated for other experiments with ACCESS-ESM?

Also, it seems right now we have a copy of update_cable_vegfrac.py in each ACCESS-ESM experiment with payu. Should we think of a way to have only 1 copy of the script and all experiments using the same script? Are there cases we want the script to do different things?

tammasloughran · 30 January 2023 06:08

Is this updated script valid for all land-use change scenarios or is it only limited to large changes?

Yes, it can be used for any land-use change, while the existing one should only be used as an initialization script at the start of a run. The new is more generalised because it allows for land use change where new tiles are created and need to be initialized with realistic soil and snow temperatures.

I am wondering if it needs to be updated for other experiments with ACCESS-ESM?

I would recommend so, because of problem 6 above causing the step changes each year. I would first feel more reassured about this if someone else read through this script thoroughly and reviewed to make sure that it’s correct. e.g. besides temperature, is it okay to initialize everything else to 0?
Also, because it doesn’t create a new restart file if the cover fractions have not changed, the pre.sh script needs to be slightly modified to restore the backed-up restart file when this happens.

Also, it seems right now we have a copy of update_cable_vegfrac.py in each ACCESS-ESM experiment with payu. Should we think of a way to have only 1 copy of the script and all experiments using the same script?

I agree. Updating this would be easier if all of the scipts in the experiments are the same and are kept in the same place. Should the scripts directory (or a subset of it) be kept in a separate repository and include it in the experiment configurations as a git submodule?

Are there cases we want the script to do different things?

I can’t think of anything at the moment.

tammasloughran · 30 January 2023 06:16

I agree. Updating this would be easier if all of the scipts in the experiments are the same and are kept in the same place. Should the scripts directory (or a subset of it) be kept in a separate repository and include it in the experiment configurations as a git submodule?

I guess the downside of this is that to update and clone a configuration you need to do the additional step of initializing and updating the submodule

git submodule update --init

or

git clone --recursive ...

Aidan · 30 January 2023 22:53

A separate repository with some testing would be ideal. If there are other scripts of a similar nature then some thought might need to go into how to organise this.

There are other options to using submodule to include the code:

We could create a conda package and install this in the same environment as payu (currently the CLEX CMS environment).

Use git subtree which inlines the code but allows for relatively easy updates when required.

Have the code in a repo, with testing and packaged for use with conda would be my preferred option.

@holger and @Paola-CMS need to be part of this conversation, as this is the ACCESS-ESM payu configs that CLEX CMS created and support?

Paola-CMS · 5 February 2023 23:21

Just back from leave, so after reading this quickly:

I recall @holger mentioning that some of these configurations might have not been used in a long time, so someone checking what the differences are, if they’re still needed and working would be a great first step
we should investigate if it’s feasible having one repository as @clairecarouge mentioned and re-organise the code accordingly

We were looking for a ACCESS-ESM project @holger could contribute to in collaboration with the ACCESS-NRI, and this might be a good fit.

clairecarouge · 6 February 2023 02:48

I agree with @Paola-CMS we should investigate how to collaborate to clean up the management of ACCESS-ESM configurations. It would be great to have @holger involved.

Topic		Replies	Views
Modifying a UM restart file for ACCESS-ESM1.5 Technical help , land , solved	38	711	13 February 2024
Updating ESM1.5 configurations for payu version `1.1.5` Earth System Model tutorial , payu , how-to , access-esm15	2	68	7 November 2024
Payu issues for ACCESS-ESM1.5 on vk83 Earth System Model help , payu , technical	6	43	22 October 2024
Changing Antarctica to ice free in ACCESS-ESM1.5 Technical help , land , earth-system , solved , access-esm	10	178	17 December 2024
Model exited with error code 139 Earth System Model help , inscope , access-esm	10	61	17 December 2024

Updating land-use change in ACCESS restart files for new tiles

Related topics