I would like to create a CO2 emissions ancillary file for ACCESS-ESM1.5. We are currently using prescribed emissions for esm historical simulations (e.g. /g/data/p66/txz599/data/ancil/CMIP6/CO2_fluxes_ESM_1750_2014.anc) which vary in time. I would like to set up an experiment that uses constant CO2 emissions which are globally homogeneous. Is there a tool available that allows me to convert a netcdf file into the required CO2 emissions ancillary file? I have used xancil in the past, but this is challenging unless there is a saved job available that provides all the options/ details required.
Hi Tilo,
I have found the following script quite useful for modifying UM binary and ancillary files for ESM1.5
/g/data/access/projects/access/apps/pythonlib/umfile_utils/um_replace_field.py
The script is set up like an executable, but for some reason, doesn’t have execute permissions, so I copied it to a local directory and did chmod+x.
It relies on other tools in that same directory, so I added the directory:
/g/data/access/projects/access/apps/pythonlib/umfile_utils
to my PYTHONPATH environment variable.
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
3
That is perfectly reasonable, but if you don’t want to do that prepending a call to a python interpreter gets over that limitation:
Thanks David. I have used this script before to replace fields in a restart file but not for ancillary files. I don’t think it will allow me to change options in the ancillary file, e.g. the existing file uses monthly fields that change in time over the historical period, whereas I need a constant field that is then used repeatedly, independent of the actual simulation year.
If I understood correctly, what you are looking for is a tool that enables you not only to set the data of the new ancillary file based on a NetCDF file, but also to change some options in your file.
In your specific case, set the value of the 10th field in the FIXED_LENGTH_HEADER (For ancillary files, indicator for times at which data is provided) from 1 (Time series) to 0 (Single time) (Reference in the UM F03 doc).
Thanks Davide. I will try to copy the new field into the existing ancil file using the um_replace_field python script, but how do I change the “FIXED_LENGTH_HEADER” from 1 to 0?
@dkhutch I tried to replace the original field in the CO2 emissions ancillary file with the um_replace_field / um_copy_field python script, but it requires a stash code. This seems to be only applicable when replacing variables in a restart file. I don’t think ancillary files necessarily have variables represented by a stash code because these are mainly inputs and stash codes refer to model outputs.
@atteggiani can you please point me to the tool/script that I can use to manipulate the CO2 emissions ancillary file? Thank you.
That is not true.
Every variable (also input ancillary files) should have a STASH code linked to them.
For the file you referenced as example (/g/data/p66/txz599/data/ancil/CMIP6/CO2_fluxes_ESM_1750_2014.anc) the stash code is 251.
You can read it for example by using mule’s command line tool mule-pumf --headers-only (the --headers-only flags prevent it from dumping all the data values as well. Also, I would suggest to save the dump to a file because the ancillary file has a lot of fields.)
I am currently building a tool to manipulate input/output files for ACCESS models but it is currently not at the state to be able to do what you want.
However, I can write a small script to do what you are looking for. I will attach it here within an hour or so.
Thanks @atteggiani. I always assumed that the STASH codes only apply to outputs, so good to know that this applies to ancillaries as well. I checked the STASH register recently and should have noticed. Apologies to @dkhutch for this wrong statement and thanks again Davide for this really useful explanation and example.
@tiloz In case you still need it, I have written a small python script that changes the FIXED_LENGTH_HEADER time_type from 1 to 0 and sets the new data of the ancillary file to the one taken from a netCDF (in the script I used the data from the first timestep of the ancil file).
The script is the following: /scratch/public/dm5220/tilo/change_co2.py
You can easily change the paths if you need to, and in case it does not work or you have more questions on it, please let me know.
Thanks @atteggiani. Really appreciate your help with this. I will try your script and let you know next week how it went. This will enable us to do some really important trial runs with ACCESS-ESM1.5 that might become part of CMIP7.
Hi @atteggiani. The script works well, but I can’t get the model to run with this new CO2 ancillary file that contains only a single 2D field. Not sure if the model expects monthly values so need to provide at least 12 months, even if they are all the same? I tried to copy a netcdf file with 12 months of CO2 data using your script but that did not work either.
There is also an entry in CNTLATM that specifies PERIOD and INTERVAL for this ancillary file (I think):
&UPANCA ANC_REF_NO=78, PERIOD=4, INTERVAL=1 /
I assume PERIOD and INTERVAL need to match the data provided? Is there a documentation for this so that I can change the values accordingly?
I only have 1 year experience in UM so apologies if I cannot help a lot.
I can only think of a stupid way to do it: use mule in python to replace the field to whatever you want for every nt. If you can point me to where you have your CO2 netcdf file, I can try write a script to do it.
But what you are exploring now is much clever way: using a single 2D file and changing the suite settings to read it correctly. I would be very interested to know how to do it if you work it out. At the same time I can check my suite to see if there is an ancillary field like this.
@tiloz I suppose the quick and dirty solution would be to leave it as a timeseries, and just update all of the timesteps to the same co2 emissions value that you are aiming for.
Thanks for your suggestions. I guess I could replace the existing time series with constant values, but I think this would then also restrict me to running this over the historical period since the dates are being read as well from the ancillary file to match the simulation year. This is something I don’t want. I think I need to figure out how to provide 12 months and use them repeatedly independent of the simulation year.
Hi @atteggiani. Do you know how to replace a field in the ancillary file with a 3D or 4D variable? Your script at the moment will only accept a 2D field (ancilFile.fields[0].set_data_provider(mule.ArrayDataProvider(data))). I am not familiar with mule, but I assume there is an option to replace the existing 4D field in the ancillary file with another 4D field. I would like to test a file with 12 months of data to see if this runs. Thank you.
My understanding from playing around with UM binary files is that 3D or 4D variables are read in as if they are a long stack of 2D lat-lon arrays. For example, when reading in the u-component of velocity from a restart file (38 x 145 x 192), you essentially read in a lat-lon array (145 x 192) for each layer, and do it 38 times, with the stash code being the same each time. There is a new set of headers in between each layer…
Davide might have more insight into how to ensure all the layers are “glued” together.