You’ll still need to set the field headers appropriately for the level and time. If you’re manually creating an ancillary with Mule it’s going to be easiest to start from an existing file as a template and put your data into that rather than create a file completely from scratch.
Sorry for my late reply, I was on leave yesterday.
As @Scott already pointed out, the UM ancillary file type (fieldsfile) is a stacked sequence of 2D data, preceded by headers (global headers first and then field-related headers).
The data follows the structure outlined by Scott and, as he suggested, it is recommended (and easier) to replace the data of an existing ancillary file having the same structure of the needed one.
@tiloz I think if you could give more details about the specific co2 structure you’re looking for (if you already know it), or otherwise give details about the experiment you are trying to simulate, we should be able to come up with the best co2 ancillary structure for it and how to create it.
Hi @atteggiani. What I need is a way of overwriting the existing field in the CO2 emissions ancillary file (CO2_fluxes_ESM_1750_2014.anc) which currently has monthly data from 1750 to 2014 with a field that has only 1 year of monthly data but then can be used repeatedly (independent of the simulation year). The script you provided does all of this in theory, but it wont let me copy a 3D field.
as @dkhutch suggested to replace this field, but this gives me an index error. If this would work then I could use your script to change the header only (without replacing the field).
I haven’t generated the 12 month emissions file yet. For testing I am simply using the first 12 months from the original file to overwrite the 1750 to 2014 time series:
I have changed my script and it now produces a CO2 ancil file with 12 monthly values (in the example I used the first 12 timesteps of the original CO2 file) to be used as a periodic time series for any simulation.
You can find the updated script in /scratch/public/dm5220/tilo/change_co2.py.
I also created a public gist: Example of how to change the CO2 ancillary (input) data for an ACCESS-ESM1.5 or ACCESS-CM2 simulation
Note you might also have to set the proper ancillary update options in your suite/experiment to make sure the CO2 is updated at the desired time frequency.
If you need help with that, please let me know.
That’s great @atteggiani! Thank you for providing this and for making it accessible to the community. I will try it later and let you know how it goes.
Hi @atteggiani. The model still crashes with the new CO2 emissions ancillary file, but it is running a couple of time steps. I think it crashes because CO2 emissions are shown as NaN in the pe0 output, which is odd because we have not changed the the actual emissions yet, so I assume it has something to do with the information in the header. I checked some other ancillary files that we use with only 12 months data repeatedly and the year in the reference data is set to 0000 in all of those cases. I am wondering if this is important? In our new CO2 ancillary file the reference year is still 1750.
I also now remember that @chloemackallah has created some ozone ancillary files for us a couple of years ago for a similar purpose: using 12 month data for only one year from the original time series that we then used repeatedly (and reference year was set to 0000). Maybe Chloe remembers what needs to be changed in the header to get this to work?
Thank you for sharing those scripts.
It is true, checking the scripts in the folder and also some other ancillary files I have, I figured I forgot to change a few time-related header entries in the new CO2 ancillary file.
I changed the script and now it should output the correct one.
Please check if it works now, and in case let me know.
As usual, you can find the updated version on Gadi in /scratch/public/dm5220/tilo or in this gist.
P.S.:
The previous days I was trying to use the new CO2 ancil file myself in an ESM historical simulation to understand what was the problem, but I cannot seem to run my ACCESS-ESM historical configuration (not even the piControl cloned directly from the repo).
Sorry about the wait.
Hi @atteggiani. Thanks for working on this and for providing a new version of your script. I will try this as soon as I can and will let you know how it goes. Sorry for not passing on the ozone scripts earlier. I completely forgot that I had them and only found them yesterday by accident while I was looking for something else! I guess that’s why it is so important to have everything in one central place (and documented) so that it is accessible to the community and we don’t have to do things again.
Hi @atteggiani. I tried the new script and unfortunately the model still crashes with the new ancillary file. The error is the same and the CO2 emissions field produces an NaN.
Sorry, I just realised the original co2 ancillary data had NaN values in it (which in the UM ancillary files are substituted with the BMDI value -1073741824.0 (-230).
I didn’t notice it and in my script I did not perform the NaN substitution before writing the data to the new ancillary file. So, the data written in the new ancil file contains NaN values which are not supported by UM.
I changed the script and added the step where I substitute all NaNs with the BDMI value.
Now everything should work!
Please find the updated version on Gadi (/scratch/public/dm5220/tilo/change_co2.py) or on gist.
Thanks @atteggiani! I tried the new ancillary file and the model has been running for a couple of years. Now that the framework is working, I can start to generate specific CO2 emissions fields and use your script to convert them into ancillary files. Thanks again for your help with this.
Hi @atteggiani. Your script has been working fine and I have almost completed the first run with constant emissions. I was wondering how easy it would be to adjust this script to generate an emissions driven ancillary file with prescribed emissions that change every year? Hopefully this should be relatively easy and would require mainly changes to the header. Similar to the existing script we would supply a netcdf file with a monthly (varying) time series over multiple years which we then need to convert into an ancillary file. I didn’t want to open a new topic since this is related.
Yes, achieving what you want is relatively simple starting from the current script.
You would just have to modify the values inside the ancillary files (based on your NetCDF file), and change a few time-related headers (both global ones and field-related ones).
If you tell me where is the NetCDF you want to use (or at least provide how many total months it encompasses), as well as the starting/ending times for your simulation, I can modify the current script (I will provide you with another script and keep the existing one as a reference).
Hi @atteggiani. I am trying to model declining emissions over a 100 year period. Start/end years would be something like year 201 to year 300. A modified version of the existing script as a template for this would be great. I am sure this is useful for others too.
Hi @atteggiani. There is a small typo in your script: “ancilFile.fixed_length_header.calendar = BMDI” The model will crash with “invalid calendar type”. I changed this to “ancilFile.fixed_length_header.calendar = 1” and now it runs fine.
