I am looking to run ACCESS-OM2 at 1deg resolution using surface forcings from ESM1.5 future projection runs (SSP585). I have been able to obtain 3 hourly atmospheric data from @tiloz (thank you!) and want to impose that as a surface forcing like how JRA55-do is used as an input to the YATM of the ACCESS-OM2 config. (There should also be a spinup, and data for the historical period exists in 1day resolution to my knowledge.)
The resolutions between JRA55-do and ESM1.5 are different, but I am not sure where the documentation is to sort this out so that I can force the 1 deg ocean model correctly (I got stuck at here). As I want to use ESM data as the forcing, my understanding is that I should be using the YATM in ACCESS-OM2 rather than straight away running a flux-forced simulation with MOM5 like Dhruv et al. here.
Any pointers/comments/help would be greatly appreciated!
Cheers,
Ellie
Hello Ellie,
Thanks for your post, this ability would be great. Just letting you know we’ve noted this post. While it’s been discussed, we don’t have it properly figured out and documented, so I will mark this outofscope for now. Though hoping the community can help out here. This might also be somewhat related to this: Easy forcing perturbation experiments in ACCESS-OM2 + Tutorial
I am looking to run ACCESS-OM2 at 1deg resolution using surface forcings from ESM1.5 future projection runs (SSP585)
Interesting to hear your plans. Following @Flick.Chun post, this sort of thing falls under a science application so is out of scope for ACCESS-NRI support. For additional background, I suggest having a look at Ryan’s tutorial and we wondered if @dkhutch@wghuneke might have done related things? Having said this, we’ve had a little a chat (@AndyHoggANU@anton@Aidan@spencerwong) around a suggested workflow:
Suggest using bulk formula not a flux forcing for ACCESS-OM2 forcing (avoids tricky sea ice issues for example).
Obtain or re-run ESM1.5 with the variables and temporal resolution required to match ACCESS-OM2’s bulk formula inputs. This post can likely help. It’s very likely that outputs from existing runs won’t be appropriate for this but @spencerwong thought (without checking) it was possible. Take care to match the height and other constraints (MOM5 unlike NEMO can’t adjust on the fly). To reduce the forcing size, you may also want to consider reducing the frequency of temporal output, at least for some variables but you’ll have to check the model can support interpolation in that regard (NEMO can).
A remaining issue is the different grids/resolution. ACCESS OM2 can re-grid on the fly but you need an appropriate weights file (calculated offline once), you may be able to find one in the CM2 configuration but this presumes that they use the same grid so one needs to check that first or re-create the file (see Changing the OASIS remapping fileshere, i.e., To remake the mapping files using the new land-sea masks use the script make_remap_weights.py in the access-om2/tools/ directory of the main access-om2 repository. ).
Looking forward:
we’d love to see this kind of workflow documented – it’s very likely other COSIMA users would like to do this kind of thing. For example, feel free to share your progress on the ACCESS OM2 configuration docs or at the training program.
We have a planned training session on doing perturbations with ACCESS-OM2 led by @aekisscoming up too.
Hi @ongqingyee,
I haven’t been down this road. But it seems like you’ve got two major tasks here:
Convert resolution from the existing JRA fields to the UM7.3 grid diagnostics.
Mimic the forcing fields that currently go into YATM for ACCESS-OM2 using ACCESS-ESM equivalents.
I have examples of re-generating the remapping weights for ACCESS-ESM1.5 paleo runs, for example in this repository see the steps in make_coupler_grids, but you also have similar examples from Chris’ post above. However, your case will look a bit different because the exact remapping weights will need to be tailored to the ACCESS-OM2 configuration.
If you’d like to get in touch to discuss, would be happy to chat. I have been meaning to set up this kind of scenario (forcing an OM2-like simulation using coupled model inputs), but never took the plunge on it yet.
First, thank you for the many links/input/recommendations. There is now a viable plan to try and get this to work.
Below is a list of the JRA55-do variables used in ACCESS-OM2 (properties are taken from here), and what I think their corresponding variables in the ESM1.5 UM are. All the UM output below is saved 3-hourly, and the time and time_0 variables are the staggered 3h time coordinate.
JRA55-do variable
UM atm variable
UM Dimension
UM Description
tas 10m air temperature (θA), units: K
fld_s03i236
(time, lat, lon)
TEMPERATURE AT 1.5M units : K
huss 10m specific humidity (qA), units: Kg kg−1
fld_s03i237
(time, lat, lon)
SPECIFIC HUMIDITY AT 1.5M
uas 10m eastward wind, units: m −s 1
fld_s03i209
(time, lat, lon_u)
10 METRE WIND U-COMP
vas 10m northward wind, units: m −s 1
fld_s03i210
( time, lat_v, lon)
10 METRE WIND V-COMP
psl Sea level pressure (SLP), units: Pa
fld_s00i409
(time, lat, lon)
PRESSURE AFTER TIMESTEP units :Pa
rsds Downward shortwave (QDSW), units: Wm−2
fld_s01i201
(time_0, lat, lon)
NET DOWN SURFACE SW FLUX: SW TS ONLY units :W m-2
rlds Downward longwave (QDLW) , units: Wm−2
fld_s02i201
(time_0, lat, lon)
NET DOWN SURFACE LW RAD FLUX units :W m-2
prra Rainfall flux, units: kg m ^{-2} s ^{-1},
fld_s05i214
(time_0, lat, lon)
RAINFALL RATE: LS+CONV KG/M2/S
prsn Snowfall flux , units: kg m ^{-2} s ^{-1}
fld_s05i215
(time_0, lat, lon)
TOTAL SNOWFALL RATE: LS+CONV KG/M2/S
friver Total river runoff , units: kg m ^{-2} s ^{-1},
missing friver from ESM?
licalvf Solid water runoff from Antarctica represented as calving flux from Depoorter et al. (2013), units: kg m ^{-2} s ^{-1}
missing from ESM?
Here I have outlined the approximate next steps and the links to the scripts/resources I will likely use. Warning, all this is hypothetical and the list will grow, but I will keep this thread updated.
what to do for missing friver/licalvf values – just use JRA55-do RYF input?
Initial conditions: I would like to use the historical ESM1.5 ocean/ice restart files from /g/data/vk83/configurations/inputs/access-esm1p5/modern/historical/restart/ and feed that into ACCESS-OM2. The file names match up, but in OM2 each nc file is split. (‘ocean_neutral.res.nc.0000’, ‘ocean_neutral.res.nc.0001’ etc in OM2 for example, vs. ‘ocean_neutral.res.nc’) Would this naming situation would affect me using the ESM ocean/ice restarts? unaffected, but can collate as suggested two replies down.
Need to interpolate the ocean restart files, as the ESM1.5 and OM2 have different ocean vertical resolution
One final thing: For science reasons, it has been thrown out there for my project that using ACCESS-OM3 would be useful. The pros are isopycnal coordinates and that changing the forcing could be easier with NUOPC (my vague understanding from what Chris said above?). The cons are that OM3 is still in development, and that there is some documentation now (here in this thread!) on how this could be done in OM2. I will be at ANU Thu-Fri this week, so will try to catch a few people in person but any opinions on this here are very welcome. @ShayneM
Cheers everyone,
Ellie
P.S. I am also aware of a running list of potential problems that can occur when changing the forcing for ACCESS-OM2 as outlined here. I don’t think they currently apply but will keep in mind.
This is not a problem - ACCESS-OM2 will handle collated restarts (without .0000 etc) just fine.
It’s best to use an identical topography and vertical grid so you don’t have any uninitialised wet points (or if not, to extrapolate fields horizontally into all the land points - there are tools for that).
Hi @ongqingyee,
Sounds great - I should be around on Thursday and Friday at ANU too.
In regards to uncollated restarts, I would just collate them yourself for simplicity of looking at and interrogating the files. You can borrow the mppnccombine executable specified from one of your config.yaml settings. If you simply run that executable on its own it gives you a user prompt on how to run it.
Please note: ACCESS-ESM1.5 and ACCESS-OM2-1deg do not use the same vertical grid. So, if transplanting an ocean restart file from one to the other, be mindful of the need to re-interpolate vertical structure, especially in relation to missing grid cells.
You could force one of them to use the vertical grid of the other… perhaps that’s breaking things too much though.
Hi @ongqingyee, just answering your email query here in case it is or interest to others:
We generate initial conditions by regridding WOA data using this script
which handles vertical regridding and horizontal extrapolation into all land points. I expect with a bit of effort it could be adapted to create ACCESS-OM2 initial conditions from ACCESS-ESM1.5 restarts.
I’ve made an incremental start on this ESM1.5 forced OM2 run and thought I’d try to share what I have been doing. Because of how many things need to change to get to what I want, I have opted to only add the ESM1.5 forcings in the table above* and their regridding files (regridding made by @dkhutch) and check if the model even starts to run. For now, I am keeping the NRI release OM2 ocean initial conditions and resolution (75 vertical levels unlike the ocean restarts from ESM), but the main missing factor is the 10m air temp and humidity forcings – now they are at 1.5m because the 3 hourly ESM1.5 run output at saved by @tiloz here (/g/data/p73/archive/CMIP6/ACCESS-ESM1-5/SSP-585-10-re1/history/atm/netCDF) save the 1.5m height temp and humidity.
Initially, I figured the 1.5m air temp and humidity might cause problems and I would do a comparison between 10m and 2m air temp similar to @aekisshere but the daily/monthly air temp and humidity data I can find for these ESM simulations is on pressure levels. Also based on the comparison Andrew did I figured there might not be a big difference between 1.5m and 10m. **
The next possible issue I considered was that my initial ocean conditions are not suitable for the ESM forcing I want to impose, as I was using the release defaults. However, going through the release configs the RYF and IAF seem to use the same initial conditions.
If the community has any experience with this that would be amazing. Will keep trying otherwise.
* I ended up testing one year only and thus took friver and licalvf from the JRA55-do RYF forcings. When modifying the configs I followed a lot of the ERA5 config changes here.
** As an aside maybe for people who use UM, I realised that for the surface temp. variables height = 10 but height_0 = 1.5. My understanding is that one is terrain following and one is vertical height, as implied by this but could not find it explicitly stated on the forum or the internet. If the vertical height is indeed 10m then is all good?
This may be hard to track down! It could be many things.
How long does it run for before failing ? It should print to ice diag file the location(s) where the crash occurs. (Have a look for the ‘Global i and j:’,). Note that fortran indexes start at 1, whilst python indexes start at 0.
My first guess would be its related to one of the forcing fields. You can try setting chk_a2i_fields = .true. in the ice/input.nml namelist. Then re-run the model, it should produce a fields_a2i_in_ice.nc file with the forcing fields saved. (No promises this works! I haven’t tested it).
Alternatively - you could try turning on timestep diagnostics in cice to help find this (assuming it runs more than one timestep). In cice_in.nml:
set:
histfreq = 'd', 'm', '1', 'x', 'x'
then set the diagnostics that might be useful to with ‘1’
I looked at the ice_diag.d file and it fails after one month. This is the output. I’ve also check my radiation fields and they are way off which is why everything blew up. Will fix the forcings and try again. Thanks @anton!
(ice_pio_wopen) create file ./OUTPUT/iceh.2015-01.nc
Finished writing ./OUTPUT/iceh.2015-01.nc
Thermo energy conservation error
istep1, my_task, i, j: 499 0 2 32
Flux error (W/m^2) = 1.502400711381605E+028
Energy error (J) = 8.112963841460668E+031
Initial energy = -112450896.015573
Final energy = -4.033498414583433E+047
efinal - einit = -4.033498414583433E+047
fsurfn,flatn,fswint,fhocn, fsnow*Lfresh:
8.548785907776473E+043 8.545756215085293E+043 0.732862986202482
7.472471201179017E+043 2.74450532560876
Input energy = -4.033498414583432E+047
fbot(i,j),fcondbot(ij):
-0.186976031853725 -10.6271738305758
Intermediate energy = -111672746.248751
efinal - einter = -4.033498414583433E+047
einter - einit = 778149.766822323
Conduction Error = 9.735082020051777E-002
Melt/Growth Error = 8.112963841460668E+031
Advection Error = 0.000000000000000E+000
dt*(fsurfn, flatn, fswint, fhocn, fsnow*Lfresh, fadvocn):
4.616344390199295E+047 4.614708356146058E+047 3957.46012549340
4.035134448636670E+047 14820.3287582873 0.000000000000000E+000
istep1, my_task, iblk = 499 0 1
category n = 1
Global block: 1
Global i and j: 1 31
Lat, Lon: -67.7330179507242 -279.499997834891
aice: 0.910377865254026
n: 1 aicen: 0.657484995909043
The surface downward latent heat (flatn) there is crazy. flatnlooks like it would depend on downward longwave and specific humidity.
Its failing in the first timestep in February (istep1 = 499, so 499 timesteps of 1.5 hours = 31.1875 days) , so i guess there is some issue with the forcing in the second month.
Cheers, I did correct radiation but not specific humidity.
The specific humidity data saved from ESM I was using has only nans from the second month onwards for the rest of the year (still to check if that is the case for the rest of the ESM data). This is after the 2015-02-01 time step, the 3hourly data from then on is empty. I will see if the missing data can be replaced/found. Thanks again!
Hi Ellie, yes I was wondering if missing data is involved somewhere, as the energy values are order e47 and e28 etc, which is off the scale, I would check where they are coming from, it looks like the latent heat you term as Anton said. I would compare the values against the January ones that ran successfully.
Thanks for the update @ongqingyee and for the tips @anton. It’s good to hear that it ran successfully for one month, and you have a good idea of where the error might be coming from.
Silly question: Is there anyway to run it just for the month and get the output without any errors? I tried to change the dates in accessom2.nml but it came up with this error. I had to start from 2015-01-02 because the 3 hourly data starts at 2015-01-01T01:30:00 and not at 2015-01-01T00:00:00.
ERROR in accessom2_progress_date: forcing and experiment dates are out of sync.
forcing date: 2015-01-02T00:00:00.000
experiment date: 2015-02-01T00:00:00.000
My restart period is also shortened.
! Runtime for a single segment/job/submit, format is years, months, seconds,
! two of which must be zero.
restart_period = 0, 1, 0
On this error forcing and experiment dates are out of sync.
I’ve been having a similar error in OM2 01 (we’re working on some instructions at the moment, a draft preview is here) and I’ve found the following two resources useful:
This first link above made me think the solution always works but I note that it’s been a whack of a mole problem, see this chat. In my case, I’ve found that turning use_restart_time = .true. back to true might not be a good idea.
I haven’t tried @anton suggestion. What build version / config of OM2 are you using?
@cbull Thank you for the links! I will keep in mind if the error comes up again though. Re OM2 config, I’ve modified a cloned version of the NRI 1deg RYF example, pretty sure it is this one.
So far I have only run OM2 with atmospheric forcing from 2015-2016 (first year of ESM1.5 SSP585 scenario run) and therefore was able to use the default RYF JRA55-do friver and licalvf forcing files. However, I guess ESM should have comparable variables that therefore change in time under the future climate scenario. For friver following the ERA5 config github issues I think I should be using the surface_runoff_flux in ESM1.5 (see highlight in screenshot, if there are no crazy biases, TBD) but am unsure about a comparable variable for licalvf – I’ve gone through the monthly, daily and 3hourly output and could not see anything suitable as most relate to snow.
I followed the instructions here and ended up with this error, which I traced back to here.
FATAL from PE 46: fms_io(restore_state_all): Checksum of input field RHO_DZT 81F2FC58DB6DF285 does not match value C9968F680DEDF285 stored in OCEAN_THICKNESS.RES.NC.
I am slightly confused by how the MOM5 ESM1.5 component code is showing up when running ACCESS-OM2 with ESM ocean restarts. Even after I commented out the lines in config.yaml that reference the previous initial conditions, this error still shows. Perhaps I am misunderstanding how the restarts work?
@spencerwong I believe you are the person to ask about most of the above? Thank you!
