OM3 Wind perturbation crash

COSIMA Ocean
,
1

Hi all! I’ve been running some wind perturbation experiments with ACCESS-OM3, 25km IAF (and a matching control run, since I am using dye tracers). I’m getting an issue where my increased-winds run crashes at year 2005 and won’t continue, and my decreased-winds run crashes at 1988. The control run is running fine. I initially thought it was a badly generated wind forcing file that might have caused the issue, but since the two runs are crashing on different years, I’m not convinced. I also worry maybe the winds are causing issues with the sea-ice? But I’m struggling to see a clear issue in the .err file.

If anyone has had similar problems with altered wind forcings, help would be very welcome! Thank you!

2

Hi Katja

I would start by looking at the bottom / end of the access-om3.out file, and sometimes there is additional information in work/log folder.

Where are the config directories for the crashed experiments ?

4

For the decreased wind run, the bottom of the .out reads:

(abort_ice)ABORTED:
(abort_ice) called from ice_step_mod.F90
(abort_ice) line number 628
(abort_ice) error = (step_therm1)
(abort_ice)(step_therm1)

The increased wind run is looking a bit more confusing - I’ll keep digging on this one.

My experiments are located in /home/272/kc5856/access-om3/OM3_MC_25km_jra_iaf_cycle2_wind_dec/, etc (sorry not sure if that was what you meant). They are clones of the test4 om3 config (release-MC_25km_jra_iaf).

5

Crashes in step_therm1 are often due to low SSS, chances are that’s unrelated to the perturbation applied.

Options are:

  • increase the salinity restoring rate - this is a global change though
  • try the river runoff spreading from the dev- branch / the in-work test5.1
  • dig further - use a build with some extra debug statements to confirm the crash location
6

Okay, I’ll have a play around with those - thank you! I am still a little confused on the point that if it’s unrelated to the winds, is there a reason this isn’t happening in my control run as well?

7

Sorry unrelated is the wrong word - I mean it seems unlikely to be an issue with experiment design. In test4 - there were a couple of similar instances where there was a crash in step_therm1, and whilst the longer term fix was to implement spreading of river runoff, the short term fix was to increase the salinity restoring rate. In test4, the salinity restoring rate was increased twice through the run, due to crashes in step_therm1.

So it’s possible that the conditions for getting to this crash were marginal in your control run, and the perturbation creates enough random variability that the model crashes. Alternatively it’s possible that changing the winds has had a systematic impact (e.g. less surface stress → less mixing → freshwater water at a river mouth gets stuck there → low SSS). Typically these crashes occur at the mouth of rivers in the Arctic. Hopefully that helps ?

My feeling is that spreading of river runoff might make sense for this. The spreading only impact 6 high volume rivers :

If the model state near these rivers is important, we might need to think some more. But otherwise, it means the impact would be more localised (rather than changing the salinity restoring rate, which is a global change)