ACCESS-OM3 evaluation

COSIMA
,
1

The purpose of this topic is to discuss and plan the evaluation of ACCESS-OM3.

Below I’ve summarised the evaluation metrics and diagnostics that were included in the ACCESS-OM2 description paper (Kiss et al. 2020) and technical report (Kiss et al.). I’ve done this as a poll so that you can vote on your favourites. Links to code are included where I could find them.

This topic has post voting enabled. Please reply with evaluation metics/diagnostics that you would like to see included in ACCESS-OM3 evaluation. Please also up/down-vote metrics/diagnostics suggested by others.

ACCESS-OM2 evaluation

The following were used to evaluate ACCESS-OM2 as part of the ACCESS-OM2 description paper (Kiss et al. 2020) and the ACCESS-OM2 technical report.

In Kiss et al. 2020
  • Time series of global average annual mean:
    • ocean temperature
    • sea surface temperature - compared to ERSST v4
    • sea surface salinity
    • ocean kinetic energy
    See Fig 3 and and code here.
  • Time series of annual mean transport though Drake Passage. Compared to observational estimate from Donohue et al. (2016). See Fig 4 and code here.
  • Contours (latitude-longitude, global) of 1993-2012 mean dynamic sea level. Compared to CNES-CLS13. See Fig 5.
  • Contours (latitude-longitude, global) of 1993-2012 standard deviation of sea level anomaly. Compared to AVISO SSALTO/DUACS. See Fig 6.
  • Contours (potential density-latitude) of 1993-2017 mean zonally integrated overturning circulation. See Fig 7.
  • Time series of AMOC. Compared with observational estimate of McCarthy et al. (2015). See Fig 8.
  • Contours (depth-time) of horizontally averaged temperature anomaly relative to WOA13. See Fig 9.
  • Contours (latitude-longitude, global) of 1993-2017 mean sea surface temperature bias relative to WOA13. See Fig 10.
  • Contours (latitude-longitude, global) of 1993-2017 mean sea surface salinity bias relative to WOA13. See Fig 11.
  • Contours (depth-latitude) of zonally averaged temperature and salinity bias relative to WOA13. See Fig 12 and code here.
  • Lines of total meridional heat transport as a function of latitude. Compared to NCEP and ECMWF reanalyses and World Ocean Circulation Experiment. See Fig 13.
  • Meridional transects of 1993-2017 mean potential temperature and salinity near WOCE/GO-SHIP hydrographic lines:
    • SR3 (Fig 14)
    • P16 (Fig 20)
    • A16 (Fig 23)
    • IO8 (Fig 25)
    • IO9 (Fig 25)
    Compared to climatologies (1985-2013) from WOA13. See code here.
  • Meridional transects of 1993-2017 mean planetary geostrophic potential vorticity, potential density anomaly and max/min monthly mean mixed layer depth across a Subantarctic Mode Water formation region at 120W. See Fig 15.
  • Contours (latitude-longitude, regional) of 1993-2017 standard deviation of sea level anomaly and mean barotropic streamfunction in the:
    • Agulhas region (Fig 16)
    • East Australia Current region (Fig 17)
    • Kuroshio region (Fig 21)
    • Gulf Stream region (Fig 22)
    Compared to AVISO SSALTO/DUACS (sea level anomaly) and observational estimate of Colin de Verdiere and Olltrault (2016) (barotropic streamfunction).
  • Time series of annual mean transport through the Indonesian straits. Compared to estimates from INSTANT programme. See Fig 18.
  • Contours (depth-longitude @ equator, depth-latitude @ 220E) of temperature and zonal velocity. Compared to observational estimate from Johnson et al. (2002). See Fig 19.
  • Contours (latitude-longitude, SW Atlantic) of 1993-2017 mean surface current speed, standard deviation of sea level anomaly and mean barotropic streamfunction. Compared to drifter data (surface current speed), AVISO SSALTO/DUACS (sea level anomaly) and observational estimate of Colin de Verdiere and Olltrault (2016) (barotropic streamfunction). See Fig 24.
  • Contours (latitude-longitude, global) of annual 1993-2017 mean depth of the 20C isotherm. Compared to WOA13. See Fig 26.
  • Time series of running 12-month minimum, mean and maximum sea ice extent and volume for the Arctic and Antarctic. Compared to NSICD Sea Ice Index version 3. See Fig 27.
  • Lines of 1993-2017 mean annual cycle of sea ice extent for the Arctic and Antarctic. Compared to NSICD Sea Ice Index version 3. See Fig 27.
  • Contours (latitude, longitude, regional) of 1993-2017 sea ice thickness and concentration around:
    • Arctic for March and Sept (Fig 28)
    • Antarctica for Sept and Feb (Fig 29)
    Compared to NOAA G02202 V3.
0 voters
In the technical report (and not in Kiss et al. 2020)
  • Contours (latitude-longitude, global) of mean barotropic streamfunction. See Fig 13.
  • Contours (latitude-longitude, Antarctica) of Antarctic Circumpolar Current barotropic streamfunction. See Fig 14.
  • Contours (latitude-longitude, regional) of daily and climatological mean surface current speed in the:
    • East Australia Current region (Fig 17, 21)
    • Agulhas region (Fig 18, 22)
    • Kuroshio region (Fig 19, 23)
    • Gulf Stream region (Fig 20, 24)
  • Time series of annual mean transport through the Bering and Denmark straits. See Fig 25.
  • Time series of global average annual mean:
    • sea surface height
    • salinity
  • Time series of running 12-month minimum, mean and maximum sea ice area. Compared to NSICD Sea Ice Index version 3. See Fig 33.
  • Lines of 1993-2017 mean annual cycle of sea ice area for the Arctic and Antarctic. Compared to NSICD Sea Ice Index version 3. See Fig 36.
0 voters
7

To diagnose how good the simulation of Antarctic dense water formation is, it would be really useful to have plots of:

  • Transport across the 1000 m isobath around Antarctica, summed along the isobath and as a function of density.
  • Surface water mass transformation integrated over the Antarctic continental shelf.
4

Couple of points regarding the sea ice concentration:

  1. we should now be using v4 of the NSIDC CDR product, not v3 (NOAA/NSIDC Climate Data Record of Passive Microwave Sea Ice Concentration, Version 4 | National Snow and Ice Data Center)

  2. Can’t see the code for figs 27-29, but strictly speaking Walt Meier (NSIDC/NASA) recommends masking out model grids with a thickness less than 20mm/2 cm (the passive microwave algorithm struggles with ice this thin so it’s not a fair comparison with obs)

1 Like
5

For wave-ice penetration – I can (soon!) provide wave penetration into the MIZ for 2013 - present with AltiKa. Going back before 2013 is possible with other instruments but I can guarantee the 2013-present period.

For sea ice motion – Tian’s work showed that OSI-405c is basically as good as anything (large-scale) for sea ice drift (48 h) – but not for short timescale (any EVP model is probably poor at short timescale). https://osi-saf.eumetsat.int/products/osi-405-c

For thickness I guess the new ESA CCI is probably the best long-term one - Sara Kacimi might have better input here! The new CCI will be available soon from the CCI Data Portal (version 3). Version 2 is available but the ESA guys don’t have much confidence in it (compared to v3).

2 Likes
3

Just to note that we (@aekiss or I) will have the old code for other figures somewhere – but we weren’t able to keep them up to date with cookbook developments, so they might need some renovation to bring back up to date. Let me know if you want me to dig them out and push them somewhere.

1 Like
6

Apologies @AndyHoggANU, @anton and @willrhobbs, I’ve moved the poll into my first post since, as pointed out by @Aidan, the poll was slipping down in visibility due to the post voting. Would you be able to do the poll again please?

8

For equatorial/tropical Pacific verification, I would suggest the following:

  • Vertical profiles of temperature and zonal velocity compared to observations from the TAO array at key locations (e.g. 140W and 110W on the Equator). Example plots are already in COSIMA recipes (see the “Plot vertical profiles at the Equator compared to TAO, Johnson and WOA13 data” section of this notebook). The Johnson data set is pretty old now and comparing with it suffers from the issue of non-matching temporal periods (a particular issue in this region given the large internal variability). The comparison with TAO can be done robustly by restricting to times when observations exist (I have some code to do this somewhere, can dig it up if needed).

  • Equatorial slices of temperature bias compared to WOA13 (see “Plot longitude-depth temperature bias to WOA13” of this notebook, also see this notebook and Fig. 12 of Holmes et al. 2021). This will be really interesting to compare with the original ACCESS-OM2 biases because we expect that ACCESS-OM3 (depending on the vertical coordinate used) may have reduced spurious mixing that seems to have a big impact on the equatorial thermocline structure (Holmes et al. 2021).

  • Plots of Tropical Instability Wave SST and SSH variance compared to satellite observations. TIWs have a big impact on the equatorial heat budget and are particularly weak in ACCESS-OM2, and in fact in most models in OMIP-2, as described/summarized in this notebook/write-up. Again, the different numerics in ACCESS-OM3 may result in different TIWs.

2 Likes