This topic is a catch-all location for feedback for the ACCESS-rAM3 Release 1.0.
Please reply to this topic if you have feedback on the ACCESS-rAM3 release. Feedback can be to point out problems encountered, or positive to highlight what worked well.
If your feedback is involved and will require specific help feel free to
make a topic on the ACCESS-Hive Forum.
If you’re not sure, reply here and your query can be moved to a separate topic if required.
Thanks NRI for the new release, congrats on the improvements implemented. I’ve just successfully completed a test of the latest branch access_rel_1 for the ancil and running suites.
One thing I hadn’t fully understood the implications was the new netcdf functionality (as I had been working from my own branch). I know this is a complex issue, and we asked for netcdf support, but I think there is scope to improve some of this. For example:
Previously loading files with iris on inspection of an output file (e.g umnsaa_pvera) we could see almost all variable names as simple text (e.g. “air_temperature”).
Now on inspection we only get a list of stashIDs (e.g. “STASH_m01s03i236”) as variable names. This makes it difficult to navigate data, especially for new users.
Previously when loading a cube with iris and converting to xarray, metadata would come across clearly (e.g. the array would be called “air_temperature”, with dimensions [“time”, “latitude”, “longitude”].
Now metadata is converted with non-standard names (e.g. time is “VT1HR”, latitude is “grid_latitude_t” and longitude is “grid_longitude_t”).
Perhaps there are good reasons not to simply rename everything as “time” if they are defined differently (e.g. instantaneous on the hour, hour average, or max in the hour), but this is a change to previous functionality, so should be documented. Personally, I would prefer “time” as a standard coordinate name, with the method detail included in attributes.
Previously where multiple time profiles were defined for a single variable (e.g. by default for air temperature in STASHPACK1 there is VTS1 and VT1HR: for first model timestep and instantanous on the hour), iris would concatenate these into a single cube.
Now with multiple time profile for a particular variable, multiple variables are created (e.g. for air temperature there is “STASH_m01s03i236” and “STASH_m01s03i236_2”), making it more difficult again to navigate. In general it is the _2 ones that I want to use (instantaneous on the hour, rather than the first timestep output).
I know there are no quick solutions to the above, and the UM systems don’t make it easy. For example… I don’t know why the standard UM STASHPACK1 (for as long as I can remember) has set a single value output at the first timestep for each variable. In free-running mode this means every cycle has to have that first timestep removed (or there are two values at the same time). Perhaps the Met Office included this because of some verification (i.e. with RES). But we could consider removing those single timestep outputs, seeing as they cause issue with the new netcdf functionality.
I also know that designing output workflows (e.g. for AUS2200 or BARRA or ESM) takes a huge amount of development effort (so thank you). I don’t know whether the standard STASHPACK1 is useful for the ACCESS community (would need discussion), but I think future efforts should focus on adapting the work from AUS2200 to make a useful set of outputs commonly used by researchers (as opposed to the NWP validation STASHPACKS that come from the Met Office).
Perhaps agreeing on better output workflows and/or stashpacks could be part of an ongoing discussion at the Atmosphere working groups, and a breakout session at the upcoming ACCESS annual workshop?
Examples of what I’m describing is included below:
import iris
import xarray as xr
import matplotlib.pyplot as plt
xr.set_options(display_max_rows=50)
datapath = '/scratch/fy29/mjl561/cylc-run/u-dg768/share/cycle/20220226T0000Z/Lismore/d0198/RAL3P2/um'
#### 1. VARIABLE NAME ISSUE
# PREVIOUSLY with iris
cbs = iris.load(f'{datapath}/umnsaa_pvera000')
print(cbs)
'''
0: m01s01i202 / (unknown) (time: 25; latitude: 450; longitude: 450)
1: Turbulent mixing height after boundary layer / (m) (time: 25; latitude: 450; longitude: 450)
2: m01s03i253 / (unknown) (time: 24; latitude: 450; longitude: 450)
3: Cumulus capped boundary layer indicator / (1) (time: 24; latitude: 450; longitude: 450)
4: air_temperature / (K) (time: 25; latitude: 450; longitude: 450)
5: atmosphere_boundary_layer_thickness / (m) (time: 8; latitude: 450; longitude: 450)
6: dew_point_temperature / (K) (time: 25; latitude: 450; longitude: 450)
7: fog_area_fraction / (1) (time: 25; latitude: 450; longitude: 450)
8: land_binary_mask / (1) (latitude: 450; longitude: 450)
9: relative_humidity / (%) (time: 25; latitude: 450; longitude: 450)
10: sea_ice_area_fraction / (1) (time: 24; latitude: 450; longitude: 450)
11: surface_air_pressure / (Pa) (time: 8; latitude: 450; longitude: 450)
12: surface_altitude / (m) (latitude: 450; longitude: 450)
13: surface_downwelling_longwave_flux_in_air / (W m-2) (time: 24; latitude: 450; longitude: 450)
14: surface_downwelling_shortwave_flux_in_air / (W m-2) (time: 24; latitude: 450; longitude: 450)
15: surface_net_downward_longwave_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
16: surface_snow_amount / (kg m-2) (time: 24; latitude: 450; longitude: 450)
17: surface_temperature / (K) (time: 25; latitude: 450; longitude: 450)
18: surface_upward_latent_heat_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
19: surface_upward_sensible_heat_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
20: toa_incoming_shortwave_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
21: toa_outgoing_longwave_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
22: toa_outgoing_shortwave_flux / (W m-2) (time: 25; latitude: 450; longitude: 450)
23: toa_outgoing_shortwave_flux / (W m-2) (time: 24; latitude: 450; longitude: 450)
24: visibility_in_air / (m) (time: 25; latitude: 450; longitude: 450)
25: wind_speed_of_gust / (m s-1) (time: 24; latitude: 450; longitude: 450)
26: x_wind / (m s-1) (time: 25; latitude: 451; longitude: 450)
27: y_wind / (m s-1) (time: 25; latitude: 451; longitude: 450)
'''
# NOW with netcdf
ds = xr.open_dataset(f'{datapath}/nc/umnsaa_pvera000.nc')
print(ds)
'''
<xarray.Dataset> Size: 506MB
Dimensions: (grid_longitude_t: 450, grid_latitude_t: 450,
bounds2: 2, VT1HR: 12, VTS0_0: 1, VT3HR: 4,
VTS1: 1, VTS1_rad_diag: 1, VT1HR_rad_diag: 11,
grid_longitude_uv: 450, grid_latitude_uv: 451,
height_10m: 1, height_1_5m: 1, VT1HRMAX: 12)
Coordinates:
* grid_longitude_t (grid_longitude_t) float64 4kB 148.8 ... 157.7
longitude_t (grid_latitude_t, grid_longitude_t) float64 2MB ...
* grid_latitude_t (grid_latitude_t) float64 4kB -32.95 ... -24.06
latitude_t (grid_latitude_t, grid_longitude_t) float64 2MB ...
* VT1HR (VT1HR) datetime64[ns] 96B 2022-02-26T01:00:0...
* VTS0_0 (VTS0_0) datetime64[ns] 8B 2022-02-26
* VT3HR (VT3HR) datetime64[ns] 32B 2022-02-26T03:00:0...
* VTS1 (VTS1) datetime64[ns] 8B 2022-02-26T00:01:00
* VTS1_rad_diag (VTS1_rad_diag) datetime64[ns] 8B 2022-02-26T...
* VT1HR_rad_diag (VT1HR_rad_diag) datetime64[ns] 88B 2022-02-2...
* grid_longitude_uv (grid_longitude_uv) float64 4kB 148.8 ... 157.7
longitude_uv (grid_latitude_uv, grid_longitude_uv) float64 2MB ...
* grid_latitude_uv (grid_latitude_uv) float64 4kB -32.96 ... -24.05
latitude_uv (grid_latitude_uv, grid_longitude_uv) float64 2MB ...
* height_10m (height_10m) float64 8B 10.0
* height_1_5m (height_1_5m) float64 8B 1.5
* VT1HRMAX (VT1HRMAX) datetime64[ns] 96B 2022-02-26T01:0...
Dimensions without coordinates: bounds2
Data variables:
rotated_latitude_longitude |S1 1B ...
grid_longitude_t_bounds (grid_longitude_t, bounds2) float64 7kB ...
grid_latitude_t_bounds (grid_latitude_t, bounds2) float64 7kB ...
STASH_m01s00i023 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s00i024 (VTS0_0, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s00i024_2 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s00i025 (VT3HR, grid_latitude_t, grid_longitude_t) float64 6MB ...
STASH_m01s00i030 (VTS0_0, grid_latitude_t, grid_longitude_t) int32 810kB ...
STASH_m01s00i031 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s00i033 (VTS0_0, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s00i409 (VT3HR, grid_latitude_t, grid_longitude_t) float64 6MB ...
STASH_m01s01i202 (VTS1, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s01i202_2 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s01i205 (VTS1, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s01i205_2 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s01i207_2 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s01i208 (VTS1_rad_diag, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s01i208_2 (VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) float64 18MB ...
STASH_m01s01i235 (VTS1_rad_diag, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s01i235_2 (VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) float64 18MB ...
STASH_m01s02i201 (VTS1_rad_diag, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s02i201_2 (VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) float64 18MB ...
STASH_m01s02i205 (VTS1_rad_diag, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s02i205_2 (VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) float64 18MB ...
STASH_m01s02i207 (VTS1_rad_diag, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s02i207_2 (VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) float64 18MB ...
STASH_m01s03i217 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
grid_longitude_uv_bounds (grid_longitude_uv, bounds2) float64 7kB ...
grid_latitude_uv_bounds (grid_latitude_uv, bounds2) float64 7kB ...
STASH_m01s03i225 (VTS1, height_10m, grid_latitude_uv, grid_longitude_uv) float64 2MB ...
STASH_m01s03i225_2 (VT1HR, height_10m, grid_latitude_uv, grid_longitude_uv) float64 19MB ...
STASH_m01s03i226 (VTS1, height_10m, grid_latitude_uv, grid_longitude_uv) float64 2MB ...
STASH_m01s03i226_2 (VT1HR, height_10m, grid_latitude_uv, grid_longitude_uv) float64 19MB ...
STASH_m01s03i234 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i236 (VTS1, height_1_5m, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i236_2 (VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i245 (VTS1, height_1_5m, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i245_2 (VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i248 (VTS1, height_1_5m, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i248_2 (VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i250 (VTS1, height_1_5m, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i250_2 (VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i253 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i281 (VTS1, height_1_5m, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i281_2 (VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i304 (VTS1, grid_latitude_t, grid_longitude_t) float64 2MB ...
STASH_m01s03i304_2 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
STASH_m01s03i310 (VT1HR, grid_latitude_t, grid_longitude_t) float64 19MB ...
VT1HRMAX_bounds (VT1HRMAX, bounds2) datetime64[ns] 192B ...
STASH_m01s03i463 (VT1HRMAX, grid_latitude_t, grid_longitude_t) float64 19MB ...
Attributes:
Conventions: CF-1.6
source: Met Office Unified Model v13.0
'''
##### 2. DEMENSION ISSUE #####
# PREVIOUSLY with iris
cb = iris.load_cube(f'{datapath}/umnsaa_pvera000', 'air_temperature')
da = xr.DataArray().from_iris(cb)
print(da)
'''
<xarray.DataArray 'air_temperature' (time: 25, latitude: 450, longitude: 450)> Size: 20MB
dask.array<filled, shape=(25, 450, 450), dtype=float32, chunksize=(1, 450, 450), chunktype=numpy.ndarray>
Coordinates:
* time (time) datetime64[ns] 200B 2022-02-26T00:01:00 ....
* latitude (latitude) float32 2kB -32.96 -32.94 ... -24.06
* longitude (longitude) float32 2kB 148.8 148.9 ... 157.7 157.7
forecast_reference_time datetime64[ns] 8B ...
height float64 8B ...
forecast_period (time) timedelta64[ns] 200B ...
Attributes:
standard_name: air_temperature
units: K
source: Data from Met Office Unified Model
um_version: 13.0
STASH: m01s03i236
'''
# NOW with netcdf
ds = xr.open_dataset(f'{datapath}/nc/umnsaa_pvera000.nc')
da = ds['STASH_m01s03i236_2'] # note the underscore because of two clashing time profiles
print(da)
'''
<xarray.DataArray 'STASH_m01s03i236_2' (VT1HR: 12, height_1_5m: 1,
grid_latitude_t: 450,
grid_longitude_t: 450)> Size: 19MB
[2430000 values with dtype=float64]
Coordinates:
* grid_longitude_t (grid_longitude_t) float64 4kB 148.8 148.9 ... 157.7 157.7
longitude_t (grid_latitude_t, grid_longitude_t) float64 2MB ...
* grid_latitude_t (grid_latitude_t) float64 4kB -32.95 -32.94 ... -24.06
latitude_t (grid_latitude_t, grid_longitude_t) float64 2MB ...
* VT1HR (VT1HR) datetime64[ns] 96B 2022-02-26T01:00:00 ... 2022...
* height_1_5m (height_1_5m) float64 8B 1.5
Attributes:
long_name: TEMPERATURE AT 1.5M
standard_name: air_temperature
units: K
cell_methods: VT1HR: point
grid_mapping: rotated_latitude_longitude
um_version: 13.0
um_stash_source: m01s03i236
packing_method: quantization
precision_measure: binary
precision_value: -6
'''
##### 3. TIME PROFILE ISSUE #####
# PREVIOUSLY with iris, which concatenates the time profiles in a cycle with glob
cb = iris.load_cube(f'{datapath}/umnsaa_pvera*', 'air_temperature')
# NOW with netcdf, we need to know the stash name, and whether there is a clash in time profiles
ds1 = xr.open_dataset(f'{datapath}/nc/umnsaa_pvera000.nc')['STASH_m01s03i236_2']
ds2 = xr.open_dataset(f'{datapath}/nc/umnsaa_pvera012.nc')['STASH_m01s03i236_2']
ds = xr.concat([ds1, ds2], dim='VT1HR')
# and then to simplify, rename dimensions and drop the unimportant ones
ds = ds.rename({'VT1HR': 'time', 'grid_latitude_t': 'latitude', 'grid_longitude_t': 'longitude'})
ds = ds.drop_vars(['height_1_5m', 'longitude_t', 'latitude_t'])
Hi Mathew,
Thanks for the feedback on UM NetCDF. The NetCDF support is currently based on UM NetCDF as documented at UMDP C11. This uses a translation from STASH to NetCDF CF that was based on Metarelate, as documented in UM Trac ticket #3370. In that Trac ticket I comment on the fact that the mapping is out of date since Iris is now independent from Metarelate. This is one reason that we were asking for feedback on the NetCDF output. I will create a UM Trac ticket asking for the mappings in STASH_to_CF.txt to depend directly on IRIS.
This, of course will not directly address the variable name issue that you have reported. The problem is caused by the variable name being mapped to the STASH source um_stash_source rather than the standard_name. If you look at the output such as share/cycle/20220227T0000Z/Lismore/d1000/GAL9/um/umnsaa_pvera000.nc using ncdump -h you will see variables like:
double STASH_m01s03i236_2(VT1HR, height_1_5m, grid_latitude_t, grid_longitude_t) ;
STASH_m01s03i236_2:long_name = "TEMPERATURE AT 1.5M" ;
STASH_m01s03i236_2:standard_name = "air_temperature" ;
STASH_m01s03i236_2:units = "K" ;
STASH_m01s03i236_2:coordinates = "longitude_t latitude_t" ;
STASH_m01s03i236_2:cell_methods = "VT1HR: point" ;
STASH_m01s03i236_2:grid_mapping = "rotated_latitude_longitude" ;
STASH_m01s03i236_2:_FillValue = -1073741824. ;
STASH_m01s03i236_2:um_version = "13.0" ;
STASH_m01s03i236_2:um_stash_source = "m01s03i236" ;
STASH_m01s03i236_2:packing_method = "quantization" ;
STASH_m01s03i236_2:precision_measure = "binary" ;
STASH_m01s03i236_2:precision_value = -6 ;
One problem with using the standard_name as the variable name is that it may not be unique in the file, so the variable name would need to be modified to distinguish variables. In this case, both STASH_m01s03i236 and STASH_m01s03i236_2 have standard_name "air_temperature". If standard_name was used instead, you might have (e.g.) variable names air_temperature and air_temperature_2 instead.
Perhaps you could suggest a better specification for the mapping from STASH fields to NetCDF CF variables than the one that is currently being used by UM NetCDF?
I would also be interested to know what happens when you use Iris instead of Xarray to load the NetCDF output files. Would that help?
In any case, should we also provide optional STASHpacks that revert to using the previous format?
See MOSRS UM ticket #7930 for the UM to NetCDF CF translation tables.
@rbeucher How would existing MED tools help users of ACCESS-rAM3 to evaluate the UM NetCDF output of the u-dg768 Regional Nesting Suite?
CSET is the latest version of the regional evaluation suite I believe - CSET documentation
I’m not very familiar with the model’s outputs, but I suppose this is close to ESM1.6. I’d expect ILAMB should be able to handle data from the UM. We can also explore some ESMValTool diagnostics once the data has been CMORised. If you can share an example output, we can start drafting a few examples. The main thing we’ll need is a clear mapping of the variables to their CMIP equivalents. Tagging @RhaegarZeng
I agree with Romain — a sample output along with a clear variable mapping would be very helpful for further developing
Thanks Paul for your reply.
I would also be interested to know what happens when you use Iris instead of Xarray to load the NetCDF output files. Would that help?
Iris does load the netcdf with readable variable names. I do currently use iris in my workflow, but I assumed a big draw for netcdf outputs would be so people can use tools they are much more familiar with like xarray and ncview. This solution would still require most users to convert from iris to xarray and then to netcdf, which negates a big reason to convert to netcdf in the suite.
Perhaps you could suggest a better specification for the mapping from STASH fields to NetCDF CF variables than the one that is currently being used by UM NetCDF?
um2nc.py renamed known variables to CMIP short names. Again, not ideal, just different. But CMIP short names are somewhat more understandable than stash codes.
Another suggestion for mapping to variable names (and my preference) is simply human readable forms as you suggested: air_temperature and air_temperature_2 etc. based on the standard_name, along with a simple way for users to add additional mapping (because we typically need to customise stashpacks). But I’m just one voice, and I feel this output discussion is a big change that affects all users and workflows, so would need a greater amount of consultation, testing and feedback (which I know you already called for). So until then I would suggest reverting to the previous format until we have broader community agreement (perhaps through the atmosphere WG?).
@mlipson could you copy a representative data file to /scratch/public that we could use as a common reference, then we can look at how the data might be accessed in a convenient manner, e.g. the cf-xarray package allows for data to be accessed by standard_name
Using cf-xarray is arguably a good idea anyway, as it means you can write much more model agnostic analyses, e.g.
Hi @Aidan, a high-level readable netcdf format is needed for end-users, not just an in-analysis solution. In my opinion, if files can’t be accessed quickly and easily human-readably e.g. from the command-line with ncdump or opened with ncview then it is not a sufficient solution - with the risk being that people won’t use the model.
Hi Aiden,
I’ve copied representative output data from an unmodified ACCESS-rAM suite to /scratch/public/mjl561. This is a single day’s output for one type of file (umnsaa_pvera), both netcdf (current) and pp (previous) versions.
Doing this I also see the netcdf is approximately twice the size of the pp, indicating the netcdfs could be optimised with compression/dtype selection.
Thanks for the feedback @bethanwhite. We’ll discuss this and let you know how we can address this in future releases.
In the mean-time I’ll try and provide some suggestions about how the current output format can be used most effectively.
Thanks. I’ve made a copy.
The netCDF files are compressed, and the chunk sizes seem reasonable (from a compression point of view). They do have metadata about quantisation and precision, I suppose these are inherited from the pp files as they are type double, and don’t appear to have any packing applied.
double STASH_m01s02i207_2(VT1HR_rad_diag, grid_latitude_t, grid_longitude_t) ;
STASH_m01s02i207_2:long_name = "DOWNWARD LW RAD FLUX: SURFACE" ;
STASH_m01s02i207_2:standard_name = "surface_downwelling_longwave_flux_in_air" ;
STASH_m01s02i207_2:units = "W m-2" ;
STASH_m01s02i207_2:coordinates = "longitude_t latitude_t" ;
STASH_m01s02i207_2:cell_methods = "VT1HR_rad_diag: point" ;
STASH_m01s02i207_2:grid_mapping = "rotated_latitude_longitude" ;
STASH_m01s02i207_2:_FillValue = -1073741824. ;
STASH_m01s02i207_2:um_version = "13.0" ;
STASH_m01s02i207_2:um_stash_source = "m01s02i207" ;
STASH_m01s02i207_2:packing_method = "quantization" ;
STASH_m01s02i207_2:precision_measure = "binary" ;
STASH_m01s02i207_2:precision_value = -6 ;
STASH_m01s02i207_2:_Storage = "chunked" ;
STASH_m01s02i207_2:_ChunkSizes = 1, 450, 450 ;
STASH_m01s02i207_2:_DeflateLevel = 1 ;
STASH_m01s02i207_2:_Endianness = "little" ;
Is this what you were referring to?