Convert UM-output to netcdf: um2nc not working

Atmosphere
1

Dear ACCESS-NRI,
I tired to use um2nc:

module use /g/data/vk83/modules
module load payu
cd /scratch/k10/dxd565/tmp/
um2nc dj863a.pm0269dec test.nc

but the output cannot be read by “xconv” → “Segmentation fault”

Can someone please help me to convert the UM-output files to netcdf?

best regards
Dietmar

2

Hi @Dietmar_Dommenget,

Our staff that work on um2nc are both unavailable at the moment, but I will try and help.

Is it more than one file that you would like to convert or exactly that one?

Did you know that you can open UM files with xconv and convert individual fields to netcdf using the xconv application?

Another option is to open the file in iris and save the file as netcdf?
See Saving Iris Cubes — Iris 3.13.0.dev62 documentation

Iris is available in many of the existing conda environments. If you are interested in trying that out I am happy to help. I don’t have access to the k10 group though so we will have to find a way to pass over files.

1 Like
3

Hi @Dietmar_Dommenget,
I copied your test file to project lg87 which might be more accessible to ACCESS-NRI folks:

/scratch/lg87/dkh157/dietmar_test/dj863a.pm0269dec

I also tried running the um2nc command as you did:

um2nc dj863a.pm0269dec test.nc

And this seemed to work for me. I could open the test.nc file in ncview and in xconv ok. But there were some warnings and perhaps not every field was converted correctly.

I also ran the alternative script method which is a script called conv2nc.tcl, which is also copied into that folder.
Running the following:

conv2nc.tcl dj863a.pm0269dec

Creates a netcdf file called dj863a.pm0269dec.nc
this also seems ok (variable names are a bit different). I did notice that the file sizes ended up being a bit different. I removed the netcdf compression from each of the outputs to get a proper comparison of file sizes. The resulting files are saved as:

out_convsh.nc
out_um2nc.nc

(Both have compression removed.) The “out_convsh.nc” file from my script comes out at 40 MB, while the out_um2nc.nc file comes out as 21 MB. The original UM file is 40 MB. This suggests that a fair portion of the data was never converted properly by um2nc.

3 Likes
4

Hi @Dietmar_Dommenget,

I’ve tried reconverting the um file with um2nc and also ran into the same segmentation fault when opening the result with xconv. This appears to be a limitation with xconv, which can crash when a variable’s “long_name” attribute is too long. The netCDF file test.nc contains two variables with long_name attributes of length 110, field1221 and field1225 with long names product_of_effective_radius_of_stratiform_cloud_liquid_water_particle_and_stratiform_cloud_liquid_water_area_f and product_of_effective_radius_of_convective_cloud_liquid_water_particle_and_convective_cloud_liquid_water_area_f respectively.

Using nco to shorten these attributes:

ncatted -a,long_name,field1221,m,c,<shorter-name> -a,long_name,field1225,m,c,<shorter-name> test.nc shorter_name_test.nc

let me open the file with xconv. 105 characters seems to be the limit for xconv to work.

3 Likes