OK so I copied @mmr0 's configuration :
4x4 decomposition on a domain measuring 70x125 at 0.1 resolution,
to another directory and regenerated the default domain :
10x10 decomposition on a domain 140x249 at 0.5 resolution.
During the domain decomposition, the notebook generated the following:
Running GFDL's FRE Tools. The following information is all printed by the FRE tools themselves
NOTE from make_solo_mosaic: there are 0 contacts (align-contact)
congradulation: You have successfully run make_solo_mosaic
OUTPUT FROM MAKE SOLO MOSAIC:
CompletedProcess(args='/g/data/ik11/mom6_tools/tools/make_solo_mosaic/make_solo_mosaic --num_tiles 1 --dir . --mosaic_name ocean_mosaic --tile_file hgrid.nc', returncode=0)
cp: './ocean_mosaic.nc' and 'ocean_mosaic.nc' are the same file
cp: './hgrid.nc' and 'hgrid.nc' are the same file
cp ./hgrid.nc hgrid.nc
NOTE from make_coupler_mosaic: the ocean land/sea mask will be determined by field depth from file bathymetry.nc
mosaic_file is grid_spec.nc
***** Congratulation! You have successfully run make_quick_mosaic
OUTPUT FROM QUICK MOSAIC:
CompletedProcess(args='/g/data/ik11/mom6_tools/tools/make_quick_mosaic/make_quick_mosaic --input_mosaic ocean_mosaic.nc --mosaic_name grid_spec --ocean_topog bathymetry.nc', returncode=0)
===>NOTE from check_mask: when layout is specified, min_pe and max_pe is set to layout(1)*layout(2)=100
===>NOTE from check_mask: Below is the list of command line arguments.
grid_file = ocean_mosaic.nc
topog_file = bathymetry.nc
min_pe = 100
max_pe = 100
layout = 10, 10
halo = 4
sea_level = 0
show_valid_only is not set
nobc = 0
===>NOTE from check_mask: End of command line arguments.
===>NOTE from check_mask: the grid file is version 2 (mosaic grid) grid which contains field gridfiles
==>NOTE from get_boundary_type: x_boundary_type is solid_walls
==>NOTE from get_boundary_type: y_boundary_type is solid_walls
==>NOTE from check_mask: Checking for possible masking:
==>NOTE from check_mask: Assume 4 halo rows
==>NOTE from check_mask: Total domain size is 140, 249
_______________________________________________________________________
NOTE from check_mask: The following is for using model source code with version older than siena_201207,
Possible setting to mask out all-land points region, for use in coupler_nmlTotal number of domains = 100
Number of tasks (excluded all-land region) to be used is 98
Number of regions to be masked out = 2
The layout is 10, 10
Masked and used tasks, 1: used, 0: masked
1111111111
1111111111
1111111111
1111001111
1111111111
1111111111
1111111111
1111111111
1111111111
1111111111
domain decomposition
14 14 14 14 14 14 14 14 14 14
25 25 25 25 25 25 25 25 25 24
used=98, masked=2, layout=10,10
To chose this mask layout please put the following lines in ocean_model_nml and/or ice_model_nml
nmask = 2
layout = 10, 10
mask_list = 5,7,6,7
_______________________________________________________________________
NOTE from check_mask: The following is for using model source code with version siena_201207 or newer,
specify ocean_model_nml/ice_model_nml/atmos_model_nml/land_model/nml
variable mask_table with the mask_table created here.
Also specify the layout variable in each namelist using corresponding layout
***** Congratulation! You have successfully run check_mask
OUTPUT FROM CHECK MASK:
I then wrote the following the code to check the values of ocean_mask.nc and the ESMF mesh files.
input_dir=Path('/scratch/gb02/pag548/regional_mom6_configs/tassie-access-om2-forced')
# Load mask files
ocean_mask = xr.open_dataset(f'{input_dir}/ocean_mask.nc')
ESMF_mesh = xr.open_dataset(f'{input_dir}/access-rom3-ESMFmesh.nc')
ESMF_nomask_mesh = xr.open_dataset(f'{input_dir}/access-rom3-nomask-ESMFmesh.nc')
# Reconstruct the ESMF mesh as a 2-D array
ESMF_mask = xr.DataArray(ESMF_mesh.elementMask.values.reshape(249,140),
dims=['ny','nx'],
coords={'ny':ocean_mask.ny,
'nx':ocean_mask.nx})
fig,ax=plt.subplots(1,3,figsize=(15,4.5))
ocean_mask.mask.plot(ax=ax[0])
ESMF_mask.plot(ax=ax[1])
delta = ocean_mask.mask - ESMF_mask
delta.plot(ax=ax[2])
fig.suptitle('Delta b/w ocean_mask and ESMF_mask')
plt.tight_layout()
The two meshes are identical.
The delta min/max value is zero.
Additionally, the value of access-rom3-nomask-ESMFmesh.nc is 1.0 everywhere.
The pays run task fails with the same error:
FATAL from PE 86: ERROR: ESMF mesh and MOM6 domain masks are inconsistent! - MOM n, maskMesh(n), mask(n) = 306 1 0
FATAL from PE 82: ERROR: ESMF mesh and MOM6 domain masks are inconsistent! - MOM n, maskMesh(n), mask(n) = 81 1 0
Is this related to the output of check_mask from the notebook earlier?
Total number of domains = 100
Number of tasks (excluded all-land region) to be used is 98
Number of regions to be masked out = 2
...
domain decomposition
14 14 14 14 14 14 14 14 14 14
25 25 25 25 25 25 25 25 25 24
used=98, masked=2, layout=10,10
i.e. there are two regions in the domain that don’t align somehow?
Do I need to follow the advice in check_mask output?
To chose this mask layout please put the following lines in ocean_model_nml and/or ice_model_nml
nmask = 2
layout = 10, 10
mask_list = 5,7,6,7
EDIT : I’m guessing not as this corresponds to contents of mask_table.2.10x10 ?
2
10, 10
5,7
6,7