I am trying to set up a quasi-2d version of Panan with 0.1deg resolution, same latitude extension as panan01, but a very small 4 x-cells domain on t-points (0.4deg extent)
I set up the domain as reentrant on X, but not on Y. Boundary conditions at 37S same as panan01, and used the domain tools (GitHub - COSIMA/domain-tools) to create the topography and masks ( the same set of steps I used when adapting panan01 to panan0025).
if I run the model with any land point, the model crashes with no error on mom6.out, and the following one on mom6.err: [gadi-cpu-clx-2769:476035:0:476035] Caught signal 11 (Segmentation fault: Sent by the kernel at address (nil))
However, the model runs fine without any land points (or all land points masked).
Anybody has any suggestion on where could the problem be coming from? I uploaded here mom6 logs and MOM/SIS input. SIS_input.txt (1.6 KB) mom6.out.txt (11.8 KB) mom6.err.txt (13.0 KB) MOM_input.txt (7.6 KB)
I think you said it worked with 10 cells?
Did it work with 5?
Also, doe sit matter what “shape” your land is? Is the coastline entirely zonal, or does it vary in x, in which case there might be a jump across the re-entrant channel?
So anything above 4 cells seems to work in a case where we have no land points. I tried progressively lower from 10 to 4 t-cells, and all worked (If we use less the 4 cells the mom6.out error states that the domain has to be positive).
The segmentation fault only happens when I add a land point in the South of the domain. The coast is entirely zonal:
I can run the model if I fully mask the CPUs dealing with the land points using the mask table. But that doesn’t seem ideal, because to apply the mask table I have to increase the total numbers of CPUs used to run the model, and in this case the ocean model starts applying the boundary conditions in the south cells too, making the shelf too warm. ( I also find odd not being able to run the model with a land cell, so I was wondering if there was any rookie mistake on the set up).
The crash happens during initialisation when FMS tries to regrid the initial temperature state onto the model grid (because TEMP_SALT_INIT_VERTICAL_REMAP_ONLY = False in your configuration).
From the output, the following is printed from the FMS find_neighbor routine, indicating that there’s something wrong with the x-direction extent, which is what leads to your crash:
is,ie= 0 -1795032128
js,je= 98 245
I can run the model if I fully mask the CPUs dealing with the land points using the mask table
I’m not sure why you would mask an entire CPU if you only add a single land point? Most of its domain would still be ocean, so I think that is just hiding the problem.
Anyway, I think the most likely way forward is to make sure your IC is on the model grid, and flip the setting so it doesn’t need to do the initial remapping.
I’m trying out setting the IC to the same grid and the model. One question, do you know where the model gets the is,ie from? Thinking what part of the model I need to set up so I don’t get the same error in the domain extent in the future
In the snippet I posted, that’s just part of the find_neighbor, which I was showing as an indication of why it is crashing. I don’t really know all the parameters for that routine, it depends on the tile sizes for each rank, and the size of your IC domain and model domain. An error like that kind of suggests to me that the IC and model don’t overlap, or something like that, but it’s hard to say for sure.
The figure you have looks like an idealized channel setup rather than a quasi-2D panantarctic? Perhaps consider changing the issue title as it’s a bit misleading. (I opened the issue thinking that it might be a case where regional-mom6 package might come to the rescue but I realised that it’s about setting up an idealised config.)
Perhaps this title is more appropriate: “Issue creating an idealised channel configuration mimicking the Antarctic shelves-margins”
