From @rbeucher:
I found out that the ucx and python-ucx packages were not installed in the conda/analysis3-25.03 and conda/analysis3-25.04. I have no idea how they got dropped.
I have added them to conda/analysis3-25.05. UCX control communication between Gadi nodes so this might be the cause of some issues we had recently.
For anyone reading this in the future & experiencing segmentation faults in either the conda/analysis3-25.03 & conda/analysis3-25.04, either upgrading to conda/analysis3-25.05 or later or downgrading to conda/analysis3-25.02 is probably a good first step.