This suggests conda/analysis3-25.05 will be deprecated which is needed when using cylc7, see some older discussions here and here.
E.g., I load the 25.05 environment in my CM2 suites like so: suite.rc: [[ocean_ke_check]] inherit = POSTPROC, NCI, SHARE pre-script = """ module use /g/data/xp65/public/modules module unload python module load conda/analysis3-25.05
So my question is whether there are suggested workarounds?
I have re-enabled analysis3-25.05 for now. However, I think it would be worth looking at an alternative way of running your models. In particular, could you consider setting up and using your own conda environment?
It would also be good to check whether updating cylc is feasible on your side.
@rbeucher
Although I agree on Cylc7 being old, it’s not that easy to switch to Cylc8 straight away.
We will slowly transition all ACCESS models that currently use Cylc7 to Cylc8, but there will definitely be some transition period during which it would be simpler if analysis3-25.05 was available so everything could still work.
Open to discussion and I can leave it available for now, however analysis3 is a massive kitchen sink environment with 100s of packages. I suggest moving to a dedicated environment for Cylc7 if the community wants to have it available.
It would be helpful if the conda environment modules could be made more hygenic - there shouldn’t be a need for them to imitate a ‘conda load’ in the module file to set environment variables outside of the container.
Ideally loading the conda environment shouldn’t be breaking other things - the Cylc setup does make an effort to have its python environment separate from any user environment that might be loaded.
I just double-checked and the latest analysis3-26-02 module doesn’t seem to export the _CONDA_PYTHON_SYSCONFIGDATA_NAME env variable (that based on the related links in Wilma’s original message seem to be the cause of incompatibility).
@wilma have you tried by chance to use analysis3-26-02 and see if it works?
I haven’t tried using analysis3-26-02as an older recommendation was that any analysis3-26 and older won’t work.
How would I upgrade to cylc8 - is that model/suite specific? I work with different CM2 configurations and because it is the latest (although old) published ACCESS-CM model, I think it is important that we can still run it.
The majority of these variables shouldn’t be getting set by the module, as they may cause conflicts with other packages as seen in this thread. Since you’re using a container already, make use of it to isolate the conda environment from the outside environment.
You should only need something like this (not tested), set the rest of the variables in the container launch script so they don’t affect outside of the container
We let the host environment handle the environment variables, the container approach is only meant to load the squashfs files systems where the conda environments are deployed. The main goal was to solve the issue of the inodes. I was not the one who designed the system so I might be missing some details.
I suppose we could have a hybrid system where minimal essential variables are defined inside the container. I think it is quite low priority though and I would rather investigate solution to replace that monolithic system
-