# All netcdf\_conversion jobs failed from last Saturday **URL:** \ **Category:** AM3\ **Tags:** help\ **Created:** [31 March 2026 03:59 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164 "2026-03-31T03:59:24Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [31 March 2026 03:59 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/1 "2026-03-31T03:59:24Z")
Hi Team All my netcdf\_conversion jobs within ACCESS-AM3 alpha release run failed since last Saturday. Before that time, all jobs succeeded. I am wondering whether there are changes to the gadi python environment, or because too many jobs occupied the python. It seems netcdf\_conversion does not start its job until exceeding walltime. You can find logs to a failed job here `/scratch/public/qg8515/jobf.out`, and logs to a succeeded job here `/scratch/public/qg8515/jobs.out` Any insights would be appreciated. Regards, Qinggang ---
**Author:** ![atteggiani](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/atteggiani/32/212_2.png) [@atteggiani](https://forum.access-hive.org.au/u/atteggiani)\ **Post date:** [1 April 2026 02:51 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/2 "2026-04-01T02:51:46Z")
Hi @qinggangg, can you please also share the error log for the failed run? > [@qinggangg](#): > > and logs to a succeeded job here `/scratch/public/qg8515/jobs.out` ~~Also, the `/scratch/public/qg8515/jobs.out` path doesn’t seem to exist~~ EDIT: Found! It’s at `/scratch/public/qg8515/.jobs.out` ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [1 April 2026 03:24 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/3 "2026-04-01T03:24:44Z")
Hi @qinggangg, which branch of the configurations were you using as your start point for this experiment? I can’t replicate your errors from the current `dev-n96e` branch. Would you be able to show what additions you’ve made to your `.bash_profile`? The `PATH` in `jobf.out` looks a bit odd. There are some miniconda paths in there, which could be causing conflicts with the `um2netcdf4` environment. ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 03:31 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/4 "2026-04-01T03:31:11Z")
Hi @atteggiani Thank you. I copied the succeeded and failed job.err files to the folder `/scratch/public/qg8515` as well. ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 03:34 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/5 "2026-04-01T03:34:49Z")
Hi @lachlanswhyborn I added the following two lines in .bashrc files. But it was done a long while ago rather than last Saturday. ```auto module load ncview module load netcdf ``` I can also share the whole settings in .bashrc but they are not modified recently and are mainly alias and export commands. There is also a conda section: ```auto # >>> conda initialize >>> # !! Contents within this block are managed by 'conda init' !! __conda_setup="$('/home/563/qg8515/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" if [$? -eq 0]; then eval "$__conda_setup" else if [-f "/home/563/qg8515/miniconda3/etc/profile.d/conda.sh"]; then . "/home/563/qg8515/miniconda3/etc/profile.d/conda.sh" else export PATH="/home/563/qg8515/miniconda3/bin:$PATH" fi fi unset __conda_setup # <<< conda initialize <<< ``` ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 03:37 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/6 "2026-04-01T03:37:50Z")
Would it also be easy to do netcdf\_conversion offline? ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 03:45 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/7 "2026-04-01T03:45:36Z")
I created my own branch from dev-n96e in a forked repository. I added many output variables which makes the netcdf\_conversion job very memory heavy. ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [1 April 2026 04:06 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/8 "2026-04-01T04:06:10Z")
Hmm, the `netcdf_conversion` job normally takes on the order of a few minutes, so it seems unlikely that any reasonable amount of added variables would cause it to take 4 hours. I see you added a couple more log files to that shared `scratch` directory, which suggest the netcdf conversion was successful- did you make any changes to achieve this? ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 04:07 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/9 "2026-04-01T04:07:52Z")
The successful netcdf\_conversion jobs were run before last Saturday, which took around 2.5 hours. From last Saturday, all netcdf\_conversion jobs failed by exceeding walltime. ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [1 April 2026 05:04 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/10 "2026-04-01T05:04:45Z")
The version of `conda/analysis` is different between runs- the successful run used `conda/analysis-25.08`, while the failed run used `conda/analysis3-26.01`. I know that `ants` was removed from the `conda` environments for all past `25.08`, as it was placing untenable restrictions on other package versions. The conda environment is actually loaded when loading `pythonlib/um2netcdf4/xp65`. This modulefile was updated on Friday afternoon, which lines up with the behaviour you’re seeing. We’ll do some investigation internally to see if we can work out if it actually this change in version causing the problem, and if there’s a way around it. In the meantime, a temporary workaround may be to add `module use /g/data/xp65/public/modules` and `module load conda/analysis3-25.08` to the NetCDF conversion task in `suite.rc`, before the `module load pythonlib/um2netcdf4/xp65` line. The `um2netcdf4` only loads the default `conda/analysis` if a version of it is not already loaded, so by loading a specific version, you should be able to restore the previous behaviour. ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 05:10 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/11 "2026-04-01T05:10:48Z")
Thank you. I will check now. ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [1 April 2026 11:38 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/12 "2026-04-01T11:38:49Z")
Hi @lachlanswhyborn Thank you for the suggestion, but the job failed again by exceeding walltime. I copied the failed and succeeded logs here `/scratch/public/qg8515` in `netcdf_conversionf` and `netcdf_conversions` for comparison. ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [1 April 2026 22:32 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/13 "2026-04-01T22:32:21Z")
Looks like it still used `conda/analysis-26.01`, so it must be checking for that specific module version rather than any version. Can you try: 1. Removing the current contents of `pre-script` in the `[[netcdf_conversion]]` task, and just have `module use /g/data/xp65/public/modules` and `module load conda/analysis-25.08`, so we load the specific version of `conda/analysis`. 2. Copy the files `um2netcdf4.py` and `stashvar_cmip6.py` from `/g/data/access/apps/pythonlib/um2netcdf4/2.1` into the `app/netcdf_conversion/file` directory in the configuration. The other thing the original `um2netcdf4` was add this directory to the `PYTHONPATH`, but we can just bypass this by adding them to the working path. 3. Re-run the NetCDF conversion task. If it still uses `conda/analysis-26.01`, then I’ll be very confused and will have to call for some backup. ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [1 April 2026 23:41 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/16 "2026-04-01T23:41:53Z")
---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [2 April 2026 00:39 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/17 "2026-04-02T00:39:05Z")
I’ll check now. ---
**Author:** ![lachlanswhyborn](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/lachlanswhyborn/32/1837_2.png) [@lachlanswhyborn](https://forum.access-hive.org.au/u/lachlanswhyborn)\ **Post date:** [2 April 2026 07:23 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/18 "2026-04-02T07:23:32Z")
This should now be fixed on the default configuration, with a reversion of the `um2netcdf4/xp65` modulefile. ---
**Author:** ![qinggangg](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/qinggangg/32/4460_2.png) [@qinggangg](https://forum.access-hive.org.au/u/qinggangg)\ **Post date:** [2 April 2026 23:09 UTC](https://forum.access-hive.org.au/t/all-netcdf-conversion-jobs-failed-from-last-saturday/6164/19 "2026-04-02T23:09:43Z")
Thank you. This issue is fixed for me now.