new spack user here - with the aim of using spack/payu as part of code development around ESM1.6 (so I’m having to deal with environments, concretization etc.)
I think I’ve navigated to the point, following the hive guidelines, where I can compile/build from source ESM1.6 (had to update the spack.yaml that comes with the git clone of spack). The general idea is that I will then be able to update the spack yaml file to a different um7 branch. When I do the
If the issue is what I’m thinking of, it could be an issue with how spack fetches the repositories that are used to build the packages - I’ve been dealing with that for a bit, now, but I have a workaround that I’ll share if it’s that issue. Alternatively, it could be an issue with a shallow clone of spack, but I’m not certain at the moment.
If you don’t mind, would you be able to run the same command with the debug and stacktrace commands enabled:
The output from that command was a lot. It looks like it was okay until
lib/spack/spack/util/module_cmd.py:78 ==> [2024-12-13-08:55:02.589104] module_cmd.load_module: gcc/14.1.0
lib/spack/spack/util/executable.py:233 ==> [2024-12-13-08:55:03.043590] ‘/apps/gcc/14.1.0/wrappers/gcc’ ‘-v’ ‘/scratch/p66/inh599/tmp/spack-implicit-link-infol4gcg42j/main.c’ ‘-o’ ‘/scratch/p66/inh599/tmp/spack-implicit-link-infol4gcg42j/output’
lib/spack/spack/solver/asp.py:2993 ==> [2024-12-13-08:55:03.829233] the C compiler None does not exist, or does not run correctly. The compiler intel@=2021.11.1 will not be used during concretization.
shortly followed by the error message above, then a long line of python error output which ends with
File “/g/data/p66/inh599/spack/0.22/spack/lib/spack/spack/version/version_types.py”, line 600, in ref_lookup
self._ref_lookup.get(self.ref)
File “/g/data/p66/inh599/spack/0.22/spack/lib/spack/spack/version/git_ref_lookup.py”, line 118, in get
self.data[ref] = self.lookup_ref(ref)
File “/g/data/p66/inh599/spack/0.22/spack/lib/spack/spack/version/git_ref_lookup.py”, line 149, in lookup_ref
self.fetcher.git(“fetch”, “–tags”, output=os.devnull, error=os.devnull)
File “/g/data/p66/inh599/spack/0.22/spack/lib/spack/spack/util/executable.py”, line 252, in call
raise ProcessError(“Command exited with status %d:” % proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 1:
‘/bin/git’ ‘-c’ ‘advice.detachedHead=false’ ‘-c’ ‘advice.detachedHead=false’ ‘fetch’ ‘–tags’
This looks like that one of the git addresses in the spack.yaml file (which I got from the ESM1.6.git suite) doesn’t exist - but I’ve no idea which one.
Hi Ian, I’m wondering if it’s due to tags being potentially clobbered - there was an issue with some git tags being moved in some repositories which could lead to the above error.
If you don’t mind, would you be able to give me the output of the following command (it is force-fetching updates to the tags in the repositories that are used). This command assumes you are cded into the git_repos directory that is in the directory above the spack installation:
for repo in *; do git -C "./$repo" fetch --verbose --update-head-ok --tags --force; done
Alright, that output is good news - the t [tag update] dev_2024.12.0 → dev_2024.12.0 sections mean that we’ve updated the tags that were failing earlier.
If you attempt to run spack concretize -f --fresh now on the model you want to build, what happens - does it succeed? I hope that this fix gets you a little bit further along!
Now having gone through this - thank you for your assistance - it maybe better to stop and pause at this point. I was hoping to assist with some last minute code edits as part of the immediate effort around ESM1.6 - but we’ve largely moved to a different mode of operation for now. It’s unlikely I’ll need to solve this before next year (if at all).
Of course - if you want me/us to continue trying as it would help identify something for more wider community benefit then let me know.
Ah, yep. So I’ve confirmed that instances in the packages section are fine, it’s the ones in the specs and modules section that are an issue. This is because it’s looking for the dev_2024.12.0 tag in the ACCESS-NRI/ACCESS-ESM1.6 repository, which doesn’t exist.
Instead, would you be able to use the following references:
In the specs section: - access-esm1p6@latest.
In the modules section: access-esm1p6: '{name}/latest' (or indeed, whatever modulefile name you would like to demarcate between other builds)
The spack concretize command worked with only a deprecation warning thrown
==> Warning: using “cmake@3.24.2” which is a deprecated version
This suggests that the spack.yaml file that forms part of the current/official/default release needs to be updated (or the ACCESS-ESM1.6 repo updated with that tag).
Thanks for the assistance in getting past that roadblock.
Interestingly during the spack install command it needed my gadi ssh/git passcode when completing
