# Shared access spack configuration **URL:** \ **Category:** spack\ **Tags:** python, spack\ **Created:** [2 December 2022 06:16 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227 "2022-12-02T06:16:14Z")\ **Posts on this page:** 15\ **Page:** 2
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [4 April 2023 07:19 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/21 "2023-04-04T07:19:30Z")
Hi @Aidan, just wondering why `/g/data/access/spack/modules/spack/config` installs to `/scratch`? ```auto setenv SPACK_USER_CACHE_PATH /scratch/$project/$user/spack_user_cache setenv SPACK_INSTALL_TREE /scratch/$project/$user/opt/spack/ ``` I guess it’s for testing, but we’re [linking ACCESS-OM3 to these libraries](https://github.com/COSIMA/access-om3/tree/43bd9fbe630bfc1673d3001fc8dca851887ddcd9/build.sh), so it’ll break after 100 days… See [Spack libs shouldn't be in /scratch · Issue #18 · COSIMA/access-om3 · GitHub](https://github.com/COSIMA/access-om3/issues/18) ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [4 April 2023 07:32 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/22 "2023-04-04T07:32:56Z")
> [@aekiss](#): > > Why [spack] installs to `/scratch`? Yeah, that probably isn’t a great design decision. The whole thing needs a bit of a re-think. I’ll reply on this topic shortly\*\* \*\* when not deluged ---
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [4 April 2023 11:35 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/23 "2023-04-04T11:35:12Z")
Why not have this in `/g/data/access/spack/modules/spack/config` instead? ```auto setenv SPACK_USER_CACHE_PATH /g/data/$project/$user/spack_user_cache setenv SPACK_INSTALL_TREE /g/data/$project/$user/opt/spack/ ``` ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [17 April 2023 06:55 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/24 "2023-04-17T06:55:50Z")
The simplistic answer is that not all projects have `/g/data` allocations. Or they didn’t used to. That might have changed. I’m not sure if there aren’t other ways to do this entirely, but yes, as a first step that is a good change to make. ---
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [17 April 2023 06:59 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/25 "2023-04-17T06:59:05Z")
I made an attempt [here](https://github.com/COSIMA/access-om3/issues/18), but then hit an error I didn’t understand ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [17 April 2023 07:27 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/26 "2023-04-17T07:27:59Z")
I have changed the `spack/config` module to prefer using `/g/data/$PROJECT/$USER/` as the location for the install tree and cache. I guess the cache could stay in `/scratch` … hmm … but I changed them both for the moment. When I did a fresh clone of `spack` and checked out `releases/v0.19` I could successfully build `zlib`: ```auto $ spack install zlib ==> Installing zlib-1.2.13-qbwlozg4cobphkvyaizqmi47xw3ara5o ==> No binary for zlib-1.2.13-qbwlozg4cobphkvyaizqmi47xw3ara5o found: installing from source ==> Fetching https://mirror.spack.io/_source-cache/archive/b3/b3a24de97a8fdbc835b9833169501030b8977031bcb54b3b3ac13740f846ab30.tar.gz ==> No patches needed for zlib ==> zlib: Executing phase: 'edit' ==> zlib: Executing phase: 'build' ==> zlib: Executing phase: 'install' ==> zlib: Successfully installed zlib-1.2.13-qbwlozg4cobphkvyaizqmi47xw3ara5o Fetch: 0.12s. Build: 16.84s. Total: 16.96s. [+] /g/data/tm70/aph502/opt/spack/linux-rocky8-cascadelake/intel-2021.6.0/zlib-1.2.13-qbwlozg4cobphkvyaizqmi47xw3ara5o ``` It’d take me longer to figure out why your approach didn’t work, but if you’re happy to start from scratch that should work. Aside: make sure your first `spack` invocation is on a login node if you’re starting from scratch, as it needs internet access to bootstrap itself. ---
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [17 April 2023 23:09 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/27 "2023-04-17T23:09:43Z")
Thanks @Aidan. On second thoughts, if I want to make an executable for widespread use (e.g. access-om3) that lives in `ik11`, it would be better for its spack dependencies to live there too, say in `/g/data/ik11/spack` (NB: doesn’t involve $user) so that users only have to join `ik11` to run it. Would that be possible somehow? ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [18 April 2023 03:59 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/28 "2023-04-18T03:59:10Z")
> [@aekiss](#): > > Would that be possible somehow? Most certainly. The point of the `spack/config` module was to set up some shared config, but we can always set up something especially for your use case. I’m afraid I have limited resources this week (sick + very busy), but I do have getting a better development environment/process for ACCESS-OM3 high on my TO-DO agenda, and there are some options I’m thinking about. Maybe the best option is to actually start a topic about this for ACCESS-OM3 (or an issue on an appropriate repo). Would you be willing to do this? Outline what you want/need to achieve, what you’ve done so far and where it is failing? If you did make it a topic I’d suggest putting the topic in the COSIMA category. We can create a sub-category of ACCESS-OM3 if you think that is useful. ---
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [19 April 2023 00:30 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/29 "2023-04-19T00:30:14Z")
Thanks @Aidan, with your fix [I now have an executable that doesn’t depend on scratch](https://github.com/COSIMA/access-om3/issues/18#issuecomment-1513927063). ACCESS-OM3 issues are [here](https://github.com/COSIMA/access-om3/issues), and that’s probably a better place for discussion than the forum. Good to hear it would be possible to build dependencies into `/g/data/ik11`, but the top priority at this stage is to [fix the spack modules that don’t update `LD_LIBRARY_PATH`](https://github.com/COSIMA/access-om3/issues/17#issuecomment-1495433063), which makes it impossible to run the executable via payu. Get well soon! ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [2 May 2023 04:34 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/30 "2023-05-02T04:34:44Z")
So I don’t forget it, there are cases were `LD_LIBRARY_PATH` has to be set, notably for packages that are not compiled by `spack`. This PR is an example of this for reference > > > Since we just install the pre-compiled files and don't compile it ourself with R…PATH, this package doesn't actually work unless LD\_LIBRARY\_PATH is set. ---
**Author:** ![micael](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/micael/32/93_2.png) [@micael](https://forum.access-hive.org.au/u/micael)\ **Post date:** [10 May 2023 02:37 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/31 "2023-05-10T02:37:46Z")
For anyone interested, I’ve set up a new Spack instance on Gadi to be used by the COSIMA community. It reuses several things from @Aidan 's config, but I’ve tried to improve a bit on the design and make it a better fit for our use case. You can find all the config files here: > **[GitHub - COSIMA/spack-config](https://github.com/COSIMA/spack-config/)** > > Contribute to COSIMA/spack-config development by creating an account on GitHub. ---
**Author:** ![Scott](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/scott/32/37_2.png) [@Scott](https://forum.access-hive.org.au/u/Scott)\ **Post date:** [10 May 2023 04:36 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/32 "2023-05-10T04:36:15Z")
The containerised spack environment I’ve been working on is now available at [ScottWales/spack-environments (github.com)](https://github.com/ScottWales/spack-environments). This is a mirror since we’ve been heavily using Gitlab CI for the setup. With bind mode MPI set up performance of the container matches an uncontainerised environment, with the advantage that the container is entirely self-contained and portable to other systems ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [10 May 2023 05:40 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/33 "2023-05-10T05:40:53Z")
> [@Scott](#): > > This is a mirror since we’ve been heavily using Gitlab CI for the setup. Thanks for taking the time to make a public version available for the wider community. Much appreciated. ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [3 April 2025 04:45 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/34 "2025-04-03T04:45:06Z")
This is all pretty horribly out of date now. Use the [docs on ACCESS-Hive to setup spack](https://access-hive.org.au/getting_started/spack/) > **[Set up Spack for building ACCESS models - ACCESS-Hive Docs](https://access-hive.org.au/getting_started/spack/)** ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [3 April 2025 04:45 UTC](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227/35 "2025-04-03T04:45:12Z")
[Previous page](https://forum.access-hive.org.au/t/shared-access-spack-configuration/227.md?page=1)