# Recommended ways to install python library basemap on gadi (if at all?) for dynlib **URL:** \ **Category:** Infrastructure\ **Tags:** python, help\ **Created:** [17 September 2024 07:55 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694 "2024-09-17T07:55:38Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Paul.Gregory](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/paul.gregory/32/2566_2.png) [@Paul.Gregory](https://forum.access-hive.org.au/u/Paul.Gregory)\ **Post date:** [17 September 2024 07:55 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/1 "2024-09-17T07:55:39Z")
Hi all. Michael Barnes from Monash would like to get the `dynlib` package installed on gadi: [Clemens.Spensberger / dynlib · GitLab](https://git.app.uib.no/Clemens.Spensberger/dynlib) This package still uses `basemap` for plotting. Based on some older threads, e.g.: > [@How to create conda enviroment on Gadi?](https://forum.access-hive.org.au/t/how-to-create-conda-enviroment-on-gadi/1597): > > Hello guys, I would like to install ENSO Metrix on Gadi: Following by this link: [Install using conda forge (recommended) · CLIVAR-PRP/ENSO\_metrics Wiki · GitHub](https://github.com/CLIVAR-PRP/ENSO_metrics/wiki/Install-using-conda-forge-(recommended)) My question is how can I create the conda environment?? conda create -n [YOUR\_CONDA\_ENVIRONMENT] -c conda-forge enso\_metrics Should I do something like conda --user?? Regards, Arnold I thought this would be a good place to ask for recommended strategies. I’m aware that `basemap` is deprecated and hence shouldn’t be included in the maintained conda environments. What I’ve tried so far. 1. Using Scott’s repo to build a new conda in a container: `https://git.nci.org.au/bom/ngm/conda-container` . This failed with `/local/spool/pbs/mom_priv/jobs/124860541.gadi-pbs.SC: line 20: build.log: Read-only file system` 2. I’m not that familiar with containers so I followed NCI suggestions to install extra modules on top of an existing python environment, e.g. [Installing custom Python packages - Specialised Environments - Opus - NCI Confluence](https://opus.nci.org.au/display/DAE/Installing+custom+Python+packages) So I tried: `pip install -v --no-binary :all: --upgrade-strategy only-if-needed --prefix /g/data/gb02/pag548/EXTRA_PYTHON_LIBS basemap` , which produced: ```auto Successfully built basemap basemap-data matplotlib packaging Installing collected packages: packaging, basemap-data, matplotlib, basemap Attempting uninstall: packaging Found existing installation: packaging 24.0 Uninstalling packaging-24.0: ERROR: Could not install packages due to an OSError. Consider using the `--user` option or check the permissions. OSError: [Errno 18] Invalid cross-device link: '/g/data/hh5/public/apps/miniconda3/envs/analysis3-24.04/lib/python3.10/site-packages/packaging-24.0.dist-info/' -> '/scratch/dx2/pag548/tmp/pip-uninstall-y81rzpso' ``` 1. I’m not sure why it couldn’t link to my own `/scratch/dx2/pag548/tmp` directory so I repeated the pip command using defaults: `pip install -v --no-binary :all: --upgrade-strategy only-if-needed basemap` This builds and installs with the following warnings: ```auto ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. nctoolkit 1.1.11 requires multiprocess, which is not installed. spectrum 0.8.1 requires easydev, which is not installed. arm-pyart 1.18.6 requires netCDF4>=1.7.0, but you have netcdf4 1.6.5 which is incompatible. dask-xgboost 0.1.11 requires xgboost<=0.90, but you have xgboost 2.0.3 which is incompatible. icclim 6.5.0 requires xclim<=0.47,>=0.45, but you have xclim 0.51.0 which is incompatible. xmip 0.7.2 requires xgcm<0.7.0, but you have xgcm 0.8.1 which is incompatible. ``` 1. It installed the new packages to: `/home/548/pag548/.local/lib/python3.10/site-packages/` . But, this `basemap` is incompatible with `matplotlib` installed in analysis3: ```auto $ python Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path.append('/home/548/pag548/.local/lib/python3.10/site-packages') >>> from mpl_toolkits import basemap Traceback (most recent call last): File "", line 1, in File "/home/548/pag548/.local/lib/python3.10/site-packages/mpl_toolkits/basemap/ __init__.py", line 33, in import matplotlib as mpl File "/home/548/pag548/.local/lib/python3.10/site-packages/matplotlib/ __init__.py", line 259, in _check_versions() File "/home/548/pag548/.local/lib/python3.10/site-packages/matplotlib/ __init__.py", line 244, in _check_versions from . import ft2font ImportError: /home/548/pag548/.local/lib/python3.10/site-packages/matplotlib/ft2font.cpython-310-x86_64-linux-gnu.so: undefined symbol: FT_Load_Glyph ``` So now I’m wondering if I should - build a new Conda environment just for basemap - Follow the NCI recommendation to compile all packages from source in a virtual environment using pip, e.g. [Python... - NCI Help - Opus - NCI Confluence](https://opus.nci.org.au/pages/viewpage.action?pageId=248840704) I have built fresh conda environments on gadi before, but I’m aware that this practise is frowned upon. As an aside, I recently had to compile an ancient version of basemap (v1.0.7) on a python2.7 platform. This required fiddling with macros in the GEOS header file `platform.h` but I got it working. I’m confident I can find a solution eventually, I’m just wondering if someone has done this already. I note the COSIMA team has already migrated away from basemap : [Shifting longitude domain from [-280, 80] to [0, 360]](https://forum.access-hive.org.au/t/shifting-longitude-domain-from-280-80-to-0-360/642), so perhaps another option is replacing all the `dynlib` calls to `basemap` with `cartopy`? ---
**Author:** ![navidcy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/navidcy/32/101_2.png) [@navidcy](https://forum.access-hive.org.au/u/navidcy)\ **Post date:** [17 September 2024 21:45 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/2 "2024-09-17T21:45:11Z")
Basemap which was the map package for matplotlib? Hasn’t that been deprecated looooong ago? ---
**Author:** ![navidcy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/navidcy/32/101_2.png) [@navidcy](https://forum.access-hive.org.au/u/navidcy)\ **Post date:** [17 September 2024 21:47 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/3 "2024-09-17T21:47:43Z")
oh wow… [it’s been undeprecated](https://github.com/matplotlib/basemap/issues/568#issuecomment-1331235348)! ---
**Author:** ![navidcy](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/navidcy/32/101_2.png) [@navidcy](https://forum.access-hive.org.au/u/navidcy)\ **Post date:** [17 September 2024 21:50 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/4 "2024-09-17T21:50:55Z")
I may be naïve here (meaning I didn’t read through all the log errors above @Paul.Gregory), but there are a few conda analysis environments on NCI that are curated by the CLEx CMS team (cc @Paola-CMS). Why not add `dynlib` on that environment? ---
**Author:** ![Paola-CMS](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/paola-cms/32/57_2.png) [@Paola-CMS](https://forum.access-hive.org.au/u/Paola-CMS)\ **Post date:** [17 September 2024 22:18 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/5 "2024-09-17T22:18:53Z")
Scott apparently had a gist specific to this: [Install dynlib at NCI · GitHub](https://gist.github.com/ScottWales/623cc0bd8a669643597b890b3f665865) NB it’s 4 years old ---
**Author:** ![Paul.Gregory](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/paul.gregory/32/2566_2.png) [@Paul.Gregory](https://forum.access-hive.org.au/u/Paul.Gregory)\ **Post date:** [17 September 2024 23:36 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/6 "2024-09-17T23:36:24Z")
Thanks for finding that Paola. I’ll have a look at that. ---
**Author:** ![Scott](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/scott/32/37_2.png) [@Scott](https://forum.access-hive.org.au/u/Scott)\ **Post date:** [18 September 2024 00:20 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/7 "2024-09-18T00:20:06Z")
Hi Paul, do you have the environment.yaml that failed in the container? I’ll see if I can reproduce ---
**Author:** ![Paul.Gregory](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/paul.gregory/32/2566_2.png) [@Paul.Gregory](https://forum.access-hive.org.au/u/Paul.Gregory)\ **Post date:** [18 September 2024 01:55 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/8 "2024-09-18T01:55:27Z")
Hi Scott. I had deleted it so I cloned your repo again and this time I kept `environment.yaml` as supplied. It generated the same error as before : `/local/spool/pbs/mom_priv/jobs/124909747.gadi-pbs.SC: line 20: build.log: Read-only file system` Now, I was checking out the repo in `/g/data/gb02/pag548` so I cloned it again in `~pag548/code` and this time it worked fine. Within this environment, what’s the recommended way to install numpy/scipy etc? Edit `environment.yaml` file and run the script again? ---
**Author:** ![Scott](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/scott/32/37_2.png) [@Scott](https://forum.access-hive.org.au/u/Scott)\ **Post date:** [18 September 2024 02:08 UTC](https://forum.access-hive.org.au/t/recommended-ways-to-install-python-library-basemap-on-gadi-if-at-all-for-dynlib/3694/9 "2024-09-18T02:08:24Z")
Could be that it needs a storage flag if you’re running in gdata. Yeah just add numpy etc to environment.yaml