Clef error when searching for local data

General
, , , , ,
1

Hello,
I’m encountering an error using Clef when searching for local CMIP6 data. E.g. this search returns the following error:

module use /g/data3/hh5/public/modules
module load conda/analysis3-unstable

clef --local cmip5 --model MPI-ESM-LR --variable tas --table day

df = pd.read_sql(r.selectable, con=session.connection())
ERROR: Query must be a string unless using sqlalchemy.

The error goes away if I remove the “–local” flag but I then get search results that are not available on NCI and have to manually remove those. Any ideas on how to fix this would be welcome!

Thanks,
Anna

2

Hi @aukkola

Just FYI Clef is no longer supported as CLEX is closing. @holger might still be available until the end of the year.

I suspect this might be a pandas/sqalchemy version issue. I saw a similar problem with a different project that required rewriting some code from

pd.read_sql_query(qstring.format(namefield=name_sql ...)

to a form like so:

pd.DataFrame(data=list(self.db.query(qstring.format(namefield, ..)

I’m not expecting you to re-write the code, but thought it worth noting in case someone else does.

For you it might work to use an older version of the conda/analysis3 environment that has an older pandas/sqlalchemy combination.

1 Like
3

Hi Aidan,

Thanks, that makes sense. Would you be able to advise how to access older conda versions?

I also understood that hh5 continues to be supported and Clef is one of the tools available under hh5. Is it only the hh5 python environment that will be looked after by ACCESS-NRI?

Thanks,
Anna

4

Sure. You can use module avail to see available versions:

$ module use /g/data/hh5/public/modules/
$ module avail conda/analysis
---------------------------------- /g/data/hh5/public/modules -----------------------------------
conda/analysis3-22.01  conda/analysis3-23.01  conda/analysis3-24.01                              
conda/analysis3-22.04  conda/analysis3-23.04  conda/analysis3-24.04(analysis:analysis3:default)  
conda/analysis3-22.07  conda/analysis3-23.07  conda/analysis3-24.07(analysis3-unstable)          
conda/analysis3-22.10  conda/analysis3-23.10  conda/analysis27-18.10(analysis27)

So you could try conda/analysis3-24.01.

The short answer is yes, though as always there are nuances. I can confidently say that Clef is not currently a supported tool, and as far as I know there are no plans for that to change.

If it is a valuable tool for the community it might be possible for the community of users to take over support and updates.

Otherwise there is a small window to ask @holger for help to fix the current bug.

5

Was that an effective work-around @aukkola?

6

Hi Aidan,

Thanks for the help and clarifying what’s happening with hh5. I finally got round to testing the Clef code again and yes it works when I load an older conda environment.

Cheers,
Anna