ILAMB help: ilamb-tree-generator, adding datasets

I am trying to use ilamb-tree-generator to set up an ilamb run. The example yaml file in the documentation only shows CMIP6 historical runs. I appear to have successfully added amip and esm-hist runs using the exp field, but when I tried to add hist-noLu no files appeared in the relevant MODELS sub-directory. hist-noLu is in the LUMIP (rather than CMIP) directory under CMIP6 so is this the issue? Also, is there a way to use the yaml file to include cmorised but non-CMIP6 data? Maybe @RhaegarZeng best to help.

1 Like

Hi, Rachel:

I am really glad to hear that you’re using ilamb-tree-generator and also thanks for report the issue, that will help us keep update the tool.

I think this problem is because the directory structure have some slight differences between ‘CMIP’ and LUMIP. We didn’t test LUMIP yet but I will test later and reply to you soon.

1 Like

@RachelLaw I moved the topic into the Technical category as I think it is more appropriate for it. Just mentioning it in case you have problems finding it now.

Hi, @RachelLaw:
We have update the ilamb-tree-generator in xp65 and the problem you mentioned has solved. Now you can use ilamb-tree-generator to create hist-noLu in the LUMIP directory.

if you have any questions, feel free to contact me or post it in forum :grinning:.

2 Likes

Hi, @RachelLaw:
Has the problem been solved? if so could you please mark the reply as solution then we can close the topic.

Hi @RhaegarZeng, I seem to be getting an error message when I run ilamb-tree-generator : add_model_to_tree() missing 1 required positional argument: ‘mip’. Is this a new field that is needed in the yaml file?

I added this and it looks like it worked OK. The example yaml file in the documentation will need to be updated.

Also, if I have a model run that isn’t a CMIP6 dataset but has been cmorised, how do I include it in the ilamb analysis? Do I need to add it to the ILAMB_ROOT directory manually?

HI, @RachelLaw
Yes, we have slightly changed the format of ymal file and didn’t update the documentation yet, sorry for that, I will do it right away.

For the question, I think you are right, you can manually add the data to ILAMB_ROOT, and add the dataset to your ILAMB_setup file(which is the .txt file) as well if you use it. I suggest that you could have a test run on the dataset to make sure it can be read by ILAMB correctly. Not sure if this solve your problem, if not, you can show me an example so I can answer more specifically.

feel free to tell me if you have any issue while running ilamb.

Hi @RhaegarZeng,
The extra cases I want to try are ones that you had already run for me - but I want to run them alongside some of the standard CMIP6 cases so that I can directly compare them with iLamb.

One case is CMORised data on p73 e.g. /g/data/p73/archive/non-CMIP/CMORised/CNP-MIP/CSIRO/ACCESS-ESM1-5/hist-CN/r1i1p1f1

The other case isn’t CMORised e.g. /scratch/p66/txz599/ACCESS_output/HI-cable3-01

I guess I’m still not clear on the easiest way to point ilamb to this model output without using the ilamb_tree_generator.

Hi, @RachelLaw
About dataset saved in p73, now ilamb_tree_generator can not work on p73, so you need to manually create a ilamb_model_filetree, you can follow the format create by ilamb_tree_generator to do so, or you can follow instructions here. We do consider to add /g/data/p73/archive/non-CMIP/CMORised dataset in the next version of ilamb_tree_generator, and we will let you know when it’s available.

for non_cmip data /scratch/p66/txz599/ACCESS_output/HI-cable3-01 we have a tool to deal with it but it not release yet, I will try to bind this function to ilamb_tree_generator in the future. This time I can CMORise this data for you so you can use it directly.

Hi, @RachelLaw :grinning:

I CMORised the non-cmip dataset/scratch/p66/txz599/ACCESS_output/HI-cable3-01 and move it to /scratch/p66/yz9299/, it was named HI-cable3-01-CMORised, it has already been normalised to format which ILAMB can read so you can use it directly.

Hi @RhaegarZeng,
I have now successfully run a case that includes output from the p73 archive as well as the cable3 case - so I will tick the solution box. Thanks for the help. Sometime it might be useful to run a ‘drop-in’ session on iLAMB for people to ask/answer questions and share what they have learnt about using iLAMB.
Also we need to check the use of ra/rh in the CMIP6.cfg file for generating nee and ecosystem respiration. I suspect our CMIP6 ra is incorrect as it doesn’t include leaf respiration so we need to calculate nee and ecosystem respiration from other fields. I will try and check this.

2 Likes

Hi, @RachelLaw:

It’s very glad to hear that the solution we provide could help you.
I agree with you about the ‘drop-in’ session on iLAMB, it’s not just about help people to learn how to use ilamb and provide an opportunity for ilamb users to ask/answer questions, and also help us to find a direction about what we could do more and how to help the community in the future. So we will consider the suggestion.

For the ra/rh issue, we are willing to hear from you if you make any progress.

Since we solved the problem, I will close this topic. Feel free to ask on forum if you have other question. :grinning: