I can confirm that the default cookbook database can still NOT open RYF data for me. I am trying to run cosima-recipes/Recipes/Surface_Water_Mass_Transformation.ipynb at main · COSIMA/cosima-recipes · GitHub with no changes, and it can not find the data.
The RYF is accessible by Intake (we updated the catalog already).
if the RYF is accessible by intake then let’s suggest that route to @sb4233 and to anybody else
I think until all of the COSIMA recipes no longer use Intake, we should still maintain the cookbook database.
true, I agree – but that’s a general comment, right?
this thread was specifically discussing RYF 0.1 deg which, if I understand correctly, is available by intake; so why not suggest using that?
I also see that my response is bit outdated since it seems that solutions have been provided. But I wanted to make the point that if effort is gonna be put, then the effort should be on making the modern system work.
Yes, I totally agree with that. outofscope reads bit bad and discourages people
I am marking this as the solution; right?
No this is not solved. I tried to submit a PR for the SWMT COSIMA recipe today to fix the bug identified this morning. But the recipe still uses cookbook not intake, and it wouldn’t run. So I couldn’t submit the PR.
Though I’m confused why the cookbook database works for @sb4233 and not me. Could someone else try running the SWMT COSIMA recipe (or any other RYF script) and see if they also have problems?
I’m bit confused. The issue raised in the first post of this thread isn’t it resolved by Can't access files in 01deg_jra55v13_ryf9091 simulation - #14 by sb4233? That’s what I marked as solution.
But this thread identified some other issues related with the cookbook which are yet not resolved. But I only marked the above as the solution to the original question by @sb4233.
Actually, sorry scrap that last message (getting years mixed up, clearly too early in the morning). But I still can’t open any RYF data.
@sb4233 what COSIMA cookbook session are you using?
This does not work for me (and is needed to run the current version of https://github.com/COSIMA/cosima-recipes/blob/main/Recipes/Surface_Water_Mass_Transformation.ipynb which I want to submit a PR for):
session = cc.database.create_session()
temp = cc.querying.getvar('01deg_jra55v13_ryf9091', 'temp' , session, frequency='1 monthly', start_time = '2080-01-01', end_time = '2080-12-31', chunks={})
I get a VariableNotFoundError: No files were found containing 'temp' in the '01deg_jra55v13_ryf9091' experiment.
@sb4233 - my suggestion is that if intake works for you, stick with it. You will need to convert soon anyway.
1 Like
There was also a post about the year 2080 but it seems to have disappeared. Anyway, I’ve checked, and the last output in ik11 is output1115 which ends at the start of 2080.
When I use the time range as you have here with the cookbook, it throws me the same error.
But when I try without the time range It works, but as you can see here: Can't access files in 01deg_jra55v13_ryf9091 simulation - #27 by sb4233, it has time dimension of only time:369.
Clearly it is missing the 2080 data or maybe it’s not picked up by the cookbook?
Whereas if I try my esm-datastore , you can see below it has a larger time dimension (time:2760)
Hi all, I’m trying to access the ryf data using the NRI intake catalogue. My understanding is that the catalogue has been updated to link to cj50, but I fail to do so as it still links to ik11.
@sb4233 seemed to have created their own catalogue (if I read this thread correctly) - are we supposed to create our own catalogues?
Here a minimal example which fails for me:
import intake
catalog = intake.cat.access_nri
expt = '01deg_jra55v13_ryf9091'
freq = '1mon'
cat_subset = catalog[expt]
cat_subset.search(variable='temp', frequency="1mon").to_dask()
Error message:
PermissionError: [Errno 13] Permission denied: '/g/data/ik11/outputs/access-om2-01/01deg_jra55v13_ryf9091/output890/ocean/ocean.nc'
The above exception was the direct cause of the following exception:
ESMDataSourceError Traceback (most recent call last)
Cell In[1], line 8
5 freq = '1mon'
6 cat_subset = catalog[expt]
----> 8 cat_subset.search(variable='temp', frequency="1mon").to_dask()
Am I doing something wrong?
Thanks for checking. I’m running this on ARE on the hh5 conda environment - that might be the issue!
Intake is picking up the complete experiment, but the cookbook has a bunch of years missing, amongst them 2080 
1 Like
I can do an index of that experiment specifically to (hopefully) force an update of its view on things?
Sounds awesome, thanks Angus!
Looks like I don’t have write permission to the database directory. The updated database is at /scratch/x77/ahg157/cosima_master_2025-05-28.db – maybe if you could do the finalisation step @aekiss?