# "database disk image is malformed" // COSIMA Cookbook failure? **URL:** \ **Category:** COSIMA\ **Tags:** cosima, cosima-cookbook\ **Created:** [12 October 2023 03:02 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505 "2023-10-12T03:02:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Thomas-Moore](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/thomas-moore/32/329_2.png) [@Thomas-Moore](https://forum.access-hive.org.au/u/Thomas-Moore)\ **Post date:** [12 October 2023 03:02 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/1 "2023-10-12T03:02:14Z")
Hi all. Thanks to those credited with all the different cool tools & documentation for data discovery across the millions of netcdf files of model output we have before us. I’m trying to 100% confirm that while the **01deg\_jra55v140\_iaf\_cycle4** OM2 10th of a degree run does have daily 3D BGC output, it does not have 3D daily output for physical variables like `temp` or `salt`. So far I’ve confirmed this using (A) Mk1 eyeball squinting my way through the `ik11` forest of `netcdf` files and (B) making use of the new [ACCESS-NRI intake catalog](https://access-nri-intake-catalog.readthedocs.io/en/latest/index.html). Because I’ve got trust issues and also always assume I’ve done something wrong I’m trying to triple confirm this fact using the **COSIMA COOKBOOK**. However I can’t get [the CC tutorial](https://nbviewer.org/github/COSIMA/cosima-recipes/blob/master/Tutorials/COSIMA_CookBook_Tutorial.ipynb) working? `DatabaseError: (sqlite3.DatabaseError) database disk image is malformed` is the error I get when I run `cc.querying.get_experiments(session)`. I’m following the tutorial and working on ARE using the `conda/analysis3-unstable` module on NCI, as directed. What am I doing wrong that is silly? Thanks in advance for the support. ---
**Author:** ![aekiss](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aekiss/32/46_2.png) [@aekiss](https://forum.access-hive.org.au/u/aekiss)\ **Post date:** [12 October 2023 04:20 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/2 "2023-10-12T04:20:20Z")
01deg\_jra55v140\_iaf\_cycle4 doesn’t have any daily 3d output (the 3d BGC is 5-daily samples of 1-day means). See here: [https://cosima.org.au/index.php/2020/07/29/data-available-0-1-1958-2018-access-om2-iaf-run/](https://cosima.org.au/index.php/2020/07/29/data-available-0-1-1958-2018-access-om2-iaf-run/) ---
**Author:** ![Thomas-Moore](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/thomas-moore/32/329_2.png) [@Thomas-Moore](https://forum.access-hive.org.au/u/Thomas-Moore)\ **Post date:** [12 October 2023 04:23 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/3 "2023-10-12T04:23:17Z")
Thanks for that confirmation @aekiss, much appreciated. ---
**Author:** ![Thomas-Moore](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/thomas-moore/32/329_2.png) [@Thomas-Moore](https://forum.access-hive.org.au/u/Thomas-Moore)\ **Post date:** [12 October 2023 04:26 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/4 "2023-10-12T04:26:54Z")
RE: CC database While some tutorial code appears to fail, other commands work and provide answers on the frequencies available for 3D `temp` across the **01deg\_jra55v140\_iaf\_cycle4** experiment. Example: ```auto cc.querying.get_variables(frequency='1 monthly',session=session,experiment='01deg_jra55v140_iaf_cycle4',search='temp') ``` vs ```auto cc.querying.get_variables(frequency='1 daily',session=session,experiment='01deg_jra55v140_iaf_cycle4',search='temp') ``` It appears possibly that any CC database query that hits **all** experiments results in the `database disk image is malformed` error? ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [12 October 2023 04:52 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/5 "2023-10-12T04:52:07Z")
> [@Thomas-Moore](#): > > It appears possibly that any CC database query that hits **all** experiments results in the `database disk image is malformed` error? Hey @Thomas-Moore. Sometimes the DB gets corrupted, and you don’t necessarily see it if you’re not hitting the bad bits. So yeah, if you’re doing something that accesses more of it, you’re more likely to get the error. There are DB backups, so you could try and older one and see if that works. If so maybe let @micael know, and he can make sure the bad ones are removed. Edit: Forgot to mention where to find the DB backups ```auto /g/data/ik11/databases/daily/ ``` ---
**Author:** ![Thomas-Moore](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/thomas-moore/32/329_2.png) [@Thomas-Moore](https://forum.access-hive.org.au/u/Thomas-Moore)\ **Post date:** [12 October 2023 05:06 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/6 "2023-10-12T05:06:15Z")
Thanks for your help @Aidan - I’m basically a noob to the CC who couldn’t get the tutorial to run so thought I’d flag the error. I can get the answer I need via CC and also now have the authoritive confirmation from @aekiss. I guess it’s just an issue for anyone new trying the CC tutorial? `database disk image is malformed` is a scary looking error - doesn’t build confidence if you’re new to something. 😄 Actually this little exploration has shown how robust and useful the ACCESS-NRI intake catalog is. I never should have doubted it. 😉 ---
**Author:** ![Aidan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/aidan/32/42_2.png) [@Aidan](https://forum.access-hive.org.au/u/Aidan)\ **Post date:** [12 October 2023 05:11 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/7 "2023-10-12T05:11:09Z")
> [@Thomas-Moore](#): > > Actually this little exploration has shown how robust and useful the ACCESS-NRI intake catalog is. I never should have doubted it. Pull quote for the poster @dougiesquire ---
**Author:** ![micael](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.access-hive.org.au/micael/32/93_2.png) [@micael](https://forum.access-hive.org.au/u/micael)\ **Post date:** [12 October 2023 22:56 UTC](https://forum.access-hive.org.au/t/database-disk-image-is-malformed-cosima-cookbook-failure/1505/8 "2023-10-12T22:56:32Z")
Oh no, looks like the database has been corrupted ☹ I’ve removed the corrupted versions. Let’s see if the next update works correctly. > [@Thomas-Moore](#): > > Actually this little exploration has shown how robust and useful the ACCESS-NRI intake catalog is. I never should have doubted it. It seems clear that intake is the future for the COSIMA recipes.