As mentioned by @rbeucher in How can we get new COSIMA recipes?, a great way to get more recipes would be to encourage (perhaps even require?) people who use the cookbook for publication figures to upload their notebooks to cosima recipes (perhaps in a “publications” folder, with a sub-folder for each paper), once the paper is accepted for publication.
I’m thinking there would be no obligation to maintain the notebook (it would simply be a record of what was done in a particular paper), but this could be still be super helpful for anyone wanting to reproduce an analysis with different data, or make a side-by-side comparison with published work, or understand how to make a particularly elegant style of plot, or whatever. It could also be beneficial for authors as a way to satisfy journal open access requirements. The whole community would benefit, for very little extra effort.
The danger with this would be that the repository would fill up with poorly maintained/documented scripts. However, nowadays journals are requiring authors to link to their analysis code and data and so if this repository was used as that location there is some requirement for cleanliness/documentation.
Aidan
(Aidan Heerdegen, ACCESS-NRI Release Team Lead)
3
To address this I’d suggest a separate repo for these. Could call it publications. Makes the purpose quite obvious and keeps the recipes a more focussed, curated collection.
Could add to zenodo and get a DOI for the publications repo that people could use. Not sure how that plays with commits/versions.
Worth keeping @rbeucher in the loop: he has plans for a service where folks can upload notebooks for sharing and review, which can provide a DOI, satisfying some of the requirements of publication. If that were to happen you might prefer to let people put their stuff there and just tag it COSIMA as it would be less work.
So I don’t like the idea because it will imply that we need to maintain all those scripts notebooks!
But we could have a list in the docs and/or in the README where we point to other GitHub repositories where people have put the scripts/notebooks that they used for a paper they published. E.g.,
This way there is a way to find more examples of cosima-recipes but the cosima-recipes repository don’t have to maintain those.
Perhaps, as @aidanheerdegen suggests, a separate repository where we gather this information is not a bad idea? Because oftentimes every publication is a bit different. For example “notebooks are here and output is in this doi on gadi but some also in zenodo and etc etc”. So each publication could have an entry in the README of the cosima/cosima-publications repository. And then in cosima-recipes we just point to that repository!
There are many ways to skin a cat. The main idea was to have a centralised collection/list of scripts used to create figures for papers, with minimal up-front and ongoing effort. So this could be
a folder in the cosima-recipes repo, with a subfolder for each paper
a separate publications repo, with a subfolder for each paper, linked to from the cosima-recipes README
a separate repo for each publication, with a separate link to each repo in the cosima-recipes README
In any of these cases, I think these contributions to the community should come with no obligation for the contributor or anyone else to maintain the scripts. They’d just be a record of what was done in a paper, with no additional effort on the script before submission, or ongoing maintenance burden. That keeps the barrier to contributing low, but the fact that they are from papers means they have passed peer-review so there is a quality threshold (even if they might break in future as packages change). Even scripts that no longer work provide useful templates, especially for work building on a published paper.
So this serves a different role, complementary to the rest of the recipes which provide tutorials and examples of best current practices and need to be maintained. So perhaps it would be cleaner to have these scripts in a separate repository (or several).
I see what you mean! I’d go with either 2nd or 3rd suggestion which implies that the notebooks live outside cosima-recipes.
I suggest that for whatever lives inside the cosima-recipes we do strive to ensure that it runs at all times and is updated and maintained. Otherwise newcomers get put down and loose trust and then develop or affirm the establised notion that they may have that “coding is hard and not for me”.
My idea on this, and this will take a bit of time is to have a Portal where people can submit recipes.
I am thinking broader than COSIMA.
There are some example of prototypes in the Neuroscience community where people can submit a Jupyter notebook and get a DOI etc… They use the framework that JOSS (Journal of Open Source Science) uses.
Here is an example:
I have put together a prototype for the ACCESS-MED team and that’s something I think would be valuable. I was gonna ask the COSIMA community whether that is something they would like to have.
The prototype is live here (not fully functional but close)
Provides a searchable gallery of examples
Provide a review process (via GitHub)
Abstract GitHub (Possibility to have different organisation, repos etc)
Provide a DOI and treat each submission as a sort of publication
Could be use to centralize documentation for tools…
We could hook a CI/CT system onto it so that recipes are tested regularly.