# Code organisation for CABLE4 development
**URL:** https://forum.access-hive.org.au/t/code-organisation-for-cable4-development/1891
**Category:** CABLE
**Tags:** cable4-planning
**Created:** [6 March 2024 03:41 UTC](https://forum.access-hive.org.au/t/code-organisation-for-cable4-development/1891 "2024-03-06T03:41:36Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@clairecarouge](https://forum.access-hive.org.au/u/clairecarouge)
#### Post date: [6 March 2024 03:41 UTC](https://forum.access-hive.org.au/t/code-organisation-for-cable4-development/1891/1 "2024-03-06T03:41:36Z")
For CABLE4, we not only have to develop new code but we also need to merge two diverging versions of CABLE. It is worth discussing how we want to organise that work.
Below are my initial thoughts on this. Feedback is welcome.
# Type of work needed
1. **Refactorisation:** CABLE 3 started with an extensive refactorisation of the code. CABLE-POP\_TRENDY is based on CABLE 2 so has a completely different organisation of the code.
2. **Merge of the two branches:** the branches have diverged ~10 years ago. There will be both scientific and technical differences in the way both branches handle similar portions of the code. The merging can’t be automated as choices must be made on which version to keep.
3. **New development:** to use POP-POPLUC in ACCESS, we need to make some changes to the code (writing/reading files per processor, PFT mapping etc.). These changes can only be done on a code with a working version of POP-POPLUC.
# Aims for the work organisation
For me, the main aim should be to try and develop off the main branch as soon as possible. This would allow us to test the developments in the version of the code we are going to have in ACCESS3 from the start and not as a final phase. It should also facilitate keeping the development of ACCESS-ESM3 and ACCESS-ESM1.6 in sync with the offline development.
# Proposal
**Refactorisation**
I have come to think that doing the refactorisation first would be best. This would allow to transfer new developments between _main_ and _CABLE-POP\_TRENDY_ more easily by small blocks.
**Parallel work on merging the branches and new development**
The hope here is the merge will be done faster than the new developments so we can transition to doing all developments directly on _main_ quicker and avoid last-minute surprises that might happen by doing the merge at the end.
To do both the merge and the new developments in parallel, we need 2 development branches:
_POP-merge-dev_: One for the merge based on _main_. Write protected so all new development would need a PR. All work for the merge would be done in branches that stem from that branch and would have PR based on _CABLE4-dev_.
_CABLE4-dev_: Based on _CABLE-POP\_TRENDY_ for the new developments around POP-POPLUC (after the refactorisation happened if possible). Write protected so all new development would need a PR. All new development for CABLE4 would be done in branches that stem from that branch and would have PR based on _POP-merge-dev_.
Once _POP-merge-dev_ is done with, it is merged into _main_. Then we bring past developments from _CABLE4-dev_ into _main_. These might require to be done manually. Hopefully, things won’t have diverged much so it won’t bring surprises.
---
### Author:  [@inh599](https://forum.access-hive.org.au/u/inh599)
#### Post date: [7 March 2024 04:39 UTC](https://forum.access-hive.org.au/t/code-organisation-for-cable4-development/1891/2 "2024-03-07T04:39:47Z")
The practicalities to accomplish this CABLE4 development is worth careful consideration. The proposed way forward would certainly work - however there are implied elements that make this highly ambitious (i.e. risky in terms of completion by early 2025). The time constraint is particularly concerning given a) the starting point of the various code bases, b) the status of ACCESS-CM3 and c) the general level of (research scientist) resourcing available.
Some notes:
1. The aims of the work requires the development of the _MAIN_ offline model (and specifically an offline-as-ACCESS configuration), an ESM1.6 and ACCESS3. The choice of going to an inline solution necessitates i) development of multiple interface layers (since the parent models are not equivalent and require different methods) and ii) testing within a coupled environment (which takes substantially more time).
2. Any merge between _CABLE-POP\_TRENDY_ and _MAIN_ will be a **two way** merge - implying routine testing of capability in at least two offline and two coupled environments. Test cases will need to be established for (at least) the capability of POP, POPLUC, C13 cycle, BLAZE(?), coordination hypothesis, mesophyll conductance and ground water, across a combination of offline GSWP3, offline TRENDY, ACCESS-ESM1.x and ACCESS3x applications.
3. The two way merge would be both technical and science - and it’s likely that neither _MAIN_ nor _CABLE-POP\_TRENDY_ will be the truth for all decisions (implying that the test cases themselves will evolve over time, and necessitating testing both ways). If nothing else this means that the scientists will need to be deeply involved the discussion/process at all times.
4. From a practical perspective any merge will have to also consider the offline driver routines and MPI - substantive (multi-month) challenges in their own right.
5. Code Reviews will definitely be needed - this slows things down, if only because multiple people need to be on top of the purpose of each bit of work.
6. Our history of this kind of work indicates that this always takes **a lot** longer than initially thought (e.g. for CABLE3 into ESM1.5 there are differences in performance that we still do not understand why they have occurred nearly a year later) - and there is always a difficult challenge around determining when ‘close is good enough’.
7. (I find that) Parallel work always fails to produce the hoped for speed up (decision making becomes more complicated, conflicts and distractions occur)
Having articulated this (sub)set of concerns though - it is not obvious what an alternate approach would be. This is because development of CABLE4 needs to be done from a codebase that can run POP and POPLUC.
However - noting that CABLE4 does not require _CABLE-POP\_TRENDY_ to be merged into _MAIN_ nor be refactored - perhaps a way forward is to focus on build new developments onto _CABLE-POP\_TRENDY_, and then a one way merge (new _CABLE-POP\_TRENDY_ → _MAIN_), essentially leaving _CABLE-POP\_TRENDY_ unaltered, i.e.
1. Essentially _CABLE4-dev_ above - build new new developments around POP and POPLUC (i.e. configuration matching, processor-by-processor input) into _CABLE-POP-TRENDY_. Routine testing against _CABLE-POP-TRENDY_ POP on/off.
2. _MAIN-dev_ - based off _MAIN_, pull only those elements of _CABLE4-dev_ needed to enable POP and POPLUC into (a branch off) _MAIN_. Routine testing is done against _MAIN_ and _CABLE-POP-TRENDY_ without POP turned on, where land cover is equivalent to default POP land cover [a comparison between _MAIN_ and _CABLE-POP\_TRENDY_ in this way is a necessary early test to see how important differences in the biophysics are], and against _CABLE4-dev_ (POP on/off).
Document instances where _MAIN_ has been preferred over _CABLE-POP\_TRENDY_.
Note that the CABLE3 refactorisation, and other developments, imply that bitwise equivalence will not occur even if land cover is forced to be equivalent - so care will be needed when setting up test metrics.
3. Alter land cover to be consistent with ACCESS - test.
4. Move to tackle issues related to functioning in the coupled model(s) - variable allocation, offline spin up, pre-run splitting of POPs initial conditions, interface layers.
---
### Author:  [@clairecarouge](https://forum.access-hive.org.au/u/clairecarouge)
#### Post date: [7 March 2024 05:14 UTC](https://forum.access-hive.org.au/t/code-organisation-for-cable4-development/1891/3 "2024-03-07T05:14:05Z")
First thought: when do you tackle the refactorisation of CABLE-POP\_TRENDY. Because _MAIN\_dev_ becomes a headache if, for every change, we want to bring across, we have to also remap it from one code organisation to the other.