Continuous Integration Testing framework using GitHub actions

[rbeucher]

I have started a repo on the ACCESS-NRI organisation to automate testing of the recipes:
https://github.com/ACCESS-NRI/COSIMA-recipes-workflow

This is to eventually replace the Jenkins server on access-dev which will be shut down in Q1 2024.

This is based on what I have set up for the ESMValTool recipes:

https://github.com/ACCESS-NRI/ESMValTool-workflow?tab=readme-ov-file#recipes-current-status

Will keep you posted.

[anton-seaice]

Thanks Romain

All recipes in test/test_notebooks.py currently run with the exceptions of :

• SeaIce_Obs_Model_Compare which works fine in ARE, but fails as a Gadi job because it relies on accessing data through a URL / online. This relies on the NSIDC Sea Ice Conc dataset - which I think is being added to project CT11 for esmvaltool.

These two recipes need some review / rework and are currently not expected to pass

• Decomposing_kinetic_energy_into_mean_and_transient.ipynb (Issues with DocumentedExamples/Decomposing_kinetic_energy_into_mean_and_transient.ipynb #229)
• SeaIceSeasonality_DFA.ipynb (Update SeaIceSeasonality_DFA.ipynb notebook #271)

Some recipes have cells which are expected to fail / show errors for the purposes of the recipes. These cells are marked with a 'skip-execution' cell tag which skips that cell when run as the Jenkins job, but runs the rest of the notebook below the skipped cell. (It would be even better if they weren't skipped, but it was checked that they generated the expected error and the rest of the notebook was run!)

[rbeucher]

SeaIce_Obs_Model_Compare which works fine in ARE, but fails as a Gadi job because it relies on accessing data through a URL / online. This relies on the NSIDC Sea Ice Conc dataset - which I think is being added to project CT11 for esmvaltool.

@flicj191 is working on a CMOR compatible version for ESMValTool.

[navidcy]

@rbeucher where are we with this? in the cosima-recipes we dropped Jenking so now we have no CI at all... is there a chance that we use the framework you are proposing now?

[navidcy]

Our repo file structure changed a lot btw. Do we want the testing to be in a different repo as you have started it @rbeucher or is this just a prototype to see whether the testing works. Seems good and it's great that you get a green or red badge! I'm very interested in incorporating in this repo so people know straight up whether the notebooks are working.

[rbeucher]

Hi @navidcy ,

I think the testing should go in the COSIMA repo.
It needs a bit of refactoring and I need to adjust the folder structure. I'll have a look next week.

[navidcy]

Great! Exciting!