Deploy on Release

[surchs]

Context

It has been quite a while since our tools were released. One reason for that is that we essentially deploy directly to production from the head branch of our tools. With a growing number of remote nodes, that becomes less practical. And it's also a good idea in general to have deterministic deployments, i.e. of specific versions.

Why

• Part of Grow userbase #49
• So we have frequent, versioned builds and releases
• So we have a clear language to discuss issues (what version) and node deployments

Outcomes

• We have a simple process to make a release, ideally whenever a small set of new features have accumulated
• On release we tag builds with the new version
• We have (semi)-automatic processes to deploy newly released versions to production
• Existing processes that deploy directly from the HEAD of the main branch are removed

What

• Rename image tag built by current CD workflow to nightly api#239
• Create workflow for building + pushing stable versioned images on release api#241
  • neurobagel/api:stable -> in future, when this moves, we want to give it a last-stable or similar tag
  • neurobagel/api:v0.1.1

Auto releasing using auto:

• We don't want to use the major label b/c this will trigger major version bump
• Need to also filter out bot PRs -> this includes dependabot, syncing of workflows
  • https://intuit.github.io/auto/docs/generated/omit-release-notes
• We want to rename the default changelog title sections (based on labels) to something more conventional/informative
  • https://intuit.github.io/auto/docs/configuration/autorc#adding-additional-changelog-title-sections

What we need to do next:

• Manually add release label to CLI
• Set flag for creating release only on PRs with release back to true in CLI
• Add plugin to allow auto-release on protected branch bagel-cli#259
  • [FIX] Use different reviewer token in release.yaml bagel-cli#268 (ensure the plugin uses a different PAT)
• Customize changelog section titles #91
• Delete test releases of CLI
• Also, the build on release workflow is not successfully triggered due to automation not being able to trigger workflow, see https://github.com/orgs/community/discussions/25281
  • Solution is, provide a PAT - also fixes our branch protection problem (EDIT: nope, branch protection still blocks the release, so the plugin is still needed)
  • here is the official information on this: https://docs.github.com/en/actions/using-workflows/triggering-a-workflow#triggering-a-workflow-from-a-workflow
• Use auto locally to add default release-related labels to planning repo and ensure they are propagated
• Add auto release workflow + config to workflows repo so it gets propagated workflows#43
• Sync workflow that will build Docker image on release to auto-releasing repos workflows#45
• filter out bot PRs + contributions: dependabot, pre-commit, Seb's workflow ones
  • https://intuit.github.io/auto/docs/generated/omit-commits
• add first time contributors section https://github.com/intuit/auto/tree/main/plugins/first-time-contributor
• Update default/recommended Docker image tag for services recipes#9
• update PR template to include check box for PR label

Important reference issues:

• Default labels conflict with Dependabot intuit/auto#2412
• Remove Dependabot PRs from release notes? intuit/auto#2031
• omit-release-notes not excluding pull requests with git-tag intuit/auto#2145
• Auto cannot push to a protected branch intuit/auto#2377
• Auto Shipit with Protected Branch intuit/auto#945

Good examples of release workflows:

• https://github.com/physiopy/phys2bids/blob/master/.github/workflows/auto-release.yml
• https://github.com/dandi/dandi-cli/blob/master/.github/workflows/release.yml

Related

• [INFRA] add dependabot api#103
• Release 0.1.1 of CLI including tagged Docker Hub image bagel-cli#236
• CLI release 0.2.0 ✨ bagel-cli#235
• Build Docker on release, not (just) push bagel-cli#183
• Automate GraphDB setup recipes#18
• Cut a first release for the dashboard ✨ digest#104
• Make a new release annotation_tool#630

For reference, this is the process that Dandi uses for release automation: https://github.com/dandi/dandi-cli/blob/master/.github/workflows/release.yml

This uses: https://github.com/intuit/auto

Related

• [ENH] Have a jsonschema serialization of our data models bagel-cli#150

[alyssadai]

See also https://github.com/neurobagel/documentation/wiki/Release-process

[alyssadai]

Closing as all main tasks are complete! 🚀🎉