How to do a Deployment

How to Perform a BFD Deployment

Follow this runbook to successfully build and deploy the BFD Services code to target environment(s).

1. Create a new GitHub tag-based Release

• Visit the Build Release Job in GitHub Actions
• Click "Run Workflow". The following overrides are available and are not required for the vast majority of releases. Aside from the workflow selection branch, each override has sufficient documentation in the manual dispatch workflow interface as seen by clicking the "Run Workflow" button:

  Override	Description
  "Use workflow from"	The targeted workflow branch. Defaults to Branch: master
  releaseBranch	Defaults to whatever is selected in the above "Use workflow from"
  releaseVersion	Defaults to promoting the SNAPSHOT version found in the bfd-parent pom.xml
  developmentVersion	Defaults to incrementing the releaseVersion and appending "SNAPSHOT"
  awsRegion	Defaults to us-east-1
  forceRelease	Defaults to creating a GitHub release object for non pre-release builds, identified by appearance of a '-' in the release version

• Monitor GitHub Actions log output for your Build Release job
• Upon Completion, this workflow's output should include:
  • a git tag reference appearing here matching the expected releaseVersion
  • a GitHub Release based on this git tag reference with attached reference artifacts, e.g. data dictionary
  • matching versioned deployable artifacts stored in AWS Code Artifact (for disaster recovery: AWS Code Artifact in us-west-2)

---

2. Access WEB-based CloudBees SDA Cloud Operations Center (Jenkins) https://jenkins-east.cloud.cms.gov/bfd and select BFD from the Dashboard.

   • Optional: The pipeline includes a couple of fail-safes that can be optionally enabled to lock environments before moving on to the next stage of the pipeline. To create an exclusive lock on one or more environments, select Lockable Resources from the left-side list of actions; the UI provides an interface where each environment can be locked by toggling its checkbox. Locking an environment precludes another build from deploying to the locked environment.

3. From the list of Jenkins jobs, select the BFD Multibranch and Multistage Pipeline.

4. Scan for the new tag you created by clicking Scan Repository Now on the left sidebar.

   • Beneath the Scan Repository Now link, select the Scan Repository Log if you want to view the scan log and ensure the tag was picked up

5. Ensure no other jobs are currently building or queued up to build; it's also a good practice to check if another developer is performing work in the Test environment. This can be done by checking the Slack channel #bfd-builds. If the Test environment is not clear, then one should either postpone the build, or contact the developer to see if they can relinquish their claim to the Test environment.

6. While not required, it is convenient to notify BFD developers of the intent to perform a deployment; this can be done by posting a message to the #bfd-builds Slack channel. NOTE: This message should also be posted for the other environments such as prod-sbx & prod.

   • post message: /dibs on test to stake claim to the test environment.
   • post message: /dibs off test to release claim to the test environment.

7. Once the environment is clear, we can deploy from the new tag. In Jenkins, go to the "Tags" tab at the top of BFD Multibranch and Multistage Pipeline.

8. Find the tag in the list that corresponds to your new tag. Click into it.

9. When build commences, the Stage View is displayed showing the now activated current pipeline processing stage(s). If you need to see more detail of the pipeline build, then click the current build number (under Build History). Some useful actions that provide more detailed information and control:

   • Console Output provides a detailed view of the console output log.
   • Pipeline Steps provides a dynamic hierarchical view of the pipeline stages.
   • Pause/resume allows you to stop (and restart) the pipeline processing.

10. When the build pipeline finishes the deployment to Test, the pipeline may pause requesting a Manual Approval to proceed on to prod-sbx, prod. Select Proceed to cause the pipeline to continue its processing to PROD-SBX and PROD if you determine no manual testing is needed on this release.

11. Ensure all environment steps were green (successful).

    • At the conclusion of a deployment to each of the environments (prod, prod-sbx, test), the pipeline performs a step that runs a regression suite vs. the BFD API Services; the intent is to try to proactively verify that no functional or performance regression results from the deployment. It does this by capturing results from the deployment vs. a set of baseline numbers; this has historically been a top contributor to a deployment failures, mainly failing on performance degradation. If an environment fails, more detailed information can be found in:
      • for deployments based on master, we visualize historical regression results in AWS QuickSight for test, prod-sbx, and prod.
      • for for all deployments, Jenkins logs point operator to the raw AWS CloudWatch stored regression suite results

12. When the pipeline has completed the deploy from master, there's a couple of minor tasks that will need to be done:

    • Release your hold on the Test environment in #bfd-builds as needed (/dibs off test).
    • Notify our partners that a deployment to Prod has been completed; this is done by posting a message to the #bfd-users Slack channel. While there is no prescribed content of the message, a best practice is to link to the released tag you created in github which will list out all the changes present in the release:

@here :wave: [BFD has deployed release 2.45.0](https://github.com/CMSgov/beneficiary-fhir-data/releases/tag/2.45.0)