Skip to content

Latest commit

 

History

History
178 lines (145 loc) · 8.16 KB

devops-zerodowntime-automated.adoc

File metadata and controls

178 lines (145 loc) · 8.16 KB
Raw

Automated Zero-Downtime Deployment with CI/CD Pipelines

In this lab you will get learn how to automate deployments into production without the need for a service window and down time.

Background

In the previous labs you deployed and tested a code change directly in production without requiring downtime or a service window. Zero downtime deployments is an invaluable capability when building CI/CD pipelines in order to allow more frequent deployments in production without the need to go through cumbersome and bureaucratic processes for getting a change into production.

The manual blue/green deployment you followed in previous lab, is fun but not scalable for larger teams with many code commits per day. In this lab, you will automate blue/green deployment by integrating it into the Cart CI/CD pipeline. In that case, every time a change is committed to the code repository, the CI/CD pipeline would takes the code, build it and if successful in the Dev environment, promotes it into the inactive Cart container (the container that does NOT receive any production traffic) in the Prod environment. If the tests are successful in the Prod environment and after the release manager manually approves the Prod deployment, the CI/CD pipeline makes the new version live by switching the router and sending all traffic to the newly deployed container.

Blue-Green in CI/CD Pipeline

Blue-Green Deployment via CD/CD Pipeline

Open the Jenkinsfile you added to the pipelines Git repository in a text editor and replace its content with the following:

def tag,altTag

pipeline {
  agent {
    label 'maven'
  }
  stages {
    stage('Build App') {
      steps {
        sh "mvn clean package -s src/main/config/settings.xml"
      }
    }
    stage('Integration Test') {
      steps {
        sh "mvn verify -s src/main/config/settings.xml"
      }
    }
    stage('Build Image') {
      steps {
        script {
          openshift.withCluster() {
            openshift.withProject() {
              openshift.startBuild("cart", "--from-file=target/cart.jar").logs("-f")
            }
          }
        }
      }
    }
    stage('Deploy') {
      steps {
        script {
          openshift.withCluster() {
            openshift.withProject() {
              dc = openshift.selector("dc", "cart")
              dc.rollout().latest()
              timeout(10) {
                  dc.rollout().status()
              }
            }
          }
        }
      }
    }
    stage('Component Test') {
      steps {
        script {
          sh "curl -s -X POST http://cart:8080/api/cart/dummy/666/1"
          sh "curl -s http://cart:8080/api/cart/dummy | grep 'Dummy Product'"
        }
      }
    }
    stage('Promote') {
      steps {
        script {
          openshift.withCluster() {
            openshift.withProject("prod-{{PROJECT_SUFFIX}}") {
              def route = openshift.selector("route", "cart").object()
              def backends = []
              backends.add(route.spec.to)
              backends.addAll(route.spec.alternateBackends)
              def svc = backends.find {it.weight == 100}
              tag = svc.name == "cart-green" ? "blue" : "green"
              altTag = svc.name == "cart-green" ? "green" : "blue"

              openshift.tag("dev-{{PROJECT_SUFFIX}}/cart:latest", "prod-{{PROJECT_SUFFIX}}/cart:prod-${tag}")
              openshift.selector("dc", "cart-${tag}").rollout().status()
            }
          }
        }
      }
    }

    stage('End-To-End Test') {
      steps {
        script {
          sh "curl -s -X POST http://cart-${tag}.prod-{{PROJECT_SUFFIX}}.svc:8080/api/cart/dummy/444434/1"
          sh "curl -s http://cart-${tag}.prod-{{PROJECT_SUFFIX}}.svc:8080/api/cart/dummy | grep 'Pebble Smart Watch'"
        }
      }
    }

    stage('Approve Go Live') {
      steps {
        timeout(time:15, unit:'MINUTES') {
            input message: "Approve Promotion to Prod?", ok: "Promote"
        }
        script {
          openshift.withCluster() {
            openshift.withProject("prod-{{PROJECT_SUFFIX}}") {
              openshift.set("route-backends", "cart" , "cart-${tag}=100", "cart-${altTag}=0")
            }
          }
        }
      }
    }
  }
}

The above pipeline makes the following changes compared to the previous lab:

  • Promote stage is made aware of blue/green deployments

  • Approve stage is postponed till all tests are complete in the Prod environment

  • Go Live stage flips the router to send traffic to the new deployment

Notice that in the Promote stage, the pipeline retrieves the Cart route to check which version (blue or green) is receiving production flow and promotes the new Cart docker image to the inactive image which does not receive any product traffic. The Pipeline DSL is a Groovy syntax which is powerful for creating complex scenarios and enables dynamic decision making during pipeline execution similar to above.

Save the Jenkinsfile and push it to the Git repository and enter your credentials:

  • Username: {{GIT_USER}}

  • Password: {{GIT_PASSWORD}}

$ cd ~/pipelines
$ git add Jenkinsfile
$ git commit -m "Added blue-green deployment to the pipeline"
$ git push origin master

The purchase reports from CoolStore online shop show that reducing the minimum order to 40$ has had minimal effects on conversion rate which seem to be attributed to increased competition online. Pricing department has requested to reduce the minimum order to 20$ and evaluate the conversion rate after a week. Let’s push this change to production via the automated blue/green deployment in the pipeline.

Point your browser to the Git server web URL:

In the cart-service repository, browse to the promotion service at src/main/java/com/redhat/coolstore/service/PromoService.java, click on the pencil icon to open PromoService.java in the web editor and set the minimum order to 20$. The PromoService.java should look like this after the edit:

        //PROMO: if cart total is greater than 20, free shipping
        if ( shoppingCart.getCartItemTotal() >= 20) {
            ...
        }

Click on Commit Changes button to commit the new shipping promotion rule to the Git repository. Go back to OpenShift Web Console and in the Dev project, click on Builds → Pipelines on the left sidebar menu and wait until pipeline pauses at the Approve Go Live stage.

Approve Go Live in Deployment Pipeline

At this stage, a new version of the Cart service is deployed in the Prod environment as Cart Blue which does not receive any production traffic yet. Note that if you have executed the Cart CI/CD pipeline multiple times, Cart Green might be the inactive deployment in your environment.

After deployment is ready, verify that the new minimum order for free shipping is working correctly in the inactive container by adding 1 smartwatch to the test shopping cart :

Caution
 Replace the Cart route with routes in your project 
$ curl -X POST http://{{CART_BLUE_ROUTE}}/api/cart/FOO/444434/1
{"cartItemTotal":24.0,"cartItemPromoSavings":0.0,"shippingTotal":0.0,"shippingPromoSavings":-2.99,"cartTotal":24.0,"shoppingCartItemList":[{"price":24.0,"quantity":1,"promoSavings":0.0,"product":{"itemId":"444434","name":"Pebble Smart Watch","desc":"Smart glasses and smart watches are perhaps two of the most exciting developments in recent years. ","price":24.0}}]}

Notice that the shipping cost is zero since the total order is above the 20$ minimum order. Click on the Web UI route URL and add a Pebble Smart Watch to your shopping cart. As expected, the shipping cost is not zero.

Now that the new minimum order rule is verified in the new version of Cart service in the Prod environment, you can approve the Go Live. Go back to Builds → Pipelines and click on Input Required and then Promote to approve the Go Live. Add a Pebble Smart Watch to your shopping cart again and verify that shipping is now free in the live version.

Deployment Pipeline Complete