rollouts-plugin-trafficrouter-consul

The Argo Rollouts plugin implementing traffic routing using Consul on Kubernetes to manage advanced subset-based routing for Canary deployments.

Consul's support for Argo Rollouts is currently limited to subset-based routing.

Install Argo Rollouts Controller

There are three methods for installing the Argo Rollouts Controller with Consul on Kubernetes:

1. Install Rollouts Using Helm and init containers. We recommend installing the Argo Rollouts Controllor using this method.
2. Install Rollouts Using Helm and binary
3. Standalone installation

After installing the controller, you must apply the RBAC CRD to your Kubernetes cluster.

Install Rollouts Using Helm and init containers

We recommend using this method to install this plugin.

Add the following code to your values.yaml file to configure the plugin:

controller:
  initContainers:
    - name: copy-consul-plugin
      image: hashicorp/rollouts-plugin-trafficrouter-consul
      command: ["/bin/sh", "-c"]
      args:
        # Copy the binary from the image to the rollout container
        - cp /bin/rollouts-plugin-trafficrouter-consul /plugin-bin/hashicorp
      volumeMounts:
        - name: consul-plugin
          mountPath: /plugin-bin/hashicorp
  trafficRouterPlugins:
    trafficRouterPlugins: |-
      - name: "hashicorp/consul"
        location: "file:///plugin-bin/hashicorp/rollouts-plugin-trafficrouter-consul"
  volumes:
    - name: consul-plugin
      emptyDir: {}
  volumeMounts:
    - name: consul-plugin
      mountPath: /plugin-bin/hashicorp

Then install the argo-rollouts and apply your updated values using Helm:

$ helm install argo-rollouts argo/argo-rollouts -f values.yaml -n argo-rollouts

Install Rollouts Using Helm and binary

To build the binary and install Rollouts, complete the following steps:

1. Build this plugin using your preferred tool. For example, make build.
2. Mount the built plugin onto the argo-rollouts container.
3. Add the following code to your values.yaml file to configure the plugin:

controller:
  trafficRouterPlugins:
    trafficRouterPlugins: |-
      - name: "argoproj-labs/consul"
        location: "file:///plugin-bin/hashicorp/rollouts-plugin-trafficrouter-consul"
  volumes:
    - name: consul-route-plugin
      hostPath:
        # The path being mounted to, change this depending on your mount path
        path: /rollouts-plugin-trafficrouter-consul
        type: DirectoryOrCreate
  volumeMounts:
    - name: consul-route-plugin
      mountPath: /plugin-bin/hashicorp

Then install the argo-rollouts and apply your updated values using Helm:

  $ helm install argo-rollouts argo/argo-rollouts -f values.yaml -n argo-rollouts

Stand-alone installation

This section describes the process to create a stand-alone installation. These instructions are for illustrative purposes. We recommend using init containers to create and install this plugin.

To create a stand-alone installation of the Rollouts plugin, complete the following steps:

1. Build this plugin.
2. Put the plugin on the path and mount it to the argo-rolloutscontainer.
3. Create a ConfigMap to configure argo-rollouts with the plugin's location.

The following example schedules a Deployment and mounts it to the argo-rollouts container:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: argo-rollouts
  namespace: argo-rollouts
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: argo-rollouts
  template:
    spec:
      # ...
      volumes:
        # ...
         - name: consul-plugin
           hostPath:
             path: /plugin-bin/hashicorp/rollouts-plugin-trafficrouter-consul
             type: DirectoryOrCreate
      containers:
        - name: argo-rollouts
        # ...
          volumeMounts:
             - name: consul-route-plugin
               mountPath: /plugin-bin/hashicorp

The following example creates the ConfigMap with the location of the plugin:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argo-rollouts-config
  namespace: argo-rollouts
data:
  trafficRouterPlugins: |-
    - name: "argoproj-labs/consul"
      location: "file:///plugin-bin/hashicorp/rollouts-plugin-trafficrouter-consul"
binaryData: {}

Install the RBAC

After either mounting the binary or using an init container, configure an RBAC using Argo Rollout Consul plugin rbac.yaml:

$ kubectl apply -f https://raw.githubusercontent.com/argoproj-labs/rollouts-plugin-trafficrouter-consul/main/yaml/rbac.yaml

Use the Argo Rollouts Consul plugin

Schedule the Kubernetes Service utilized by the service being rolled out. Additionally, configure any service defaults and proxy defaults required for the service.

apiVersion: v1
kind: Service
metadata:
  name: test-service
spec:
  selector:
    app: test-service
  ports:
    - name: http
      port: 80
      targetPort: 8080

Next, create the service resolver and service splitter CRDs for your stable service. Argo automatically modifies these CRDs during canary deployments.

The following example demonstrates the configuration of the service resolver CRD, which creates service subsets and sets the stable subset as the default:

apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceResolver
metadata:
  name: test-service
spec:
  subsets:
    stable:
      filter: Service.Meta.version == 1
    canary:
      filter: ""
  defaultSubset: stable

The following example demonstrates the configuration of the service splitter CRD, which initially sends 100% of traffic to the stable deployment:

apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceSplitter
metadata:
  name: test-service
spec:
  splits:
    - weight: 100
      serviceSubset: stable
    - weight: 0
      serviceSubset: canary

Then configure your Argo Rollout resource to incrementally rollout the canary deployment:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: test-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: test-service
  template:
    metadata:
      labels:
        app: test-service
      annotations:
        consul.hashicorp.com/connect-inject: "true"
        consul.hashicorp.com/service-meta-version: "1"
        consul.hashicorp.com/service-tags: "v1"
    spec:
      containers:
        - name: test-service
          # Using alpine vs latest as there is a build issue with M1s. Also other tests in multiport-app reference
          # alpine so standardizing this.
          image: docker.mirror.hashicorp.services/hashicorp/http-echo:alpine
          args:
            - -text="I am v1"
            - -listen=:8080
          ports:
            - containerPort: 8080
              name: http
      serviceAccountName: test-service
      terminationGracePeriodSeconds: 0 # so deletion is quick
  strategy:
    canary:
      trafficRouting:
        plugins:
          hashicorp/consul:
            stableSubsetName: stable # subset name of the stable service
            canarySubsetName: canary # subset name of the canary service
            serviceName: test-service
      steps:
      - setWeight: 20
      - pause: {}
      - setWeight: 40
      - pause: {duration: 10}
      - setWeight: 60
      - pause: {duration: 10}
      - setWeight: 80
      - pause: {duration: 10}

Finally, perform the Rollout operation using the Argo Rollouts Kubectl plugin.

kubectl argo rollouts promote test-service

Testing

To run unit tests use go test ./.... For end-to-end verification follow the steps in ./testing/README.md.