Pulumi Kubernetes Operator

A Kubernetes operator that provides a CI/CD workflow for Pulumi stacks using Kubernetes primitives. To learn more about the Pulumi Kubernetes Operator visit the Pulumi documentation.

Overview

• Pulumi Kubernetes Operator
  • Overview
    • What is Pulumi?
    • When To Use the Pulumi Kubernetes Operator?
    • Prerequisites
      • Install Pulumi CLI
      • Login to Your Chosen State Backend
  • Deploy the Operator
    • Using kubectl
    • Using Pulumi
  • Create Pulumi Stack CustomResources
    • Using kubectl
    • Using Pulumi
    • Extended Examples
  • Stack CR Documentation
  • Prometheus Metrics Integration
  • Development

What is Pulumi?

Pulumi is an open source infrastructure-as-code tool for creating, deploying, and managing cloud infrastructure in the programming language of your choice. If you are new to Pulumi, please consider visiting the getting started first to familiarize yourself with Pulumi and concepts such as Pulumi stacks and backends.

When To Use the Pulumi Kubernetes Operator?

The Pulumi Kubernetes Operator enables Kubernetes users to create a Pulumi Stack as a first-class Kubernetes API resource, and use the StackController to drive the updates. It allows users to adopt a GitOps workflow for managing their cloud infrastructure using Pulumi. This infrastructure includes Kubernetes resources in addition to over 60 cloud providers including AWS, Azure, and Google Cloud. The operator provides an alternative to Pulumi's other CI/CD integrations such as Github Actions, Gitlab CI, Jenkins etc. See the full list of Pulumi's CI/CD integrations here. Since the Pulumi Kubernetes Operator can be deployed on any Kubernetes cluster, it provides turnkey GitOps functionality for Pulumi users running in self-hosted or restricted settings. The Kubernetes Operator pattern, lends itself nicely to automation scenarios by driving to the specified state and automatically retrying if transient failures are encountered.

Prerequisites

The following steps should be completed before starting on Pulumi:

Install Pulumi CLI

Follow the Pulumi installation instructions for your OS. For instance, on Mac OS, the easiest way to install Pulumi CLI is from Homebrew:

$ brew install pulumi

Login to Your Chosen State Backend

The operator stores additional metadata about provisioned resources. By default, Pulumi (and the Pulumi Kubernetes Operator) uses the Pulumi managed SaaS backend to store this state and manage concurrency. However, in addition to the managed backend, Pulumi also readily integrates with a variety of state backends, like S3, Azure Blob Storage, Google Cloud Storage, etc. See here for a detailed discussion on choosing a state backend.

Login to Pulumi using your chosen state backend. For simplicity we will only cover the Pulumi managed SaaS state backend and AWS S3 here:

Pulumi SaaS Backend

$ pulumi login

This will display a prompt that asks for you to provide an access token or automatically request an access token:

Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account/tokens
    or hit <ENTER> to log in using your browser                   :

In order to configure the Pulumi Kubernetes Operator to use Stacks with state stored on the SaaS backend, you will also need to manually generate access tokens. This can be done by accessing the Access Tokens page. Setting the environment variable PULUMI_ACCESS_TOKEN to the manually generated token will obviate the need for a pulumi login.

At this point your pulumi CLI is configured to work with the Pulumi SaaS backend.

AWS S3 Backend

1. First, you will need to create an S3 bucket manually, either through the AWS CLI or the AWS Console.
2. If you have already configured the AWS CLI to use credential files, single sign-on etc., Pulumi will automatically respect and use these settings. Alternatively you can set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables to the access key and secret access key respectively.
3. To use the AWS S3 backend, pass the s3://<bucket-name> as your <backend-url> to pulumi login, i.e.:

   $ pulumi login s3://<bucket-name>
   

   For additional options, refer to the Pulumi documentation.
4. You will need the AWS credentials when configuring Stack CRs for stacks you wish to be backed by the S3 bucket.
5. Lastly you will need to create an AWS Key Management Service (KMS) key. This key will be used by Pulumi to encrypt secret configuration values or outputs associated with stacks. Pulumi ensures all secrets are stored encrypted in transit and at rest. By default, the SaaS backend creates per-stack encryption keys to do this, however, Pulumi can leverage KMS as one of several supported encryption providers instead, thus allowing users to self-manage their encryption keys.

Deploy the Operator

Deploy the operator to a Kubernetes cluster.

You can use an existing cluster, or get started by creating a new managed Kubernetes cluster. We will assume that your target Kubernetes cluster is already created and you have configured kubectl to point to it. Note that Pulumi doesn't actually use kubectl but for convenience can use the same mechanism to authenticate against clusters.

Using kubectl

First, download the latest release source code tar ball and expand it locally.

Deploy the CustomResourceDefinitions (CRDs) for the operator.

kubectl apply -f deploy/crds/

Deploy the API resources for the operator.

kubectl apply -f deploy/yaml

Using Pulumi

First, make sure you have reviewed and performed the tasks identified in the prerequisite section.

Download the latest release source code tar ball and expand it locally. Change directory to the expanded tarball (e.g. pulumi-kubernetes-operator-x.y.z).

TypeScript

1. Go to the deploy/deploy-operator-ts directory
2. Run pulumi stack init and choose a stack name. If you are using the S3 backend, you may specify the KMS key as well:

   $ pulumi stack init --secrets-provider="awskms:///arn:aws:kms:...?region=<region>"

3. Run yarn install to install the dependencies
4. Run pulumi up

Python

1. Go to the deploy/deploy-operator-py directory
2. Run pulumi stack init and choose a stack name. If you are using the S3 backend, you may specify the KMS key as well:

   $ pulumi stack init --secrets-provider="awskms:///arn:aws:kms:...?region=<region>"

3. Run pulumi up

C#

1. Go to the deploy/deploy-operator-cs directory
2. Run pulumi stack init and choose a stack name. If you are using the S3 backend, you may specify the KMS key as well:

   $ pulumi stack init --secrets-provider="awskms:///arn:aws:kms:...?region=<region>"

3. Run pulumi up

Go

1. Go to the deploy/deploy-operator-go directory
2. Run pulumi stack init and choose a stack name. If you are using the S3 backend, you may specify the KMS key as well:

   $ pulumi stack init --secrets-provider="awskms:///arn:aws:kms:...?region=<region>"

3. Run pulumi up

Create Pulumi Stack CustomResources

The following are examples to create Pulumi Stacks in Kubernetes that are managed and run by the operator.

Using kubectl

Check out Create Pulumi Stacks using kubectl for YAML examples.

Using Pulumi

Check out Create Pulumi Stacks using Pulumi for Typescript, Python, Go, and .NET examples.

Extended Examples

• Managing a Kubernetes Blue/Green Deployment
• AWS S3 Buckets

If you'd like to use your own Pulumi Stack, ensure that you have an existing Pulumi program in a git repo, and update the CR with:

• An existing github project and/or commit,
• A Pulumi stack name that exists and will be selected, or a new stack that will be created and selected.
• A Kubernetes Secret for your Pulumi API accessToken,
• A Kubernetes Secret for other sensitive settings like cloud provider credentials, and
• Environment variables and stack config as needed.

Stack CR Documentation

Detailed documentation on Stack Custom Resource is available here.

Prometheus Metrics Integration

Details on metrics emitted by the Pulumi Kubernetes Operator as instructions on getting them to flow to Prometheus are available here.

Development

Check out docs/build.md for more details on building and working with the operator locally.