Skip to content

Example of yaml needed to create a new per namespace operator

Christian Muirhead edited this page Jun 15, 2021 · 4 revisions

(This isn't intended for use as documentation, I'm just keeping note of the YAML so I can find it later.)

Once the target-namespaces PR is landed, you will be able to deploy multiple instances of the operator watching different sets of namespaces, using different credentials to reconcile them with Azure.

This is some example YAML I've used to set up an additional operator instance (in altoperator-system). (It's pointing at a specific image in my dev container registry, so it will need to be adapted for other people - probably to use the released image.) This can be cut down further - since the webhooks won't be fired on this deployment, the webhook port and certificate mounts aren't needed, and the certificate and issuer resources can be removed.

To use:

  1. Create altoperator-system namespace.
  2. Create the azureoperatorsettings secret in altoperator-system with credentials and target namespaces.
  3. If you're using managed identity, create the AzureIdentity and AzureIdentityBinding resources needed.
  4. Apply this yaml.
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: azure-service-operator-v1
    control-plane: controller-manager
  name: azureoperator-controller-manager
  namespace: altoperator-system
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      control-plane: controller-manager
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        aadpodidbinding: aso_manager_binding
        control-plane: controller-manager
    spec:
      containers:
      - args:
        - --secure-listen-address=0.0.0.0:8443
        - --upstream=http://127.0.0.1:8080/
        - --logtostderr=true
        - --v=10
        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
        imagePullPolicy: IfNotPresent
        name: kube-rbac-proxy
        ports:
        - containerPort: 8443
          name: https
          protocol: TCP
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - args:
        - --metrics-addr=127.0.0.1:8080
        command:
        - /manager
        env:
        - name: AZURE_CLIENT_ID
          valueFrom:
            secretKeyRef:
              key: AZURE_CLIENT_ID
              name: azureoperatorsettings
              optional: true
        - name: AZURE_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              key: AZURE_CLIENT_SECRET
              name: azureoperatorsettings
              optional: true
        - name: AZURE_TENANT_ID
          valueFrom:
            secretKeyRef:
              key: AZURE_TENANT_ID
              name: azureoperatorsettings
              optional: true
        - name: AZURE_SUBSCRIPTION_ID
          valueFrom:
            secretKeyRef:
              key: AZURE_SUBSCRIPTION_ID
              name: azureoperatorsettings
              optional: true
        - name: AZURE_USE_MI
          valueFrom:
            secretKeyRef:
              key: AZURE_USE_MI
              name: azureoperatorsettings
              optional: true
        - name: AZURE_OPERATOR_KEYVAULT
          valueFrom:
            secretKeyRef:
              key: AZURE_OPERATOR_KEYVAULT
              name: azureoperatorsettings
              optional: true
        - name: AZURE_CLOUD_ENV
          valueFrom:
            secretKeyRef:
              key: AZURE_CLOUD_ENV
              name: azureoperatorsettings
              optional: true
        - name: AZURE_SECRET_NAMING_VERSION
          valueFrom:
            secretKeyRef:
              key: AZURE_SECRET_NAMING_VERSION
              name: azureoperatorsettings
              optional: true
        - name: AZURE_TARGET_NAMESPACES
          valueFrom:
            secretKeyRef:
              key: AZURE_TARGET_NAMESPACES
              name: azureoperatorsettings
              optional: true
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: REQUEUE_AFTER
          value: "30"
        image: xtianregistry.azurecr.io/candidate/k8s/azureserviceoperator:test42
        imagePullPolicy: IfNotPresent
        name: manager
        ports:
        - containerPort: 9443
          name: webhook-server
          protocol: TCP
        - containerPort: 8081
          name: liveness-port
          protocol: TCP
        resources:
          limits:
            cpu: 500m
            memory: 512Mi
          requests:
            cpu: 200m
            memory: 256Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /tmp/k8s-webhook-server/serving-certs
          name: cert
          readOnly: true
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      terminationGracePeriodSeconds: 10
      volumes:
      - name: cert
        secret:
          defaultMode: 420
          secretName: webhook-server-cert
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: azureoperator-serving-cert
  namespace: altoperator-system
spec:
  dnsNames:
  - azureoperator-webhook-service.altoperator-system.svc
  - azureoperator-webhook-service.altoperator-system.svc.cluster.local
  issuerRef:
    kind: Issuer
    name: azureoperator-selfsigned-issuer
  secretName: webhook-server-cert
---
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: azureoperator-selfsigned-issuer
  namespace: altoperator-system
spec:
  selfSigned: {}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: altoperator-manager-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: azureoperator-manager-role
subjects:
- kind: ServiceAccount
  name: default
  namespace: altoperator-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: altoperator-proxy-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: azureoperator-proxy-role
subjects:
- kind: ServiceAccount
  name: default
  namespace: altoperator-system
Clone this wiki locally