-
Notifications
You must be signed in to change notification settings - Fork 196
Example of yaml needed to create a new per namespace operator
(This isn't intended for use as documentation, I'm just keeping note of the YAML so I can find it later.)
Once the target-namespaces PR is landed, you will be able to deploy multiple instances of the operator watching different sets of namespaces, using different credentials to reconcile them with Azure.
This is some example YAML I've used to set up an additional operator instance (in altoperator-system). (It's pointing at a specific image in my dev container registry, so it will need to be adapted for other people - probably to use the released image.) This can be cut down further - since the webhooks won't be fired on this deployment, the webhook port and certificate mounts aren't needed, and the certificate and issuer resources can be removed.
To use:
- Create altoperator-system namespace.
- Create the azureoperatorsettings secret in altoperator-system with credentials and target namespaces.
- If you're using managed identity, create the AzureIdentity and AzureIdentityBinding resources needed.
- Apply this yaml.
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: azure-service-operator-v1
control-plane: controller-manager
name: azureoperator-controller-manager
namespace: altoperator-system
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
control-plane: controller-manager
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
aadpodidbinding: aso_manager_binding
control-plane: controller-manager
spec:
containers:
- args:
- --secure-listen-address=0.0.0.0:8443
- --upstream=http://127.0.0.1:8080/
- --logtostderr=true
- --v=10
image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
imagePullPolicy: IfNotPresent
name: kube-rbac-proxy
ports:
- containerPort: 8443
name: https
protocol: TCP
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
- args:
- --metrics-addr=127.0.0.1:8080
command:
- /manager
env:
- name: AZURE_CLIENT_ID
valueFrom:
secretKeyRef:
key: AZURE_CLIENT_ID
name: azureoperatorsettings
optional: true
- name: AZURE_CLIENT_SECRET
valueFrom:
secretKeyRef:
key: AZURE_CLIENT_SECRET
name: azureoperatorsettings
optional: true
- name: AZURE_TENANT_ID
valueFrom:
secretKeyRef:
key: AZURE_TENANT_ID
name: azureoperatorsettings
optional: true
- name: AZURE_SUBSCRIPTION_ID
valueFrom:
secretKeyRef:
key: AZURE_SUBSCRIPTION_ID
name: azureoperatorsettings
optional: true
- name: AZURE_USE_MI
valueFrom:
secretKeyRef:
key: AZURE_USE_MI
name: azureoperatorsettings
optional: true
- name: AZURE_OPERATOR_KEYVAULT
valueFrom:
secretKeyRef:
key: AZURE_OPERATOR_KEYVAULT
name: azureoperatorsettings
optional: true
- name: AZURE_CLOUD_ENV
valueFrom:
secretKeyRef:
key: AZURE_CLOUD_ENV
name: azureoperatorsettings
optional: true
- name: AZURE_SECRET_NAMING_VERSION
valueFrom:
secretKeyRef:
key: AZURE_SECRET_NAMING_VERSION
name: azureoperatorsettings
optional: true
- name: AZURE_TARGET_NAMESPACES
valueFrom:
secretKeyRef:
key: AZURE_TARGET_NAMESPACES
name: azureoperatorsettings
optional: true
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: REQUEUE_AFTER
value: "30"
image: xtianregistry.azurecr.io/candidate/k8s/azureserviceoperator:test42
imagePullPolicy: IfNotPresent
name: manager
ports:
- containerPort: 9443
name: webhook-server
protocol: TCP
- containerPort: 8081
name: liveness-port
protocol: TCP
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 200m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /tmp/k8s-webhook-server/serving-certs
name: cert
readOnly: true
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
terminationGracePeriodSeconds: 10
volumes:
- name: cert
secret:
defaultMode: 420
secretName: webhook-server-cert
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: azureoperator-serving-cert
namespace: altoperator-system
spec:
dnsNames:
- azureoperator-webhook-service.altoperator-system.svc
- azureoperator-webhook-service.altoperator-system.svc.cluster.local
issuerRef:
kind: Issuer
name: azureoperator-selfsigned-issuer
secretName: webhook-server-cert
---
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: azureoperator-selfsigned-issuer
namespace: altoperator-system
spec:
selfSigned: {}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: altoperator-manager-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: azureoperator-manager-role
subjects:
- kind: ServiceAccount
name: default
namespace: altoperator-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: altoperator-proxy-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: azureoperator-proxy-role
subjects:
- kind: ServiceAccount
name: default
namespace: altoperator-system