Skip to content

Simplifying K8s Cluster Deployment: Leveraging CAPI on PowerVC for Developer Friendly Solutions

Jump to bottom
Prajyot Parab edited this page May 31, 2023 · 1 revision

Prerequisites

  1. Install kubectl tool (see here)
  2. Install kind tool (see here)
  3. An appropriately configured Go development environment
  4. Install clusterctl tool (see here)
  5. Install openstack tool (see here)

PowerVC Prerequisites

  1. Create Network - A public network is required for your kubernetes cluster.
  2. Create SSH Key Pair
  3. Import the machine boot image (Download from here)
  4. Create PowerVC network port (VIP)

Note: PowerVC host group should not contain spaces or any special characters. This is because the CCM expects the host group name to be without spaces or special characters. For example, the host group in PowerVC should be used as 'DefaultGroup' instead of 'Default Group'.

Create Kind Cluster

~ kind create cluster --name capi-test

Clone Projects

~ mkdir ~/projects
~ cd ~/projects
~ git clone https://github.com/kubernetes-sigs/cluster-api.git
~ git clone https://github.com/Prajyot-Parab/cluster-api-provider-openstack.git

Create clouds.yaml configuration file

~ cd ~/projects/cluster-api-provider-openstack
~ git checkout powervc
~ cat > clouds.yaml <<EOF
clouds:
  capo-pvc:
    auth:
      auth_url: <OS_AUTH_URL>
      domain_name: <OS_PROJECT_NAME>
      password: <OS_PASSWORD>
      project_domain_name: <OS_PROJECT_DOMAIN_NAME>
      project_id: <OS_PROJECT_ID>
      project_name: <OS_PROJECT_NAME>
      user_domain_name: <OS_USER_DOMAIN_NAME>
      username: <OS_USERNAME>
    cacert: <OS_CACERT>
    region_name: <OS_REGION_NAME>
EOF

Note: Specify valid <OS_AUTH_URL>, <OS_PROJECT_NAME>, <OS_PASSWORD>, <OS_PROJECT_DOMAIN_NAME>, <OS_PROJECT_ID>, <OS_PROJECT_NAME>, <OS_USER_DOMAIN_NAME>, <OS_USERNAME>, <OS_CACERT>, <OS_REGION_NAME>

~ source ./templates/env.rc clouds.yaml capo-pvc

Run the following commands in a separate terminal

~ cd ~/projects/cluster-api
~ cat > tilt-settings.yaml <<EOF
default_registry: <IMAGE_REGISTRY>
provider_repos:
- ../cluster-api-provider-openstack
enable_providers:
- openstack
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  EXP_CLUSTER_RESOURCE_SET: "true"
extra_args:
  openstack:
    - '-v=6'
EOF

Note: Specify valid <IMAGE_REGISTRY>, e.g gcr.io/pparab

~ tilt up

Note: Check UI and wait for all controllers to be up before moving to next step.

Create a Cluster

OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR=medium \
OPENSTACK_NODE_MACHINE_FLAVOR=medium \
OPENSTACK_IMAGE_NAME=<OPENSTACK_IMAGE_NAME> \
OPENSTACK_SSH_KEY_NAME=<OPENSTACK_SSH_KEY_NAME> \
OPENSTACK_FAILURE_DOMAIN=<OPENSTACK_FAILURE_DOMAIN> \
POWERVC_VIP_CIDR=<POWERVC_VIP_CIDR> \
POWERVC_VIP=<POWERVC_VIP> \
OPENSTACK_EXTERNAL_NETWORK_ID=<OPENSTACK_EXTERNAL_NETWORK_ID> \
clusterctl generate cluster capo-pvc \
  --kubernetes-version v1.26.2 \
  --control-plane-machine-count=1 \
  --worker-machine-count=1 \
  --from ./cluster-template-powervc.yaml | kubectl apply -f -

Note: Specify valid <OPENSTACK_IMAGE_NAME>, <OPENSTACK_SSH_KEY_NAME>, <OPENSTACK_FAILURE_DOMAIN>, <POWERVC_VIP_CIDR>, <POWERVC_VIP>, <OPENSTACK_EXTERNAL_NETWORK_ID>

Example -

OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR=medium \
OPENSTACK_NODE_MACHINE_FLAVOR=medium \
OPENSTACK_IMAGE_NAME=capibm-powervs-centos-streams8-1-26-2 \
OPENSTACK_SSH_KEY_NAME=capo-pvc-key \
OPENSTACK_FAILURE_DOMAIN=Default \
POWERVC_VIP_CIDR=9.47.88.0/24 \
POWERVC_VIP=9.47.88.220 \
OPENSTACK_EXTERNAL_NETWORK_ID=fc0dbf60-c5bd-4ec2-804f-f7f12d851f3f \
clusterctl generate cluster capo-pvc \
  --kubernetes-version v1.26.2 \
  --control-plane-machine-count=1 \
  --worker-machine-count=1 \
  --from ./cluster-template-powervc.yaml | kubectl apply -f -

Output:

secret/capo-pvc-cloud-config created
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/capo-pvc-md-0 created
cluster.cluster.x-k8s.io/capo-pvc created
machinedeployment.cluster.x-k8s.io/capo-pvc-md-0 created
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/capo-pvc-control-plane created
openstackcluster.infrastructure.cluster.x-k8s.io/capo-pvc created
openstackmachinetemplate.infrastructure.cluster.x-k8s.io/capo-pvc-control-plane created
openstackmachinetemplate.infrastructure.cluster.x-k8s.io/capo-pvc-md-0 created
clusterresourceset.addons.cluster.x-k8s.io/crs-cloud-conf created
secret/powervc-cacert-credential created
secret/powervc-cloud-credential created
configmap/cloud-controller-manager-addon created

Check the state of the provisioned cluster and machine objects within the local management cluster

Clusters

% kubectl get cluster
NAME       PHASE         AGE   VERSION
capo-pvc   Provisioned   10m

Kubeadm Control Plane

% kubectl get kubeadmcontrolplane
NAME                     CLUSTER    INITIALIZED   API SERVER AVAILABLE   REPLICAS   READY   UPDATED   UNAVAILABLE   AGE   VERSION
capo-pvc-control-plane   capo-pvc   true                                 1                  1         1             13m   v1.26.2

Machines

% kubectl get machines           
NAME                                   CLUSTER    NODENAME                       PROVIDERID                                          PHASE     AGE   VERSION
capo-pvc-control-plane-9gb8l           capo-pvc   capo-pvc-control-plane-2m8qs   openstack:///d3cdb4b0-40ed-4450-b79c-9909ab1f469b   Running   52m   v1.26.2
capo-pvc-md-0-7b4888c4f7xgn94p-kvn29   capo-pvc   capo-pvc-md-0-b427q            openstack:///19d286ab-3880-47a9-a850-cafd730007d5   Running   52m   v1.26.2

Deploy Container Network Interface (CNI)

% clusterctl get kubeconfig capo-pvc > ~/.kube/capo-pvc 
% export KUBECONFIG=~/.kube/capo-pvc 
% kubectl apply -f https://docs.projectcalico.org/v3.15/manifests/calico.yaml

Check the state of the newly provisioned cluster within PowerVC

% kubectl get nodes
NAME                           STATUS   ROLES           AGE   VERSION
capo-pvc-control-plane-2m8qs   Ready    control-plane   49m   v1.26.2
capo-pvc-md-0-b427q            Ready    <none>          42m   v1.26.2
Clone this wiki locally