In order to deploy the application with Kubernetes you will use the following resources:
- Deployments - replicates our application across our kubernetes nodes and allows us to do a controlled rolling update of our software across the fleet of application instances
- Services - load balancing and service discovery for our internal services
- Ingress - external load balancing and SSL termination for our external service
- Secrets - secure storage of non public configuration information, SSL certs specifically in our case
In order to accomplish this goal you will use the following Jenkins plugins:
- Jenkins Kubernetes Plugin - start Jenkins build executor containers in the Kubernetes cluster when builds are requested, terminate those containers when builds complete, freeing resources up for the rest of the cluster
- Jenkins Pipelines - define our build pipeline declaratively and keep it checked into source code management alongside our application code
- Google Oauth Plugin - allows you to add your google oauth credentials to jenkins
Create the jenkins
namespace:
$ kubectl create ns jenkins
In order to pre-populate Jenkins with the necessary plugins and configuration for the rest of the tutorial, you will create a volume from an existing tarball of that data.
gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v2.tar.gz
gcloud compute disks create jenkins-home --image jenkins-home-image
Here you'll create a Deployment running a Jenkins container with a persistent disk attached containing the Jenkins home directory.
First, set the password for the default Jenkins user. Edit the password in jenkins/k8s/options
with the password of your choice by replacing CHANGE_ME. To Generate a random password and replace it in the file, you can run:
$ PASSWORD=`openssl rand -base64 15`; echo "Your password is $PASSWORD"; sed -i.bak s#CHANGE_ME#$PASSWORD# jenkins/k8s/options
Your password is 2UyiEo2ezG/CKnUcgPxt
Now create the secret using kubectl
:
$ kubectl create secret generic jenkins --from-file=jenkins/k8s/options --namespace=jenkins
secret "jenkins" created
Additionally you will have a service that will route requests to the controller.
Note: All of the files that define the Kubernetes resources you will be creating for Jenkins are in the
jenkins/k8s
folder. You are encouraged to take a look at them before running the create commands.
The Jenkins Deployment is defined in kubernetes/jenkins.yaml
. Create the Deployment and confirm the pod was scheduled:
$ kubectl apply -f jenkins/k8s/
deployment "jenkins" created
service "jenkins-ui" created
service "jenkins-discovery" created
Check that your master pod is in the running state
$ kubectl get pods --namespace jenkins
NAME READY STATUS RESTARTS AGE
jenkins-master-to8xg 1/1 Running 0 30s
Now, check that the Jenkins Service was created properly:
$ kubectl get svc --namespace jenkins
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-discovery 10.79.254.142 <none> 50000/TCP 10m
jenkins-ui 10.79.242.143 nodes 8080/TCP 10m
We are using the Kubernetes Plugin so that our builder nodes will be automatically launched as necessary when the Jenkins master requests them. Upon completion of their work they will automatically be turned down and their resources added back to the clusters resource pool.
Notice that this service exposes ports 8080
and 50000
for any pods that match the selector
. This will expose the Jenkins web UI and builder/agent registration ports within the Kubernetes cluster.
Additionally the jenkins-ui
services is exposed using a NodePort so that our HTTP loadbalancer can reach it.
Kubernetes makes it simple to deploy an Ingress resource to act as a public load balancer and SSL terminator.
The Ingress resource is defined in jenkins/k8s/lb/ingress.yaml
. We used the Kubernetes secrets
API to add our certs securely to our cluster and ready for the Ingress to use.
In order to create your own certs run:
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/CN=jenkins/O=jenkins"
Now you can upload them to Kubernetes as secrets:
$ kubectl create secret generic tls --from-file=/tmp/tls.crt --from-file=/tmp/tls.key --namespace jenkins
Now that the secrets have been uploaded, create the ingress load balancer. Note that the secrets must be created before the ingress, otherwise the HTTPs endpoint will not be created.
$ kubectl apply -f jenkins/k8s/lb