Caution

THIS PROJECT IS UNDER CONSTRUCTION (WIP). DO NOT TAKE IT SERIOUSLY YET!

---

📝 POC: Kubernetes Custom Admission Control for Deployments

This repository contains a well detailed PoC about an specific use case: enabling custom validation and mutating webhook for your Kubernetes deployments based on a set of rules. There are many solutions out there like OPA Gatekeeper or Kyverno specialized for this. However, understanding how could you configure your custom server for validation or mutating webhook can provide you even farther flexibility if needed.

Report Bug · Request Feature

📚 Table of contents

• 📝 POC: Kubernetes Custom Admission Control for Deployments
  • 📚 Table of contents
  • 💡 Structure
  • 🚀 Installation and Execution
    • 🔨 Prerequisites
    • 🗜️ Preparations
    • Deployment Path 1: Server outside the cluster - Docker
    • Deployment Path 2: Installing it in the kubernetes cluster - Kubernetes
      • Configuring the WebHooks
  • 📍 Roadmap
  • 📎 Contributing
  • 📃 License
  • 👥 Contact

💡 Structure

🚀 Installation and Execution

🔨 Prerequisites

• Docker
• Python 3.10+
• Kubectl
• yq (only for Kubernetes deployment).

sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
sudo chmod a+x /usr/local/bin/yq

🔝 Back to top

🗜️ Preparations

1. Build required docker images and upload to your cluster registry. Other option is to use my already publish image in DockerHub.

docker build . -f docker/Dockerfile.service -t uvicorn:1.0

2. [Optional] Generate self-signed certs for the uvicorn server. Omit this step if you have your own certificates for TLS exposure.

# Make sure to set your target domain and ip
./gen_certs.sh --domain codetriarii.org --ip 172.20.140.18

Deployment Path 1: Server outside the cluster - Docker

If you want to deploy the server outside the cluster, make sure to follow these steps:

1. [Optional] If you have your own certs, place those in certs folder. Ensure naming is ca.crt, cert.crt and cert.key.

2. Start the server with docker compose.

docker compose -f docker-compose.yml up --force-recreate

Deployment Path 2: Installing it in the kubernetes cluster - Kubernetes

Important

For kubernetes, you must generate the certs with CN expected the service name to avoid TLS errors.

./gen_certs.sh --domain uvicorn-service.demo.svc --ip 127.0.0.1

1. Create the tls secret from your certs (either the ones you have placed in certs folder or the automatically generated ones.)

kubectl create ns demo
kubectl config set-context --current --namespace demo
kubectl create secret tls uvicorn-tls-secret --cert=certs/cert.crt --key=certs/cert.key -n demo

2. Create the configmap with the rules details.

kubectl create cm uvicorn-config --from-file=uvicorn_config/config.yaml -n demo

Tip

Check the rules before creation, to enable, disable, change the parameters as you require.

3. Create the deployment.

kubectl create -f kubernetes/uvicorn-deployment.yaml

Important

If you are using your own built image, then make sure to change the image of the deployment and include imagePullSecrets if required (if your registry is authorized as it should be...)

🔝 Back to top

Configuring the WebHooks

Important

Kubernetes versions enable by default ValidatingAdmissionWebhook and MutatingAdmissionWebhook. Ensure both plugins are enabled in your cluster. If not, just include both in --enable-admission-plugins= flag of /etc/kubernetes/manifests/kube-apiserver.yaml of your master(s) node.

1. Set the ca-bundle from certs folder.

{
   ca_bundle=$(cat certs/ca.crt | base64 -w0)
   yq read kubernetes/validation-webhook.yaml -j | jq ".webhooks[].clientConfig.caBundle = \"${ca_bundle}\"" | yq read -P - > kubernetes/validation-webhook.yaml.bak
   mv kubernetes/validation-webhook.yaml.bak kubernetes/validation-webhook.yaml
}

2. Create the validation and mutating webhook definitions:

kubectl create -f kubernetes/validation-webhook.yaml
kubectl create -f kubernetes/mutating-webhook.yaml

📍 Roadmap

• Create validation/mutating server with FastAPI.
• Prepare docker and kubernetes deployment (two flavors).
• Implement rules based on config.yaml
  • Support for pods - whitelist images

See the open issues for a full list of proposed features (and known issues).

🔝 Back to top

📎 Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated 💹.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".

1. Fork the Project

2. Create your Feature Branch

   git checkout -b feature/AmazingFeature

3. Commit your Changes

   git commit -m 'Add some AmazingFeature

4. Push to the Branch

   git push origin feature/AmazingFeature

5. Open a Pull Request

🔝 Back to top

📃 License

Distributed under the APACHE 2.0 License.

🔝 Back to top

👥 Contact

---

PAF TRIARII (pedroarias1015@gmail.com) a member of Code Triarii

---

As we always state, our main purpose is keep learning, contributing to the community and finding ways to collaborate in interesting initiatives. Do not hesitate to contact us at codetriariism@gmail.com

If you are interested in our content creation, also check our social media accounts. We have all sorts of training resources, blogs, hackathons, write-ups and more! Do not skip it, you will like it 😏 😏 😏 👍

Don't forget to give the project a star if you liked it! Thanks again! 🌟 💛

🔝 Back to top