Add script to quickly deploy kind cluster with custom KNP image #631
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
|
Hi @carreter. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
needs-ok-to-test
|
@avrittrohwer for visibility :) |
k8s-ci-robot
added
the
size/L
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
|
/retest |
| - operator: "Exists" | ||
| effect: "NoExecute" | ||
| nodeSelector: | ||
| kubernetes.io/os: linux |
There was a problem hiding this comment.
Could add node-role.kubernetes.io/worker: '' to prevent agents on control plane nodes?
There was a problem hiding this comment.
Actually, seems like setting this label on the node in the kind.config breaks the cluster for some reason. kind doesn't label the worker nodes as such by default for some reason, and kubelet doesn't start if I provide a custom role label. Leaving this out for now.
There was a problem hiding this comment.
Did some more testing and opened an upstream issue: kubernetes-sigs/kind#3657
There was a problem hiding this comment.
kubelet doesn't permit adding arbitrary lables in the k8s.io / kubernetes.io namespace because those are reserved for Kubernetes and need to be approved by the API review team.
You can instead use taints and tolerations to schedule only to worker nodes.
Kind removes the control plane taint only if there is a single control plane node and no other nodes (so a one node cluster), otherwise the control plane node(s) are tainted, see:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#control-plane-node-isolation
There was a problem hiding this comment.
@jkh52 Is it ok if I leave this as-is? I haven't noticed proxy agents getting spun up on the control plane anyway.
There was a problem hiding this comment.
No objections. Are taints + tolerations the reason that you don't see agents on control plane? Did you try a one-node cluster as well?
There was a problem hiding this comment.
On a one-node cluster, that node is the worker and the control plane host, so scheduling user workloads and control plane workloads would be expected, though I'm not sure it makes sense to even try deploying this application to a one node cluster ..?
On a multi-node cluster there's the well-known kubeadm taint on control plane nodes, unless you tolerate all taints or that taint specifically your pod won't schedule there. It's pretty typical that a cluster with registered control plane nodes would have a similar taint if not the same taint.
|
/lgtm |
k8s-ci-robot
added
the
lgtm
examples/kind-multinode-kcp/templates/k8s/konnectivity-agent-ds.yaml
Outdated
Show resolved
Hide resolved
k8s-ci-robot
removed
the
lgtm
|
Apologies for force-pushing and messing up the commit chain, I borked my local git repo. Addressed review comments and made it so the script also sets the appropriate |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: carreter, cheftako The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
Thought I'd share a script I wrote while setting up my local dev environment.
./quickstart-kind.shlets you set how many KCP + worker nodes the cluster has, modify which image is pulled for both the proxy server and agent, and optionally sideload the images into the cluster to avoid the need for a publishing the image on a public or local registry.