Dual Stack

How to create a K8s Dual Stack cluster with KIND

For the Impatient

1. Download the KIND binary corresponding to your OS

wget https://github.com/aojea/kind/releases/download/dualstack/kind.linux
mv kind.linux kind
chmod +x kind

2. Create your KIND configuration and store it in a file, i.e. dual-stack.yaml:

# a dual-stack cluster
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  disableDefaultCNI: true
  ipFamily: DualStack
nodes:
- role: control-plane
- role: worker
- role: worker

3. Create your cluster (KIND default CNI plugin does not support dual stack):

./kind create cluster --config dual-stack.yaml

4. Install a CNI with dual stack support, :

kubectl apply -f https://raw.githubusercontent.com/aojea/kindnet/master/install-kindnet.yaml

For the Curious

1. Create a KIND binary with dual-stack support:

# clone the fork
git clone git@github.com:aojea/kind.git
# cd into the repository
cd kind
# switch to the dualstack branch
git checkout dualstack
# and build it
go build

2. Create your node image with dual stack support (https://kind.sigs.k8s.io/docs/user/quick-start/#building-images)

./kind build node-image --image kindest/kindnode:dualstack 

3. Create the cluster using the node-image that you just built with the following configuration:

# a dual-stack cluster
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  ipFamily: DualStack
nodes:
- role: control-plane
- role: worker
- role: worker

./kind create cluster --image kindest/kindnode:dualstack --config config-kind.yaml -v3

Check your cluster is really working in Dual Stack

1. Check that pods have both IPv4 and IPv6 addresses:

kubectl run -i --tty busybox --image=busybox -- ip a
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
If you don't see a command prompt, try pressing enter.
Error attaching, falling back to logs: unable to upgrade connection: container busybox not found in pod busybox-86c4cfd46-hbd44_default
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 5e:20:cb:df:76:0e brd ff:ff:ff:ff:ff:ff
    inet 10.244.2.3/24 brd 10.244.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fd00:10:244:0:2::3/80 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::5c20:cbff:fedf:760e/64 scope link
       valid_lft forever preferred_lft forever

2. If everything went well, you can create a deployment:

kubectl apply -f https://k8s.io/examples/application/deployment.yaml

• We configured the cluster to use IPv4 as the default family, so we can create an IPv4 service with

kubectl expose deployment.apps/nginx-deployment

3. We can create an IPv6 service using the following yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    run: nginx
spec:
  ipFamily: IPv6
  ports:
  - port: 80
    protocol: TCP
  selector:
    run: nginx

4. Check services are created:

kubectl get svc
NAME               TYPE        CLUSTER-IP         EXTERNAL-IP   PORT(S)   AGE
kubernetes         ClusterIP   10.96.0.1          <none>        443/TCP   74m
nginx              ClusterIP   fd00:10:96::3cd7   <none>        80/TCP    7s
nginx-deployment   ClusterIP   10.107.244.181     <none>        80/TCP    27m

Kindnet IPv6

This release allows creating IPv6 Kubernetes clusters with kind, it adds a new option ipFamily to the v1alpha3 API that allows choosing the IP family of the cluster.

To avoid issues with the different networking options the podSubnet and the serviceSubnet kubeadm values are predefined with the following values:

	DefaultPodSubnet          = "10.244.0.0/16"
	DefaultServicesSubnet     = "10.96.0.0/12"
	DefaultPodSubnetIPv6      = "fd00:100:64::/64"
	DefaultServicesSubnetIPv6 = "fd00:10:96::/112"

Kind uses Calico as CNI network plugin

We can create a Kubernetes IPv6 cluster with the following config:

# necessary for conformance
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
  ipFamily: ipv6
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker

Calico IPv6

This release allows creating IPv6 Kubernetes clusters with kind, it adds a new option ipFamily to the v1alpha3 API that allows choosing the IP family of the cluster.

To avoid issues with the different networking options the podSubnet and the serviceSubnet kubeadm values are predefined with the following values:

	DefaultPodSubnet          = "100.64.0.0/12"
	DefaultServicesSubnet     = "10.96.0.0/12"
	DefaultPodSubnetIPv6      = "fd00:100:64::/64"
	DefaultServicesSubnetIPv6 = "fd00:10:96::/112"

Kind uses Calico as CNI network plugin

We can create a Kubernetes IPv6 cluster with the following config:

# necessary for conformance
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
  ipFamily: ipv6
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker

ipv6 and multiple cni support

It adds a new option ipFamily to the v1alpha3 API that allows to choose the IP family of the cluster.

To avoid issues with the different networking options the podSubnet and the serviceSubnet options are predefined with the following values:

	DefaultPodSubnet          = "100.64.0.0/12"
	DefaultServicesSubnet     = "10.96.0.0/12"
	DefaultPodSubnetIPv6      = "fd00:100:64::/64"
	DefaultServicesSubnetIPv6 = "fd00:10:96::/112"

Because the CNI plugins require different configurations depending on the IP family we want to use in our cluster, this PR adds an option to select the CNI plugin and flavor when building the node image.

Currently, there are only 3 CNI plugins implemented in kind: weave, calico and calico-ipv6

weave continuous being the default CNI plugin, if we want to choose a new one we have to create a new node-image:

kind build node-image --type apt --cni calico-ipv6 --image kindest/node:calico-ipv6

Using the calico-ipv6 CNI plugin we can create a Kubernetes IPv6 cluster with the following config:

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
  ipFamily: ipv6
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker

and creating a cluster with the new node image and the previous configuration

kind create cluster --image kindest/node:calico-ipv6 --config test/config-ipv6-ci.yaml --loglevel debug --wait 1m

v.0.1-beta

Add cli option to choose CNI