Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New docker images version, errors #8

Open
blabla1337 opened this issue Jun 4, 2020 · 18 comments
Open

New docker images version, errors #8

blabla1337 opened this issue Jun 4, 2020 · 18 comments

Comments

@blabla1337
Copy link
Contributor

In the last commit we changed the below versions for the docker installation

-docker_version="5:18.09.0~3-0~raspbian-stretch"
+docker_version="5:19.03.9~3-0~raspbian-stretch"

I have now tried it in combination of the project and it's now breaking the installation. I needed to reinstall all the raspberry PI's and used the project again.

I have retried it a couple of times with different issues but one that was interesting is that the Nodes overview only contained 1 Master Type node and the other Master nodes didn't have a type.

So I suggest to revert the change and add the export var again for the docker API so at least its working. Then from there we need to debug and see why its not working with the latest Docker version 5:19 but I think you have to do this as I'm not cable of understanding what's going wrong.

@lucasteligioridis
Copy link
Owner

Can you provide some logs or insights as to what was breaking?

@blabla1337
Copy link
Contributor Author

for example I got this:
root@rpi-kube-master-02:/home/pi# kubectl get nodes
NAME STATUS ROLES AGE VERSION
rpi-kube-master-02 Ready master 44m v1.16.1
rpi-kube-master-03 Ready 40m v1.16.1
rpi-kube-worker-01 Ready 44m v1.16.1
rpi-kube-worker-02 Ready 43m v1.16.1
rpi-kube-worker-03 Ready 43m v1.16.1

and Master 1 gave an error installing it, the installation of the workers went ok in all cases but the master PI's always gave errors but where also different every time I tried. Also now when I try to access them from my machine I get the message:

gibson$ kubectl get nodes
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")

@blabla1337
Copy link
Contributor Author

Hmm tried the old one now but its also giving me errors...
I had to reboot the stack and its hanging on this now:
Master 1 + Master 2

kubernetes-bootstrap: Installing haproxy 1.8.19-1+rpi1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: haproxy is already the newest version (1.8.19-1+rpi1).
kubernetes-bootstrap: 0 upgraded, 0 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: haproxy was already set on hold.
kubernetes-bootstrap: haproxy.service is not active, cannot reload.
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2

Interesting is that Master VIP is marked as Master 1 but this one is hanging, see above.
so running something like this doesn't work:

pi@rpi-kube-master-03:~ $ kubectl get nodes
The connection to the server 192.168.0.150:8443 was refused - did you specify the right host or port?

@blabla1337
Copy link
Contributor Author

Maybe if you are having some time, are you on a chatting platform like Gitter, Hangouts?
Maybe we can troubleshoot it ?

@lucasteligioridis
Copy link
Owner

In hindsight I should have referred to this: https://v1-16.docs.kubernetes.io/docs/setup/production-environment/container-runtimes/

Was that first certificate error because of the old kubernetes configuration file from the previous cluster?

@blabla1337
Copy link
Contributor Author

What I did was reinstall all the SD cards using the make build and modified the parameters according for worker and master. I kept the old SSH file and img file that was in the output file as these are working and didn't need to recreate those. I also can use the ssh to login in the machine so that's all fine.

Was that first certificate error because of the old kubernetes configuration file from the previous cluster?

Yes indeed, I needed to grab the new one from the cluster and replace it.

What I will do is try again with the new repo and report back with the logs and all.

@lucasteligioridis
Copy link
Owner

Please.
Every bit of detail helps, just wonder whether docker and kubernetes versions are having compatibility issues.

Sorry I don’t have a huge amount of time to troubleshoot heaps now, quite busy at work.

Can jump in here and respond every now and again.
I might fire up my cluster soon and attempt all this again. Probably be a week before I can get anytime though.

@blabla1337
Copy link
Contributor Author

blabla1337 commented Jun 4, 2020

So I have some logs, the workers seem to work well and installation goes smooth but the problems are with the masters.

Output Master1
Everything before the log below is not giving any form of errors but from here its getting interesting. Also after this it hanged and nothing happens, the script never finishing the installation.

kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: Did not win VIP election, joining existing cluster.
kubernetes-bootstrap: The connection to the server localhost:8080 was refused - did you specify the right host or port?
kubernetes-bootstrap: Checking to see if control-plane has been fully initialised...
kubernetes-bootstrap: This could take up to 10 minutes...
kubernetes-bootstrap: [0] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [1] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: [2] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [3] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [4] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [5] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [6] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [7] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [8] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [9] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [10] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [11] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [12] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: Kubernetes control-plane has finished initalising!
kubernetes-bootstrap: Attempting to join cluster!
kubernetes-bootstrap: I0604 21:02:17.715905    8682 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [upload-certs] Using certificate key:
kubernetes-bootstrap: 730b84a2a163f697aa45d053be4e05aae6ff500caa0f95ce01d5431373b677d0
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: #011[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.9. Latest validated version: 18.09
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [preflight] Running pre-flight checks before initializing the new control plane instance
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-01 localhost] and IPs [192.168.0.151 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-01 localhost] and IPs [192.168.0.151 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.151 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Using the existing "sa" key
kubernetes-bootstrap: [kubeconfig] Generating kubeconfig files
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [check-etcd] Checking that the etcd cluster is healthy
kubernetes-bootstrap: [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
kubernetes-bootstrap: [etcd] Announced new etcd member joining to the existing etcd cluster
kubernetes-bootstrap: [etcd] Creating static Pod manifest for "etcd"
kubernetes-bootstrap: [etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.

Master2 went well and finished installation:

kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: 192.168.0.152 192.168.0.150 172.17.0.1 fd20:da22:881e:6300:7285:f509:850b:5f69 2a00:1cf8:1200:2a5f:1e47:523f:478b:dd82 
kubernetes-bootstrap: VIP currently resides on local host and no cluster exists.
kubernetes-bootstrap: Initialising as new Kubernetes cluster
kubernetes-bootstrap: I0604 20:58:26.540158    5370 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [init] Using Kubernetes version: v1.16.10
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: #011[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.9. Latest validated version: 18.09
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "ca" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-02 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.152 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-ca" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/ca" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-02 localhost] and IPs [192.168.0.152 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-02 localhost] and IPs [192.168.0.152 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "sa" key and public key
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "kubelet.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
kubernetes-bootstrap: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.
kubernetes-bootstrap: [apiclient] All control plane components are healthy after 48.514923 seconds
kubernetes-bootstrap: [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
kubernetes-bootstrap: [kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster
kubernetes-bootstrap: [upload-certs] Skipping phase. Please see --upload-certs
kubernetes-bootstrap: [mark-control-plane] Marking the node rpi-kube-master-02 as control-plane by adding the label "node-role.kubernetes.io/master=''"
kubernetes-bootstrap: [mark-control-plane] Marking the node rpi-kube-master-02 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
kubernetes-bootstrap: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
kubernetes-bootstrap: [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
kubernetes-bootstrap: [addons] Applied essential addon: CoreDNS
kubernetes-bootstrap: [addons] Applied essential addon: kube-proxy
kubernetes-bootstrap: 
kubernetes-bootstrap: Your Kubernetes control-plane has initialized successfully!
kubernetes-bootstrap: 
kubernetes-bootstrap: To start using your cluster, you need to run the following as a regular user:
kubernetes-bootstrap: 
kubernetes-bootstrap:   mkdir -p $HOME/.kube
kubernetes-bootstrap:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
kubernetes-bootstrap:   sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubernetes-bootstrap: 
kubernetes-bootstrap: You should now deploy a pod network to the cluster.
kubernetes-bootstrap: Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
kubernetes-bootstrap:   https://kubernetes.io/docs/concepts/cluster-administration/addons/
kubernetes-bootstrap: 
kubernetes-bootstrap: You can now join any number of control-plane nodes by copying certificate authorities 
kubernetes-bootstrap: and service account keys on each node and then running the following as root:
kubernetes-bootstrap: 
kubernetes-bootstrap:   kubeadm join 192.168.0.150:6443 --token <value withheld> \
kubernetes-bootstrap:     --discovery-token-ca-cert-hash sha256:c61a0412edb77bf8952ee0935d149ab81803aa138df70d8b3cae1af6a22592a7 \
kubernetes-bootstrap:     --control-plane --certificate-key <value withheld>#011  
kubernetes-bootstrap: 
kubernetes-bootstrap: Then you can join any number of worker nodes by running the following on each as root:
kubernetes-bootstrap: 
kubernetes-bootstrap: kubeadm join 192.168.0.150:6443 --token <value withheld> \
kubernetes-bootstrap:     --discovery-token-ca-cert-hash sha256:c61a0412edb77bf8952ee0935d149ab81803aa138df70d8b3cae1af6a22592a7 
kubernetes-bootstrap: podsecuritypolicy.policy/psp.flannel.unprivileged created
kubernetes-bootstrap: clusterrole.rbac.authorization.k8s.io/flannel created
kubernetes-bootstrap: clusterrolebinding.rbac.authorization.k8s.io/flannel created
kubernetes-bootstrap: serviceaccount/flannel created
kubernetes-bootstrap: configmap/kube-flannel-cfg created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-amd64 created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-arm64 created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-arm created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-ppc64le created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-s390x created
kubernetes-bootstrap: Finished booting! Kubernetes successfully running!

Master3 output:

kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.16.10
kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: Did not win VIP election, joining existing cluster.
kubernetes-bootstrap: The connection to the server localhost:8080 was refused - did you specify the right host or port?
kubernetes-bootstrap: Checking to see if control-plane has been fully initialised...
kubernetes-bootstrap: This could take up to 10 minutes...
kubernetes-bootstrap: [0] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [1] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [2] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [3] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: [4] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [5] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [6] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [7] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [8] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [9] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [10] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [11] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [12] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [13] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [14] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: [15] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: Kubernetes control-plane has finished initalising!
kubernetes-bootstrap: Attempting to join cluster!
kubernetes-bootstrap: I0604 21:02:17.021073    8618 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [upload-certs] Using certificate key:
kubernetes-bootstrap: bce0c91ab61208b3eea98ecc8a45e2abe84a9fcb93dce232bd22b6914b906338
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: #011[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.9. Latest validated version: 18.09
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [preflight] Running pre-flight checks before initializing the new control plane instance
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-03 localhost] and IPs [192.168.0.153 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-03 localhost] and IPs [192.168.0.153 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-03 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.153 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Using the existing "sa" key
kubernetes-bootstrap: [kubeconfig] Generating kubeconfig files
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [check-etcd] Checking that the etcd cluster is healthy
kubernetes-bootstrap: [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
kubernetes-bootstrap: [etcd] Announced new etcd member joining to the existing etcd cluster
kubernetes-bootstrap: [etcd] Creating static Pod manifest for "etcd"
kubernetes-bootstrap: [etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.
kubernetes-bootstrap: {"level":"warn","ts":"2020-06-04T21:03:26.944+1000","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"passthrough:///https://192.168.0.153:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}

Hope you see what could go wrong and what to try?
Could it be related to this:
docker/cli#2533

But its not clear for me from the logs what is exactly failing of the docker commands...
Thanks for the help

@lucasteligioridis
Copy link
Owner

Couple of things that stand out to me.

I’m not sure what remote this is referring to?

kubernetes-bootstrap: I0604 20:58:26.540158    5370 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16

And this:

#011[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.9. Latest validated version: 18.09

So if you revert the docker version back to 18.09, what happens do you get consistent results?

Does it always have two successful masters and one failed one?

@blabla1337
Copy link
Contributor Author

Hmm it looks the remote version issue is regarding a kubeadm issue
With the new case I only have 1 completed Master the other 2 fails.

I will now give a try with the previous docker-ce 18.09 version that also uses the export fix we did

@blabla1337
Copy link
Contributor Author

So I tried to old one and still doesn't work, only 1 master is installing correctly in this case Master3
Master1 won the VIP according to the config but it failed to complete the installation and also Master2 failed the installation.

Master3 output

kubernetes-bootstrap: Setting hostname to: rpi-kube-master-03
kubernetes-bootstrap: Waiting for system to boot before attempting to install packages...
kubernetes-bootstrap: Updating core packages...
kubernetes-bootstrap: Get:1 http://archive.raspberrypi.org/debian buster InRelease [32.6 kB]
kubernetes-bootstrap: Get:2 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
kubernetes-bootstrap: Get:3 http://archive.raspberrypi.org/debian buster/main armhf Packages [330 kB]
kubernetes-bootstrap: Get:4 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
kubernetes-bootstrap: Get:5 http://raspbian.raspberrypi.org/raspbian buster/contrib armhf Packages [58.7 kB]
kubernetes-bootstrap: Get:6 http://raspbian.raspberrypi.org/raspbian buster/non-free armhf Packages [103 kB]
kubernetes-bootstrap: Fetched 13.5 MB in 10s (1,370 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Installing base packages...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git-man libcurl3-gnutls
kubernetes-bootstrap:   liberror-perl libgirepository-1.0-1 libjq1 libonig5 libpackagekit-glib2-18
kubernetes-bootstrap:   python3-dbus python3-gi python3-pycurl python3-software-properties
kubernetes-bootstrap:   vim-runtime
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk
kubernetes-bootstrap:   gitweb git-cvs git-mediawiki git-svn python-dbus-doc python3-dbus-dbg
kubernetes-bootstrap:   libcurl4-gnutls-dev python-pycurl-doc python3-pycurl-dbg ctags vim-doc
kubernetes-bootstrap:   vim-scripts
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   packagekit unattended-upgrades
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   apt-transport-https gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git git-man jq
kubernetes-bootstrap:   libcurl3-gnutls liberror-perl libgirepository-1.0-1 libjq1 libonig5
kubernetes-bootstrap:   libpackagekit-glib2-18 python3-dbus python3-gi python3-pycurl
kubernetes-bootstrap:   python3-software-properties software-properties-common vim vim-runtime zip
kubernetes-bootstrap: 0 upgraded, 20 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 14.4 MB of archives.
kubernetes-bootstrap: After this operation, 70.6 MB of additional disk space will be used.
kubernetes-bootstrap: Get:2 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf libgirepository-1.0-1 armhf 1.58.3-2 [78.8 kB]
kubernetes-bootstrap: Get:1 http://mirror.as43289.net/raspbian/raspbian buster/main armhf apt-transport-https all 1.8.2.1 [149 kB]
kubernetes-bootstrap: Get:3 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf gir1.2-glib-2.0 armhf 1.58.3-2 [143 kB]
kubernetes-bootstrap: Get:4 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libpackagekit-glib2-18 armhf 1.1.12-5 [98.1 kB]
kubernetes-bootstrap: Get:7 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf liberror-perl all 0.17027-2 [30.9 kB]
kubernetes-bootstrap: Get:8 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf git-man all 1:2.20.1-2+deb10u3 [1,620 kB]
kubernetes-bootstrap: Get:6 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libcurl3-gnutls armhf 7.64.0-4+deb10u1 [292 kB]
kubernetes-bootstrap: Get:10 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf libonig5 armhf 6.9.1-1 [150 kB]
kubernetes-bootstrap: Get:9 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf git armhf 1:2.20.1-2+deb10u3 [4,194 kB]
kubernetes-bootstrap: Get:11 http://mirror.as43289.net/raspbian/raspbian buster/main armhf libjq1 armhf 1.5+dfsg-2+b1 [119 kB]
kubernetes-bootstrap: Get:12 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf jq armhf 1.5+dfsg-2+b1 [59.3 kB]
kubernetes-bootstrap: Get:14 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf python3-gi armhf 3.30.4-1 [161 kB]
kubernetes-bootstrap: Get:13 http://mirror.nl.leaseweb.net/raspbian/raspbian buster/main armhf python3-dbus armhf 1.2.8-3 [95.9 kB]
kubernetes-bootstrap: Get:5 http://mirror.ehv.weppel.nl/raspbian/raspbian buster/main armhf gir1.2-packagekitglib-1.0 armhf 1.1.12-5 [35.9 kB]
kubernetes-bootstrap: Get:16 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf python3-software-properties all 0.96.20.2-2 [49.6 kB]
kubernetes-bootstrap: Get:15 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf python3-pycurl armhf 7.43.0.2-0.1 [62.8 kB]
kubernetes-bootstrap: Get:17 http://mirror.proserve.nl/raspbian/raspbian buster/main armhf software-properties-common all 0.96.20.2-2 [83.3 kB]
kubernetes-bootstrap: Get:19 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf vim armhf 2:8.1.0875-5 [1,005 kB]
kubernetes-bootstrap: Get:20 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf zip armhf 3.0-11 [212 kB]
kubernetes-bootstrap: Get:18 http://gemmei.ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf vim-runtime all 2:8.1.0875-5 [5,775 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 14.4 MB in 6s (2,587 kB/s)
kubernetes-bootstrap: Selecting previously unselected package apt-transport-https.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 39798 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../00-apt-transport-https_1.8.2.1_all.deb ...#015
kubernetes-bootstrap: Unpacking apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libgirepository-1.0-1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../01-libgirepository-1.0-1_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-glib-2.0:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../02-gir1.2-glib-2.0_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package libpackagekit-glib2-18:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../03-libpackagekit-glib2-18_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-packagekitglib-1.0.#015
kubernetes-bootstrap: Preparing to unpack .../04-gir1.2-packagekitglib-1.0_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libcurl3-gnutls:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../05-libcurl3-gnutls_7.64.0-4+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package liberror-perl.#015
kubernetes-bootstrap: Preparing to unpack .../06-liberror-perl_0.17027-2_all.deb ...#015
kubernetes-bootstrap: Unpacking liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package git-man.#015
kubernetes-bootstrap: Preparing to unpack .../07-git-man_1%3a2.20.1-2+deb10u3_all.deb ...#015
kubernetes-bootstrap: Unpacking git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package git.#015
kubernetes-bootstrap: Preparing to unpack .../08-git_1%3a2.20.1-2+deb10u3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libonig5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../09-libonig5_6.9.1-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libjq1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../10-libjq1_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package jq.#015
kubernetes-bootstrap: Preparing to unpack .../11-jq_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-dbus.#015
kubernetes-bootstrap: Preparing to unpack .../12-python3-dbus_1.2.8-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-gi.#015
kubernetes-bootstrap: Preparing to unpack .../13-python3-gi_3.30.4-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-pycurl.#015
kubernetes-bootstrap: Preparing to unpack .../14-python3-pycurl_7.43.0.2-0.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-software-properties.#015
kubernetes-bootstrap: Preparing to unpack .../15-python3-software-properties_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package software-properties-common.#015
kubernetes-bootstrap: Preparing to unpack .../16-software-properties-common_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim-runtime.#015
kubernetes-bootstrap: Preparing to unpack .../17-vim-runtime_2%3a8.1.0875-5_all.deb ...#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/help.txt to /usr/share/vim/vim81/doc/help.txt.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/tags to /usr/share/vim/vim81/doc/tags.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Unpacking vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim.#015
kubernetes-bootstrap: Preparing to unpack .../18-vim_2%3a8.1.0875-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package zip.#015
kubernetes-bootstrap: Preparing to unpack .../19-zip_3.0-11_armhf.deb ...#015
kubernetes-bootstrap: Unpacking zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Setting up apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Setting up libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Setting up liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Setting up zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Setting up python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Setting up git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Setting up libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Setting up vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vim (vim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vimdiff (vimdiff) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rvim (rvim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rview (rview) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vi (vi) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/view (view) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/ex (ex) in auto mode#015
kubernetes-bootstrap: Setting up libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Setting up software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Installing ebtables and arptables...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   arptables ebtables
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 110 kB of archives.
kubernetes-bootstrap: After this operation, 289 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf arptables armhf 0.0.4+snapshot20181021-4 [32.8 kB]
kubernetes-bootstrap: Get:2 http://ftp.igh.cnrs.fr/pub/os/linux/raspbian/raspbian buster/main armhf ebtables armhf 2.0.10.4+snapshot20181205-3 [77.4 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 110 kB in 1s (114 kB/s)
kubernetes-bootstrap: Selecting previously unselected package arptables.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42807 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../arptables_0.0.4+snapshot20181021-4_armhf.deb ...#015
kubernetes-bootstrap: Unpacking arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Selecting previously unselected package ebtables.#015
kubernetes-bootstrap: Preparing to unpack .../ebtables_2.0.10.4+snapshot20181205-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Setting to use legacy tables for compatibility issues
kubernetes-bootstrap: update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ebtables-legacy to provide /usr/sbin/ebtables (ebtables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/arptables-legacy to provide /usr/sbin/arptables (arptables) in manual mode
kubernetes-bootstrap: net.bridge.bridge-nf-call-iptables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-ip6tables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-arptables = 1
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing docker 5:18.09.0~3-0~raspbian-stretch...
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch InRelease [31.1 kB]
kubernetes-bootstrap: Get:4 https://download.docker.com/linux/raspbian stretch/stable armhf Packages [10.4 kB]
kubernetes-bootstrap: Fetched 41.5 kB in 1s (33.8 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce-cli
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   aufs-tools cgroupfs-mount | cgroup-lite pigz libltdl7
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce docker-ce-cli
kubernetes-bootstrap: 0 upgraded, 3 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 54.0 MB of archives.
kubernetes-bootstrap: After this operation, 240 MB of additional disk space will be used.
kubernetes-bootstrap: Get:1 https://download.docker.com/linux/raspbian stretch/stable armhf containerd.io armhf 1.2.13-2 [15.1 MB]
kubernetes-bootstrap: Get:2 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce-cli armhf 5:19.03.11~3-0~raspbian-stretch [28.2 MB]
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce armhf 5:18.09.0~3-0~raspbian-stretch [10.8 MB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 54.0 MB in 19s (2,776 kB/s)
kubernetes-bootstrap: Selecting previously unselected package containerd.io.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42843 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../containerd.io_1.2.13-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce-cli.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce-cli_5%3a19.03.11~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce_5%3a18.09.0~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.#015
kubernetes-bootstrap: Setting up docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/dockerd-ce to provide /usr/bin/dockerd (dockerd) in auto mode#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: docker-ce set on hold.
kubernetes-bootstrap: Disabling swap and ensuring it doesn't turn back on after reboot
kubernetes-bootstrap: Synchronizing state of dphys-swapfile.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install disable dphys-swapfile
kubernetes-bootstrap: Removed /etc/systemd/system/multi-user.target.wants/dphys-swapfile.service.
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing kubernetes 1.16.1-00...
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Err:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap:   404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: Fetched 8,993 B in 2s (4,478 B/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: E: Failed to fetch https://packages.cloud.google.com/apt/dists/kubernetes-xenial/main/binary-armhf/by-hash/SHA256/e13738cb7ff20c8f4554a33aef53ff6e3697aab6840b46187f6741e709b7e02e  404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: E: Some index files failed to download. They have been ignored, or old ones used instead.
kubernetes-bootstrap: Retrying to update apt mirrors
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Err:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap:   404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: E: Failed to fetch https://packages.cloud.google.com/apt/dists/kubernetes-xenial/main/binary-armhf/by-hash/SHA256/e13738cb7ff20c8f4554a33aef53ff6e3697aab6840b46187f6741e709b7e02e  404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: E: Some index files failed to download. They have been ignored, or old ones used instead.
kubernetes-bootstrap: Retrying to update apt mirrors
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages [34.7 kB]
kubernetes-bootstrap: Fetched 34.7 kB in 2s (21.9 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubernetes-cni socat
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   nftables
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubeadm kubectl kubelet kubernetes-cni socat
kubernetes-bootstrap: 0 upgraded, 7 newly installed, 0 to remove and 110 not upgraded.
kubernetes-bootstrap: Need to get 47.8 MB of archives.
kubernetes-bootstrap: After this operation, 251 MB of additional disk space will be used.
kubernetes-bootstrap: Get:3 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf socat armhf 1.7.3.2-2 [324 kB]
kubernetes-bootstrap: Get:1 http://raspbian.mirror.vu.lt/raspbian/raspbian buster/main armhf conntrack armhf 1:1.4.5-2 [28.5 kB]
kubernetes-bootstrap: Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf cri-tools armhf 1.13.0-00 [7,905 kB]
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubernetes-cni armhf 0.7.5-00 [5,661 kB]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubelet armhf 1.16.1-00 [18.2 MB]
kubernetes-bootstrap: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubectl armhf 1.16.1-00 [8,005 kB]
kubernetes-bootstrap: Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubeadm armhf 1.16.1-00 [7,664 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 47.8 MB in 13s (3,624 kB/s)
kubernetes-bootstrap: Selecting previously unselected package conntrack.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43072 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-conntrack_1%3a1.4.5-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package cri-tools.#015
kubernetes-bootstrap: Preparing to unpack .../1-cri-tools_1.13.0-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubernetes-cni.#015
kubernetes-bootstrap: Preparing to unpack .../2-kubernetes-cni_0.7.5-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package socat.#015
kubernetes-bootstrap: Preparing to unpack .../3-socat_1.7.3.2-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubelet.#015
kubernetes-bootstrap: Preparing to unpack .../4-kubelet_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubectl.#015
kubernetes-bootstrap: Preparing to unpack .../5-kubectl_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubeadm.#015
kubernetes-bootstrap: Preparing to unpack .../6-kubeadm_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Setting up kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Setting up cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Setting up kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Setting up kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.#015
kubernetes-bootstrap: Setting up kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: kubelet set on hold.
kubernetes-bootstrap: kubectl set on hold.
kubernetes-bootstrap: kubeadm set on hold.
kubernetes-bootstrap: Pulling down all kubeadm images...
kubernetes-bootstrap: Installing keepalived 1:2.0.10-1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   lm-sensors snmp-mibs-downloader
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   ipvsadm
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   keepalived libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: 0 upgraded, 8 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 4,514 kB of archives.
kubernetes-bootstrap: After this operation, 8,563 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf mysql-common all 5.8+1.0.5 [7,324 B]
kubernetes-bootstrap: Get:2 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf mariadb-common all 1:10.3.22-0+deb10u1 [31.9 kB]
kubernetes-bootstrap: Get:4 http://raspbian.mirror.uk.sargasso.net/raspbian buster/main armhf libsensors-config all 1:3.5.0-3 [31.6 kB]
kubernetes-bootstrap: Get:5 http://mirror.nl.leaseweb.net/raspbian/raspbian buster/main armhf libsensors5 armhf 1:3.5.0-3 [49.5 kB]
kubernetes-bootstrap: Get:6 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf libsnmp-base all 5.7.3+dfsg-5 [1,595 kB]
kubernetes-bootstrap: Get:8 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf keepalived armhf 1:2.0.10-1 [437 kB]
kubernetes-bootstrap: Get:3 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf libmariadb3 armhf 1:10.3.22-0+deb10u1 [158 kB]
kubernetes-bootstrap: Get:7 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf libsnmp30 armhf 5.7.3+dfsg-5+b1 [2,204 kB]
kubernetes-bootstrap: I0604 22:12:57.347346    4867 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 4,514 kB in 6s (797 kB/s)
kubernetes-bootstrap: Selecting previously unselected package mysql-common.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43140 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-mysql-common_5.8+1.0.5_all.deb ...#015
kubernetes-bootstrap: Unpacking mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: Selecting previously unselected package mariadb-common.#015
kubernetes-bootstrap: Preparing to unpack .../1-mariadb-common_1%3a10.3.22-0+deb10u1_all.deb ...#015
kubernetes-bootstrap: Unpacking mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libmariadb3:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../2-libmariadb3_1%3a10.3.22-0+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors-config.#015
kubernetes-bootstrap: Preparing to unpack .../3-libsensors-config_1%3a3.5.0-3_all.deb ...#015
kubernetes-bootstrap: Unpacking libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../4-libsensors5_1%3a3.5.0-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp-base.#015
kubernetes-bootstrap: Preparing to unpack .../5-libsnmp-base_5.7.3+dfsg-5_all.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp30:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../6-libsnmp30_5.7.3+dfsg-5+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package keepalived.#015
kubernetes-bootstrap: Preparing to unpack .../7-keepalived_1%3a2.0.10-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Setting up mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Setting up libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/mariadb.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Setting up libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Setting up keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /lib/systemd/system/keepalived.service.#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: keepalived set on hold.
kubernetes-bootstrap: Synchronizing state of keepalived.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install enable keepalived
kubernetes-bootstrap: Installing haproxy 1.8.19-1+rpi1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   liblua5.3-0
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   vim-haproxy haproxy-doc
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   haproxy liblua5.3-0
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 1,297 kB of archives.
kubernetes-bootstrap: After this operation, 2,753 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf liblua5.3-0 armhf 5.3.3-1.1 [94.4 kB]
kubernetes-bootstrap: Get:2 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf haproxy armhf 1.8.19-1+rpi1 [1,203 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 1,297 kB in 2s (654 kB/s)
kubernetes-bootstrap: Selecting previously unselected package liblua5.3-0:armhf.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43378 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../liblua5.3-0_5.3.3-1.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package haproxy.#015
kubernetes-bootstrap: Preparing to unpack .../haproxy_1.8.19-1+rpi1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.16.10
kubernetes-bootstrap: Setting up liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Setting up haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /lib/systemd/system/haproxy.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for rsyslog (8.1901.0-1) ...#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: haproxy set on hold.
kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: 192.168.0.153 192.168.0.150 172.17.0.1 fd20:da22:881e:6300:e5dc:4689:a7de:af9f 2a00:1cf8:1200:2a5f:d62b:4cc9:7b4e:da81 
kubernetes-bootstrap: VIP currently resides on local host and no cluster exists.
kubernetes-bootstrap: Initialising as new Kubernetes cluster
kubernetes-bootstrap: I0604 22:13:30.203075    5697 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [init] Using Kubernetes version: v1.16.10
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "ca" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-03 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.153 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-ca" certificate and key
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/ca" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-03 localhost] and IPs [192.168.0.153 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-03 localhost] and IPs [192.168.0.153 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "sa" key and public key
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "kubelet.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
kubernetes-bootstrap: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.
kubernetes-bootstrap: [apiclient] All control plane components are healthy after 50.013331 seconds
kubernetes-bootstrap: [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
kubernetes-bootstrap: [kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster
kubernetes-bootstrap: [upload-certs] Skipping phase. Please see --upload-certs
kubernetes-bootstrap: [mark-control-plane] Marking the node rpi-kube-master-03 as control-plane by adding the label "node-role.kubernetes.io/master=''"
kubernetes-bootstrap: [mark-control-plane] Marking the node rpi-kube-master-03 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
kubernetes-bootstrap: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
kubernetes-bootstrap: [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
kubernetes-bootstrap: [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
kubernetes-bootstrap: [addons] Applied essential addon: CoreDNS
kubernetes-bootstrap: [addons] Applied essential addon: kube-proxy
kubernetes-bootstrap: 
kubernetes-bootstrap: Your Kubernetes control-plane has initialized successfully!
kubernetes-bootstrap: 
kubernetes-bootstrap: To start using your cluster, you need to run the following as a regular user:
kubernetes-bootstrap: 
kubernetes-bootstrap:   mkdir -p $HOME/.kube
kubernetes-bootstrap:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
kubernetes-bootstrap:   sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubernetes-bootstrap: 
kubernetes-bootstrap: You should now deploy a pod network to the cluster.
kubernetes-bootstrap: Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
kubernetes-bootstrap:   https://kubernetes.io/docs/concepts/cluster-administration/addons/
kubernetes-bootstrap: 
kubernetes-bootstrap: You can now join any number of control-plane nodes by copying certificate authorities 
kubernetes-bootstrap: and service account keys on each node and then running the following as root:
kubernetes-bootstrap: 
kubernetes-bootstrap:   kubeadm join 192.168.0.150:6443 --token <value withheld> \
kubernetes-bootstrap:     --discovery-token-ca-cert-hash sha256:d6105e1e79a896d3cd303d79997eac78d45a321fe36aecd0a3adc186b27b0321 \
kubernetes-bootstrap:     --control-plane --certificate-key <value withheld>#011  
kubernetes-bootstrap: 
kubernetes-bootstrap: Then you can join any number of worker nodes by running the following on each as root:
kubernetes-bootstrap: 
kubernetes-bootstrap: kubeadm join 192.168.0.150:6443 --token <value withheld> \
kubernetes-bootstrap:     --discovery-token-ca-cert-hash sha256:d6105e1e79a896d3cd303d79997eac78d45a321fe36aecd0a3adc186b27b0321 
kubernetes-bootstrap: podsecuritypolicy.policy/psp.flannel.unprivileged created
kubernetes-bootstrap: clusterrole.rbac.authorization.k8s.io/flannel created
kubernetes-bootstrap: clusterrolebinding.rbac.authorization.k8s.io/flannel created
kubernetes-bootstrap: serviceaccount/flannel created
kubernetes-bootstrap: configmap/kube-flannel-cfg created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-amd64 created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-arm64 created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-arm created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-ppc64le created
kubernetes-bootstrap: daemonset.apps/kube-flannel-ds-s390x created
kubernetes-bootstrap: Finished booting! Kubernetes successfully running!

Master2 output

kubernetes-bootstrap: Setting hostname to: rpi-kube-master-02
kubernetes-bootstrap: Waiting for system to boot before attempting to install packages...
kubernetes-bootstrap: Updating core packages...
kubernetes-bootstrap: Get:1 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
kubernetes-bootstrap: Get:2 http://archive.raspberrypi.org/debian buster InRelease [32.6 kB]
kubernetes-bootstrap: Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
kubernetes-bootstrap: Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [330 kB]
kubernetes-bootstrap: Get:5 http://raspbian.raspberrypi.org/raspbian buster/contrib armhf Packages [58.7 kB]
kubernetes-bootstrap: Get:6 http://raspbian.raspberrypi.org/raspbian buster/non-free armhf Packages [103 kB]
kubernetes-bootstrap: Fetched 13.5 MB in 17s (799 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Installing base packages...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git-man libcurl3-gnutls
kubernetes-bootstrap:   liberror-perl libgirepository-1.0-1 libjq1 libonig5 libpackagekit-glib2-18
kubernetes-bootstrap:   python3-dbus python3-gi python3-pycurl python3-software-properties
kubernetes-bootstrap:   vim-runtime
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk
kubernetes-bootstrap:   gitweb git-cvs git-mediawiki git-svn python-dbus-doc python3-dbus-dbg
kubernetes-bootstrap:   libcurl4-gnutls-dev python-pycurl-doc python3-pycurl-dbg ctags vim-doc
kubernetes-bootstrap:   vim-scripts
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   packagekit unattended-upgrades
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   apt-transport-https gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git git-man jq
kubernetes-bootstrap:   libcurl3-gnutls liberror-perl libgirepository-1.0-1 libjq1 libonig5
kubernetes-bootstrap:   libpackagekit-glib2-18 python3-dbus python3-gi python3-pycurl
kubernetes-bootstrap:   python3-software-properties software-properties-common vim vim-runtime zip
kubernetes-bootstrap: 0 upgraded, 20 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 14.4 MB of archives.
kubernetes-bootstrap: After this operation, 70.6 MB of additional disk space will be used.
kubernetes-bootstrap: Get:2 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libgirepository-1.0-1 armhf 1.58.3-2 [78.8 kB]
kubernetes-bootstrap: Get:3 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf gir1.2-glib-2.0 armhf 1.58.3-2 [143 kB]
kubernetes-bootstrap: Get:5 http://mirror.serverius.net/raspbian/raspbian buster/main armhf gir1.2-packagekitglib-1.0 armhf 1.1.12-5 [35.9 kB]
kubernetes-bootstrap: Get:6 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf libcurl3-gnutls armhf 7.64.0-4+deb10u1 [292 kB]
kubernetes-bootstrap: Get:7 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf liberror-perl all 0.17027-2 [30.9 kB]
kubernetes-bootstrap: Get:9 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf git armhf 1:2.20.1-2+deb10u3 [4,194 kB]
kubernetes-bootstrap: Get:4 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf libpackagekit-glib2-18 armhf 1.1.12-5 [98.1 kB]
kubernetes-bootstrap: Get:10 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libonig5 armhf 6.9.1-1 [150 kB]
kubernetes-bootstrap: Get:11 http://mirror.as43289.net/raspbian/raspbian buster/main armhf libjq1 armhf 1.5+dfsg-2+b1 [119 kB]
kubernetes-bootstrap: Get:12 http://mirror.as43289.net/raspbian/raspbian buster/main armhf jq armhf 1.5+dfsg-2+b1 [59.3 kB]
kubernetes-bootstrap: Get:14 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf python3-gi armhf 3.30.4-1 [161 kB]
kubernetes-bootstrap: Get:15 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf python3-pycurl armhf 7.43.0.2-0.1 [62.8 kB]
kubernetes-bootstrap: Get:8 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf git-man all 1:2.20.1-2+deb10u3 [1,620 kB]
kubernetes-bootstrap: Get:17 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf software-properties-common all 0.96.20.2-2 [83.3 kB]
kubernetes-bootstrap: Get:13 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf python3-dbus armhf 1.2.8-3 [95.9 kB]
kubernetes-bootstrap: Get:16 http://debian.bio.lmu.de/raspbian/raspbian buster/main armhf python3-software-properties all 0.96.20.2-2 [49.6 kB]
kubernetes-bootstrap: Get:18 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf vim-runtime all 2:8.1.0875-5 [5,775 kB]
kubernetes-bootstrap: Get:19 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf vim armhf 2:8.1.0875-5 [1,005 kB]
kubernetes-bootstrap: Get:20 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf zip armhf 3.0-11 [212 kB]
kubernetes-bootstrap: Get:1 http://raspbian.mirror.uk.sargasso.net/raspbian buster/main armhf apt-transport-https all 1.8.2.1 [149 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 14.4 MB in 6s (2,528 kB/s)
kubernetes-bootstrap: Selecting previously unselected package apt-transport-https.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 39798 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../00-apt-transport-https_1.8.2.1_all.deb ...#015
kubernetes-bootstrap: Unpacking apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libgirepository-1.0-1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../01-libgirepository-1.0-1_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-glib-2.0:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../02-gir1.2-glib-2.0_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package libpackagekit-glib2-18:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../03-libpackagekit-glib2-18_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-packagekitglib-1.0.#015
kubernetes-bootstrap: Preparing to unpack .../04-gir1.2-packagekitglib-1.0_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libcurl3-gnutls:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../05-libcurl3-gnutls_7.64.0-4+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package liberror-perl.#015
kubernetes-bootstrap: Preparing to unpack .../06-liberror-perl_0.17027-2_all.deb ...#015
kubernetes-bootstrap: Unpacking liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package git-man.#015
kubernetes-bootstrap: Preparing to unpack .../07-git-man_1%3a2.20.1-2+deb10u3_all.deb ...#015
kubernetes-bootstrap: Unpacking git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package git.#015
kubernetes-bootstrap: Preparing to unpack .../08-git_1%3a2.20.1-2+deb10u3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libonig5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../09-libonig5_6.9.1-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libjq1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../10-libjq1_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package jq.#015
kubernetes-bootstrap: Preparing to unpack .../11-jq_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-dbus.#015
kubernetes-bootstrap: Preparing to unpack .../12-python3-dbus_1.2.8-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-gi.#015
kubernetes-bootstrap: Preparing to unpack .../13-python3-gi_3.30.4-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-pycurl.#015
kubernetes-bootstrap: Preparing to unpack .../14-python3-pycurl_7.43.0.2-0.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-software-properties.#015
kubernetes-bootstrap: Preparing to unpack .../15-python3-software-properties_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package software-properties-common.#015
kubernetes-bootstrap: Preparing to unpack .../16-software-properties-common_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim-runtime.#015
kubernetes-bootstrap: Preparing to unpack .../17-vim-runtime_2%3a8.1.0875-5_all.deb ...#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/help.txt to /usr/share/vim/vim81/doc/help.txt.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/tags to /usr/share/vim/vim81/doc/tags.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Unpacking vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim.#015
kubernetes-bootstrap: Preparing to unpack .../18-vim_2%3a8.1.0875-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package zip.#015
kubernetes-bootstrap: Preparing to unpack .../19-zip_3.0-11_armhf.deb ...#015
kubernetes-bootstrap: Unpacking zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Setting up apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Setting up libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Setting up liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Setting up zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Setting up python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Setting up git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Setting up libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Setting up vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vim (vim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vimdiff (vimdiff) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rvim (rvim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rview (rview) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vi (vi) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/view (view) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/ex (ex) in auto mode#015
kubernetes-bootstrap: Setting up libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Setting up software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Installing ebtables and arptables...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   arptables ebtables
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 110 kB of archives.
kubernetes-bootstrap: After this operation, 289 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf arptables armhf 0.0.4+snapshot20181021-4 [32.8 kB]
kubernetes-bootstrap: Get:2 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf ebtables armhf 2.0.10.4+snapshot20181205-3 [77.4 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 110 kB in 1s (150 kB/s)
kubernetes-bootstrap: Selecting previously unselected package arptables.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42807 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../arptables_0.0.4+snapshot20181021-4_armhf.deb ...#015
kubernetes-bootstrap: Unpacking arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Selecting previously unselected package ebtables.#015
kubernetes-bootstrap: Preparing to unpack .../ebtables_2.0.10.4+snapshot20181205-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Setting to use legacy tables for compatibility issues
kubernetes-bootstrap: update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ebtables-legacy to provide /usr/sbin/ebtables (ebtables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/arptables-legacy to provide /usr/sbin/arptables (arptables) in manual mode
kubernetes-bootstrap: net.bridge.bridge-nf-call-iptables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-ip6tables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-arptables = 1
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing docker 5:18.09.0~3-0~raspbian-stretch...
kubernetes-bootstrap: Hit:1 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:2 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch InRelease [31.1 kB]
kubernetes-bootstrap: Get:4 https://download.docker.com/linux/raspbian stretch/stable armhf Packages [10.4 kB]
kubernetes-bootstrap: Fetched 41.5 kB in 1s (37.0 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce-cli
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   aufs-tools cgroupfs-mount | cgroup-lite pigz libltdl7
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce docker-ce-cli
kubernetes-bootstrap: 0 upgraded, 3 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 54.0 MB of archives.
kubernetes-bootstrap: After this operation, 240 MB of additional disk space will be used.
kubernetes-bootstrap: Get:1 https://download.docker.com/linux/raspbian stretch/stable armhf containerd.io armhf 1.2.13-2 [15.1 MB]
kubernetes-bootstrap: Get:2 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce-cli armhf 5:19.03.11~3-0~raspbian-stretch [28.2 MB]
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce armhf 5:18.09.0~3-0~raspbian-stretch [10.8 MB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 54.0 MB in 33s (1,614 kB/s)
kubernetes-bootstrap: Selecting previously unselected package containerd.io.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42843 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../containerd.io_1.2.13-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce-cli.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce-cli_5%3a19.03.11~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce_5%3a18.09.0~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.#015
kubernetes-bootstrap: Setting up docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/dockerd-ce to provide /usr/bin/dockerd (dockerd) in auto mode#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: docker-ce set on hold.
kubernetes-bootstrap: Disabling swap and ensuring it doesn't turn back on after reboot
kubernetes-bootstrap: Synchronizing state of dphys-swapfile.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install disable dphys-swapfile
kubernetes-bootstrap: Removed /etc/systemd/system/multi-user.target.wants/dphys-swapfile.service.
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing kubernetes 1.16.1-00...
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Ign:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap: Err:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages
kubernetes-bootstrap:   404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: Fetched 8,993 B in 2s (4,297 B/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: E: Failed to fetch https://packages.cloud.google.com/apt/dists/kubernetes-xenial/main/binary-armhf/by-hash/SHA256/e13738cb7ff20c8f4554a33aef53ff6e3697aab6840b46187f6741e709b7e02e  404  Not Found [IP: 2a00:1450:400e:80c::200e 443]
kubernetes-bootstrap: E: Some index files failed to download. They have been ignored, or old ones used instead.
kubernetes-bootstrap: Retrying to update apt mirrors
kubernetes-bootstrap: Hit:1 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:2 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages [34.7 kB]
kubernetes-bootstrap: Fetched 34.7 kB in 2s (21.0 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubernetes-cni socat
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   nftables
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubeadm kubectl kubelet kubernetes-cni socat
kubernetes-bootstrap: 0 upgraded, 7 newly installed, 0 to remove and 110 not upgraded.
kubernetes-bootstrap: Need to get 47.8 MB of archives.
kubernetes-bootstrap: After this operation, 251 MB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://mirror.as43289.net/raspbian/raspbian buster/main armhf conntrack armhf 1:1.4.5-2 [28.5 kB]
kubernetes-bootstrap: Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf cri-tools armhf 1.13.0-00 [7,905 kB]
kubernetes-bootstrap: Get:3 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf socat armhf 1.7.3.2-2 [324 kB]
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubernetes-cni armhf 0.7.5-00 [5,661 kB]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubelet armhf 1.16.1-00 [18.2 MB]
kubernetes-bootstrap: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubectl armhf 1.16.1-00 [8,005 kB]
kubernetes-bootstrap: Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubeadm armhf 1.16.1-00 [7,664 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 47.8 MB in 24s (2,012 kB/s)
kubernetes-bootstrap: Selecting previously unselected package conntrack.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43072 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-conntrack_1%3a1.4.5-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package cri-tools.#015
kubernetes-bootstrap: Preparing to unpack .../1-cri-tools_1.13.0-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubernetes-cni.#015
kubernetes-bootstrap: Preparing to unpack .../2-kubernetes-cni_0.7.5-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package socat.#015
kubernetes-bootstrap: Preparing to unpack .../3-socat_1.7.3.2-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubelet.#015
kubernetes-bootstrap: Preparing to unpack .../4-kubelet_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubectl.#015
kubernetes-bootstrap: Preparing to unpack .../5-kubectl_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubeadm.#015
kubernetes-bootstrap: Preparing to unpack .../6-kubeadm_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Setting up kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Setting up cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Setting up kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Setting up kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.#015
kubernetes-bootstrap: Setting up kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: kubelet set on hold.
kubernetes-bootstrap: kubectl set on hold.
kubernetes-bootstrap: kubeadm set on hold.
kubernetes-bootstrap: Pulling down all kubeadm images...
kubernetes-bootstrap: Installing keepalived 1:2.0.10-1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   lm-sensors snmp-mibs-downloader
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   ipvsadm
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   keepalived libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: I0604 22:13:25.429500    4658 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: 0 upgraded, 8 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 4,514 kB of archives.
kubernetes-bootstrap: After this operation, 8,563 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf mysql-common all 5.8+1.0.5 [7,324 B]
kubernetes-bootstrap: Get:2 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf mariadb-common all 1:10.3.22-0+deb10u1 [31.9 kB]
kubernetes-bootstrap: Get:3 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libmariadb3 armhf 1:10.3.22-0+deb10u1 [158 kB]
kubernetes-bootstrap: Get:4 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf libsensors-config all 1:3.5.0-3 [31.6 kB]
kubernetes-bootstrap: Get:5 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf libsensors5 armhf 1:3.5.0-3 [49.5 kB]
kubernetes-bootstrap: Get:6 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libsnmp-base all 5.7.3+dfsg-5 [1,595 kB]
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: Get:7 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libsnmp30 armhf 5.7.3+dfsg-5+b1 [2,204 kB]
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: Get:8 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf keepalived armhf 1:2.0.10-1 [437 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 4,514 kB in 2min 53s (26.1 kB/s)
kubernetes-bootstrap: Selecting previously unselected package mysql-common.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43140 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-mysql-common_5.8+1.0.5_all.deb ...#015
kubernetes-bootstrap: Unpacking mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: Selecting previously unselected package mariadb-common.#015
kubernetes-bootstrap: Preparing to unpack .../1-mariadb-common_1%3a10.3.22-0+deb10u1_all.deb ...#015
kubernetes-bootstrap: Unpacking mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libmariadb3:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../2-libmariadb3_1%3a10.3.22-0+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors-config.#015
kubernetes-bootstrap: Preparing to unpack .../3-libsensors-config_1%3a3.5.0-3_all.deb ...#015
kubernetes-bootstrap: Unpacking libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../4-libsensors5_1%3a3.5.0-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp-base.#015
kubernetes-bootstrap: Preparing to unpack .../5-libsnmp-base_5.7.3+dfsg-5_all.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp30:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../6-libsnmp30_5.7.3+dfsg-5+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package keepalived.#015
kubernetes-bootstrap: Preparing to unpack .../7-keepalived_1%3a2.0.10-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Setting up mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Setting up libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/mariadb.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Setting up libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Setting up keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /lib/systemd/system/keepalived.service.#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: keepalived set on hold.
kubernetes-bootstrap: Synchronizing state of keepalived.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install enable keepalived
kubernetes-bootstrap: Installing haproxy 1.8.19-1+rpi1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   liblua5.3-0
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   vim-haproxy haproxy-doc
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   haproxy liblua5.3-0
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 1,297 kB of archives.
kubernetes-bootstrap: After this operation, 2,753 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf liblua5.3-0 armhf 5.3.3-1.1 [94.4 kB]
kubernetes-bootstrap: Get:2 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf haproxy armhf 1.8.19-1+rpi1 [1,203 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 1,297 kB in 3s (439 kB/s)
kubernetes-bootstrap: Selecting previously unselected package liblua5.3-0:armhf.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43378 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../liblua5.3-0_5.3.3-1.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package haproxy.#015
kubernetes-bootstrap: Preparing to unpack .../haproxy_1.8.19-1+rpi1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: Setting up liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Setting up haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /lib/systemd/system/haproxy.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for rsyslog (8.1901.0-1) ...#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: haproxy set on hold.
kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: Did not win VIP election, joining existing cluster.
kubernetes-bootstrap: The connection to the server localhost:8080 was refused - did you specify the right host or port?
kubernetes-bootstrap: Checking to see if control-plane has been fully initialised...
kubernetes-bootstrap: This could take up to 10 minutes...
kubernetes-bootstrap: [0] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [1] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: [2] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [3] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [4] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [5] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: Kubernetes control-plane has finished initalising!
kubernetes-bootstrap: Attempting to join cluster!
kubernetes-bootstrap: I0604 22:17:56.878901    8707 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [upload-certs] Using certificate key:
kubernetes-bootstrap: db2c4fb4381be21b87aff3aa9120db2b010302a5502ee09eff214eac1ae148f3
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [preflight] Running pre-flight checks before initializing the new control plane instance
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-02 localhost] and IPs [192.168.0.152 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-02 localhost] and IPs [192.168.0.152 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-02 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.152 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Using the existing "sa" key
kubernetes-bootstrap: [kubeconfig] Generating kubeconfig files
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [check-etcd] Checking that the etcd cluster is healthy
kubernetes-bootstrap: [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
kubernetes-bootstrap: [etcd] Announced new etcd member joining to the existing etcd cluster
kubernetes-bootstrap: [etcd] Creating static Pod manifest for "etcd"
kubernetes-bootstrap: [etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
kubernetes-bootstrap: {"level":"warn","ts":"2020-06-04T22:19:26.475+1000","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"passthrough:///https://192.168.0.152:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.
kubernetes-bootstrap: [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
kubernetes-bootstrap: error execution phase control-plane-join/update-status: error uploading configuration: unable to create ConfigMap: Post https://192.168.0.150:6443/api/v1/namespaces/kube-system/configmaps: write tcp 192.168.0.152:57168->192.168.0.150:6443: write: connection reset by peer
kubernetes-bootstrap: To see the stack trace of this error execute with --v=5 or higher

Master1 output

kubernetes-bootstrap: Setting hostname to: rpi-kube-master-01
kubernetes-bootstrap: Waiting for system to boot before attempting to install packages...
kubernetes-bootstrap: Updating core packages...
kubernetes-bootstrap: Get:1 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
kubernetes-bootstrap: Get:2 http://archive.raspberrypi.org/debian buster InRelease [32.6 kB]
kubernetes-bootstrap: Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
kubernetes-bootstrap: Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [330 kB]
kubernetes-bootstrap: Get:5 http://raspbian.raspberrypi.org/raspbian buster/contrib armhf Packages [58.7 kB]
kubernetes-bootstrap: Get:6 http://raspbian.raspberrypi.org/raspbian buster/non-free armhf Packages [103 kB]
kubernetes-bootstrap: Fetched 13.5 MB in 9s (1,529 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Installing base packages...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git-man libcurl3-gnutls
kubernetes-bootstrap:   liberror-perl libgirepository-1.0-1 libjq1 libonig5 libpackagekit-glib2-18
kubernetes-bootstrap:   python3-dbus python3-gi python3-pycurl python3-software-properties
kubernetes-bootstrap:   vim-runtime
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk
kubernetes-bootstrap:   gitweb git-cvs git-mediawiki git-svn python-dbus-doc python3-dbus-dbg
kubernetes-bootstrap:   libcurl4-gnutls-dev python-pycurl-doc python3-pycurl-dbg ctags vim-doc
kubernetes-bootstrap:   vim-scripts
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   packagekit unattended-upgrades
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   apt-transport-https gir1.2-glib-2.0 gir1.2-packagekitglib-1.0 git git-man jq
kubernetes-bootstrap:   libcurl3-gnutls liberror-perl libgirepository-1.0-1 libjq1 libonig5
kubernetes-bootstrap:   libpackagekit-glib2-18 python3-dbus python3-gi python3-pycurl
kubernetes-bootstrap:   python3-software-properties software-properties-common vim vim-runtime zip
kubernetes-bootstrap: 0 upgraded, 20 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 14.4 MB of archives.
kubernetes-bootstrap: After this operation, 70.6 MB of additional disk space will be used.
kubernetes-bootstrap: Get:2 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libgirepository-1.0-1 armhf 1.58.3-2 [78.8 kB]
kubernetes-bootstrap: Get:1 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf apt-transport-https all 1.8.2.1 [149 kB]
kubernetes-bootstrap: Get:3 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf gir1.2-glib-2.0 armhf 1.58.3-2 [143 kB]
kubernetes-bootstrap: Get:4 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libpackagekit-glib2-18 armhf 1.1.12-5 [98.1 kB]
kubernetes-bootstrap: Get:5 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf gir1.2-packagekitglib-1.0 armhf 1.1.12-5 [35.9 kB]
kubernetes-bootstrap: Get:6 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libcurl3-gnutls armhf 7.64.0-4+deb10u1 [292 kB]
kubernetes-bootstrap: Get:7 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf liberror-perl all 0.17027-2 [30.9 kB]
kubernetes-bootstrap: Get:8 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf git-man all 1:2.20.1-2+deb10u3 [1,620 kB]
kubernetes-bootstrap: Get:9 http://raspbian.mirror.liteserver.nl/raspbian buster/main armhf git armhf 1:2.20.1-2+deb10u3 [4,194 kB]
kubernetes-bootstrap: Get:11 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf libjq1 armhf 1.5+dfsg-2+b1 [119 kB]
kubernetes-bootstrap: Get:10 http://mirror.netcologne.de/raspbian/raspbian buster/main armhf libonig5 armhf 6.9.1-1 [150 kB]
kubernetes-bootstrap: Get:12 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf jq armhf 1.5+dfsg-2+b1 [59.3 kB]
kubernetes-bootstrap: Get:13 http://mirror.serverius.net/raspbian/raspbian buster/main armhf python3-dbus armhf 1.2.8-3 [95.9 kB]
kubernetes-bootstrap: Get:16 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf python3-software-properties all 0.96.20.2-2 [49.6 kB]
kubernetes-bootstrap: Get:14 http://mirror.as43289.net/raspbian/raspbian buster/main armhf python3-gi armhf 3.30.4-1 [161 kB]
kubernetes-bootstrap: Get:17 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf software-properties-common all 0.96.20.2-2 [83.3 kB]
kubernetes-bootstrap: Get:15 http://debian.bio.lmu.de/raspbian/raspbian buster/main armhf python3-pycurl armhf 7.43.0.2-0.1 [62.8 kB]
kubernetes-bootstrap: Get:18 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf vim-runtime all 2:8.1.0875-5 [5,775 kB]
kubernetes-bootstrap: Get:19 http://mirror.de.leaseweb.net/raspbian/raspbian buster/main armhf vim armhf 2:8.1.0875-5 [1,005 kB]
kubernetes-bootstrap: Get:20 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf zip armhf 3.0-11 [212 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 14.4 MB in 8s (1,879 kB/s)
kubernetes-bootstrap: Selecting previously unselected package apt-transport-https.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 39798 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../00-apt-transport-https_1.8.2.1_all.deb ...#015
kubernetes-bootstrap: Unpacking apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libgirepository-1.0-1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../01-libgirepository-1.0-1_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-glib-2.0:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../02-gir1.2-glib-2.0_1.58.3-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package libpackagekit-glib2-18:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../03-libpackagekit-glib2-18_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package gir1.2-packagekitglib-1.0.#015
kubernetes-bootstrap: Preparing to unpack .../04-gir1.2-packagekitglib-1.0_1.1.12-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libcurl3-gnutls:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../05-libcurl3-gnutls_7.64.0-4+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package liberror-perl.#015
kubernetes-bootstrap: Preparing to unpack .../06-liberror-perl_0.17027-2_all.deb ...#015
kubernetes-bootstrap: Unpacking liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package git-man.#015
kubernetes-bootstrap: Preparing to unpack .../07-git-man_1%3a2.20.1-2+deb10u3_all.deb ...#015
kubernetes-bootstrap: Unpacking git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package git.#015
kubernetes-bootstrap: Preparing to unpack .../08-git_1%3a2.20.1-2+deb10u3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libonig5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../09-libonig5_6.9.1-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libjq1:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../10-libjq1_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package jq.#015
kubernetes-bootstrap: Preparing to unpack .../11-jq_1.5+dfsg-2+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-dbus.#015
kubernetes-bootstrap: Preparing to unpack .../12-python3-dbus_1.2.8-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-gi.#015
kubernetes-bootstrap: Preparing to unpack .../13-python3-gi_3.30.4-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-pycurl.#015
kubernetes-bootstrap: Preparing to unpack .../14-python3-pycurl_7.43.0.2-0.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package python3-software-properties.#015
kubernetes-bootstrap: Preparing to unpack .../15-python3-software-properties_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package software-properties-common.#015
kubernetes-bootstrap: Preparing to unpack .../16-software-properties-common_0.96.20.2-2_all.deb ...#015
kubernetes-bootstrap: Unpacking software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim-runtime.#015
kubernetes-bootstrap: Preparing to unpack .../17-vim-runtime_2%3a8.1.0875-5_all.deb ...#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/help.txt to /usr/share/vim/vim81/doc/help.txt.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Adding 'diversion of /usr/share/vim/vim81/doc/tags to /usr/share/vim/vim81/doc/tags.vim-tiny by vim-runtime'#015
kubernetes-bootstrap: Unpacking vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package vim.#015
kubernetes-bootstrap: Preparing to unpack .../18-vim_2%3a8.1.0875-5_armhf.deb ...#015
kubernetes-bootstrap: Unpacking vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package zip.#015
kubernetes-bootstrap: Preparing to unpack .../19-zip_3.0-11_armhf.deb ...#015
kubernetes-bootstrap: Unpacking zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-dbus (1.2.8-3) ...#015
kubernetes-bootstrap: Setting up apt-transport-https (1.8.2.1) ...#015
kubernetes-bootstrap: Setting up libpackagekit-glib2-18:armhf (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...#015
kubernetes-bootstrap: Setting up liberror-perl (0.17027-2) ...#015
kubernetes-bootstrap: Setting up zip (3.0-11) ...#015
kubernetes-bootstrap: Setting up python3-pycurl (7.43.0.2-0.1) ...#015
kubernetes-bootstrap: Setting up python3-software-properties (0.96.20.2-2) ...#015
kubernetes-bootstrap: Setting up git-man (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up libgirepository-1.0-1:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up vim-runtime (2:8.1.0875-5) ...#015
kubernetes-bootstrap: Setting up libonig5:armhf (6.9.1-1) ...#015
kubernetes-bootstrap: Setting up vim (2:8.1.0875-5) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vim (vim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vimdiff (vimdiff) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rvim (rvim) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rview (rview) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vi (vi) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/view (view) in auto mode#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/ex (ex) in auto mode#015
kubernetes-bootstrap: Setting up libjq1:armhf (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up git (1:2.20.1-2+deb10u3) ...#015
kubernetes-bootstrap: Setting up gir1.2-glib-2.0:armhf (1.58.3-2) ...#015
kubernetes-bootstrap: Setting up jq (1.5+dfsg-2+b1) ...#015
kubernetes-bootstrap: Setting up gir1.2-packagekitglib-1.0 (1.1.12-5) ...#015
kubernetes-bootstrap: Setting up python3-gi (3.30.4-1) ...#015
kubernetes-bootstrap: Setting up software-properties-common (0.96.20.2-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Installing ebtables and arptables...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   arptables ebtables
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 110 kB of archives.
kubernetes-bootstrap: After this operation, 289 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf arptables armhf 0.0.4+snapshot20181021-4 [32.8 kB]
kubernetes-bootstrap: Get:2 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf ebtables armhf 2.0.10.4+snapshot20181205-3 [77.4 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 110 kB in 1s (132 kB/s)
kubernetes-bootstrap: Selecting previously unselected package arptables.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42807 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../arptables_0.0.4+snapshot20181021-4_armhf.deb ...#015
kubernetes-bootstrap: Unpacking arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Selecting previously unselected package ebtables.#015
kubernetes-bootstrap: Preparing to unpack .../ebtables_2.0.10.4+snapshot20181205-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up ebtables (2.0.10.4+snapshot20181205-3) ...#015
kubernetes-bootstrap: Setting up arptables (0.0.4+snapshot20181021-4) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Setting to use legacy tables for compatibility issues
kubernetes-bootstrap: update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/ebtables-legacy to provide /usr/sbin/ebtables (ebtables) in manual mode
kubernetes-bootstrap: update-alternatives: using /usr/sbin/arptables-legacy to provide /usr/sbin/arptables (arptables) in manual mode
kubernetes-bootstrap: net.bridge.bridge-nf-call-iptables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-ip6tables = 1
kubernetes-bootstrap: net.bridge.bridge-nf-call-arptables = 1
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing docker 5:18.09.0~3-0~raspbian-stretch...
kubernetes-bootstrap: Hit:1 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:2 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch InRelease [31.1 kB]
kubernetes-bootstrap: Get:4 https://download.docker.com/linux/raspbian stretch/stable armhf Packages [10.4 kB]
kubernetes-bootstrap: Fetched 41.5 kB in 1s (34.0 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce-cli
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   aufs-tools cgroupfs-mount | cgroup-lite pigz libltdl7
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   containerd.io docker-ce docker-ce-cli
kubernetes-bootstrap: 0 upgraded, 3 newly installed, 0 to remove and 109 not upgraded.
kubernetes-bootstrap: Need to get 54.0 MB of archives.
kubernetes-bootstrap: After this operation, 240 MB of additional disk space will be used.
kubernetes-bootstrap: Get:1 https://download.docker.com/linux/raspbian stretch/stable armhf containerd.io armhf 1.2.13-2 [15.1 MB]
kubernetes-bootstrap: Get:2 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce-cli armhf 5:19.03.11~3-0~raspbian-stretch [28.2 MB]
kubernetes-bootstrap: Get:3 https://download.docker.com/linux/raspbian stretch/stable armhf docker-ce armhf 5:18.09.0~3-0~raspbian-stretch [10.8 MB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 54.0 MB in 36s (1,486 kB/s)
kubernetes-bootstrap: Selecting previously unselected package containerd.io.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 42843 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../containerd.io_1.2.13-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce-cli.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce-cli_5%3a19.03.11~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Selecting previously unselected package docker-ce.#015
kubernetes-bootstrap: Preparing to unpack .../docker-ce_5%3a18.09.0~3-0~raspbian-stretch_armhf.deb ...#015
kubernetes-bootstrap: Unpacking docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up containerd.io (1.2.13-2) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.#015
kubernetes-bootstrap: Setting up docker-ce-cli (5:19.03.11~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: Setting up docker-ce (5:18.09.0~3-0~raspbian-stretch) ...#015
kubernetes-bootstrap: update-alternatives: using /usr/bin/dockerd-ce to provide /usr/bin/dockerd (dockerd) in auto mode#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: docker-ce set on hold.
kubernetes-bootstrap: Disabling swap and ensuring it doesn't turn back on after reboot
kubernetes-bootstrap: Synchronizing state of dphys-swapfile.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install disable dphys-swapfile
kubernetes-bootstrap: Removed /etc/systemd/system/multi-user.target.wants/dphys-swapfile.service.
kubernetes-bootstrap: Warning: apt-key output should not be parsed (stdout is not a terminal)
kubernetes-bootstrap: OK
kubernetes-bootstrap: Installing kubernetes 1.16.1-00...
kubernetes-bootstrap: Hit:1 http://archive.raspberrypi.org/debian buster InRelease
kubernetes-bootstrap: Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
kubernetes-bootstrap: Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf Packages [34.7 kB]
kubernetes-bootstrap: Fetched 43.7 kB in 2s (23.9 kB/s)
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubernetes-cni socat
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   nftables
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   conntrack cri-tools kubeadm kubectl kubelet kubernetes-cni socat
kubernetes-bootstrap: 0 upgraded, 7 newly installed, 0 to remove and 110 not upgraded.
kubernetes-bootstrap: Need to get 47.8 MB of archives.
kubernetes-bootstrap: After this operation, 251 MB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://debian.bio.lmu.de/raspbian/raspbian buster/main armhf conntrack armhf 1:1.4.5-2 [28.5 kB]
kubernetes-bootstrap: Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf cri-tools armhf 1.13.0-00 [7,905 kB]
kubernetes-bootstrap: Get:3 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf socat armhf 1.7.3.2-2 [324 kB]
kubernetes-bootstrap: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubernetes-cni armhf 0.7.5-00 [5,661 kB]
kubernetes-bootstrap: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubelet armhf 1.16.1-00 [18.2 MB]
kubernetes-bootstrap: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubectl armhf 1.16.1-00 [8,005 kB]
kubernetes-bootstrap: Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main armhf kubeadm armhf 1.16.1-00 [7,664 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 47.8 MB in 15s (3,161 kB/s)
kubernetes-bootstrap: Selecting previously unselected package conntrack.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43072 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-conntrack_1%3a1.4.5-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package cri-tools.#015
kubernetes-bootstrap: Preparing to unpack .../1-cri-tools_1.13.0-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubernetes-cni.#015
kubernetes-bootstrap: Preparing to unpack .../2-kubernetes-cni_0.7.5-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package socat.#015
kubernetes-bootstrap: Preparing to unpack .../3-socat_1.7.3.2-2_armhf.deb ...#015
kubernetes-bootstrap: Unpacking socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubelet.#015
kubernetes-bootstrap: Preparing to unpack .../4-kubelet_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubectl.#015
kubernetes-bootstrap: Preparing to unpack .../5-kubectl_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Selecting previously unselected package kubeadm.#015
kubernetes-bootstrap: Preparing to unpack .../6-kubeadm_1.16.1-00_armhf.deb ...#015
kubernetes-bootstrap: Unpacking kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up conntrack (1:1.4.5-2) ...#015
kubernetes-bootstrap: Setting up kubectl (1.16.1-00) ...#015
kubernetes-bootstrap: Setting up socat (1.7.3.2-2) ...#015
kubernetes-bootstrap: Setting up cri-tools (1.13.0-00) ...#015
kubernetes-bootstrap: Setting up kubernetes-cni (0.7.5-00) ...#015
kubernetes-bootstrap: Setting up kubelet (1.16.1-00) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service.#015
kubernetes-bootstrap: Setting up kubeadm (1.16.1-00) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: kubelet set on hold.
kubernetes-bootstrap: kubectl set on hold.
kubernetes-bootstrap: kubeadm set on hold.
kubernetes-bootstrap: Pulling down all kubeadm images...
kubernetes-bootstrap: Installing keepalived 1:2.0.10-1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: I0604 22:13:13.126396    4445 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   lm-sensors snmp-mibs-downloader
kubernetes-bootstrap: Recommended packages:
kubernetes-bootstrap:   ipvsadm
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   keepalived libmariadb3 libsensors-config libsensors5 libsnmp-base libsnmp30
kubernetes-bootstrap:   mariadb-common mysql-common
kubernetes-bootstrap: 0 upgraded, 8 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 4,514 kB of archives.
kubernetes-bootstrap: After this operation, 8,563 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf mysql-common all 5.8+1.0.5 [7,324 B]
kubernetes-bootstrap: Get:2 http://mirror.dkm.cz/raspbian/raspbian buster/main armhf mariadb-common all 1:10.3.22-0+deb10u1 [31.9 kB]
kubernetes-bootstrap: Get:3 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf libmariadb3 armhf 1:10.3.22-0+deb10u1 [158 kB]
kubernetes-bootstrap: Get:4 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf libsensors-config all 1:3.5.0-3 [31.6 kB]
kubernetes-bootstrap: Get:5 http://ftp.fi.muni.cz/pub/linux/raspbian/raspbian buster/main armhf libsensors5 armhf 1:3.5.0-3 [49.5 kB]
kubernetes-bootstrap: Get:6 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf libsnmp-base all 5.7.3+dfsg-5 [1,595 kB]
kubernetes-bootstrap: Get:7 http://mirror.as43289.net/raspbian/raspbian buster/main armhf libsnmp30 armhf 5.7.3+dfsg-5+b1 [2,204 kB]
kubernetes-bootstrap: Get:8 http://raspbian.mirror.uk.sargasso.net/raspbian buster/main armhf keepalived armhf 1:2.0.10-1 [437 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 4,514 kB in 3s (1,695 kB/s)
kubernetes-bootstrap: Selecting previously unselected package mysql-common.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43140 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../0-mysql-common_5.8+1.0.5_all.deb ...#015
kubernetes-bootstrap: Unpacking mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: Selecting previously unselected package mariadb-common.#015
kubernetes-bootstrap: Preparing to unpack .../1-mariadb-common_1%3a10.3.22-0+deb10u1_all.deb ...#015
kubernetes-bootstrap: Unpacking mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libmariadb3:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../2-libmariadb3_1%3a10.3.22-0+deb10u1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors-config.#015
kubernetes-bootstrap: Preparing to unpack .../3-libsensors-config_1%3a3.5.0-3_all.deb ...#015
kubernetes-bootstrap: Unpacking libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsensors5:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../4-libsensors5_1%3a3.5.0-3_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp-base.#015
kubernetes-bootstrap: Preparing to unpack .../5-libsnmp-base_5.7.3+dfsg-5_all.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Selecting previously unselected package libsnmp30:armhf.#015
kubernetes-bootstrap: Preparing to unpack .../6-libsnmp30_5.7.3+dfsg-5+b1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Selecting previously unselected package keepalived.#015
kubernetes-bootstrap: Preparing to unpack .../7-keepalived_1%3a2.0.10-1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Setting up mysql-common (5.8+1.0.5) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libsnmp-base (5.7.3+dfsg-5) ...#015
kubernetes-bootstrap: Setting up libsensors-config (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up mariadb-common (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: update-alternatives: using /etc/mysql/mariadb.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode#015
kubernetes-bootstrap: Setting up libmariadb3:armhf (1:10.3.22-0+deb10u1) ...#015
kubernetes-bootstrap: Setting up libsensors5:armhf (1:3.5.0-3) ...#015
kubernetes-bootstrap: Setting up libsnmp30:armhf (5.7.3+dfsg-5+b1) ...#015
kubernetes-bootstrap: Setting up keepalived (1:2.0.10-1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /lib/systemd/system/keepalived.service.#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for dbus (1.12.16-1) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: keepalived set on hold.
kubernetes-bootstrap: Synchronizing state of keepalived.service with SysV service script with /lib/systemd/systemd-sysv-install.
kubernetes-bootstrap: Executing: /lib/systemd/systemd-sysv-install enable keepalived
kubernetes-bootstrap: Installing haproxy 1.8.19-1+rpi1...
kubernetes-bootstrap: Reading package lists...
kubernetes-bootstrap: Building dependency tree...
kubernetes-bootstrap: Reading state information...
kubernetes-bootstrap: The following additional packages will be installed:
kubernetes-bootstrap:   liblua5.3-0
kubernetes-bootstrap: Suggested packages:
kubernetes-bootstrap:   vim-haproxy haproxy-doc
kubernetes-bootstrap: The following NEW packages will be installed:
kubernetes-bootstrap:   haproxy liblua5.3-0
kubernetes-bootstrap: 0 upgraded, 2 newly installed, 0 to remove and 113 not upgraded.
kubernetes-bootstrap: Need to get 1,297 kB of archives.
kubernetes-bootstrap: After this operation, 2,753 kB of additional disk space will be used.
kubernetes-bootstrap: Get:1 http://mirror.as43289.net/raspbian/raspbian buster/main armhf liblua5.3-0 armhf 5.3.3-1.1 [94.4 kB]
kubernetes-bootstrap: Get:2 http://mirror1.hs-esslingen.de/pub/Mirrors/archive.raspbian.org/raspbian buster/main armhf haproxy armhf 1.8.19-1+rpi1 [1,203 kB]
kubernetes-bootstrap: dpkg-preconfigure: unable to re-open stdin: No such file or directory
kubernetes-bootstrap: Fetched 1,297 kB in 12s (112 kB/s)
kubernetes-bootstrap: Selecting previously unselected package liblua5.3-0:armhf.#015
kubernetes-bootstrap: (Reading database ... #015(Reading database ... 5%#015(Reading database ... 10%#015(Reading database ... 15%#015(Reading database ... 20%#015(Reading database ... 25%#015(Reading database ... 30%#015(Reading database ... 35%#015(Reading database ... 40%#015(Reading database ... 45%#015(Reading database ... 50%#015(Reading database ... 55%#015(Reading database ... 60%#015(Reading database ... 65%#015(Reading database ... 70%#015(Reading database ... 75%#015(Reading database ... 80%#015(Reading database ... 85%#015(Reading database ... 90%#015(Reading database ... 95%#015(Reading database ... 100%#015(Reading database ... 43378 files and directories currently installed.)#015
kubernetes-bootstrap: Preparing to unpack .../liblua5.3-0_5.3.3-1.1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Selecting previously unselected package haproxy.#015
kubernetes-bootstrap: Preparing to unpack .../haproxy_1.8.19-1+rpi1_armhf.deb ...#015
kubernetes-bootstrap: Unpacking haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: Setting up liblua5.3-0:armhf (5.3.3-1.1) ...#015
kubernetes-bootstrap: Setting up haproxy (1.8.19-1+rpi1) ...#015
kubernetes-bootstrap: Created symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /lib/systemd/system/haproxy.service.#015
kubernetes-bootstrap: Processing triggers for man-db (2.8.5-2) ...#015
kubernetes-bootstrap: Processing triggers for libc-bin (2.28-10+rpi1) ...#015
kubernetes-bootstrap: Processing triggers for rsyslog (8.1901.0-1) ...#015
kubernetes-bootstrap: Processing triggers for systemd (241-7~deb10u1+rpi1) ...#015
kubernetes-bootstrap: haproxy set on hold.
kubernetes-bootstrap: Detected as master node type, need to either join existing cluster or initialise new one
kubernetes-bootstrap: curl: (7) Failed to connect to 192.168.0.150 port 6443: Connection refused
kubernetes-bootstrap: Did not win VIP election, joining existing cluster.
kubernetes-bootstrap: The connection to the server localhost:8080 was refused - did you specify the right host or port?
kubernetes-bootstrap: Checking to see if control-plane has been fully initialised...
kubernetes-bootstrap: This could take up to 10 minutes...
kubernetes-bootstrap: [0] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.16.10
kubernetes-bootstrap: [1] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [2] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.16.10
kubernetes-bootstrap: [3] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.16.10
kubernetes-bootstrap: [4] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [5] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [6] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [7] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/kube-proxy:v1.16.10
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/pause:3.1
kubernetes-bootstrap: [8] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [9] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [10] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [11] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [12] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [13] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [14] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [15] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/etcd:3.3.15-0
kubernetes-bootstrap: [16] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [config/images] Pulled k8s.gcr.io/coredns:1.6.2
kubernetes-bootstrap: [17] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [18] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [19] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [20] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [21] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: [22] Cluster still not up, sleeping for 10 seconds
kubernetes-bootstrap: Kubernetes control-plane has finished initalising!
kubernetes-bootstrap: Attempting to join cluster!
kubernetes-bootstrap: I0604 22:17:52.181389    8554 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [upload-certs] Using certificate key:
kubernetes-bootstrap: 2604565aecedd3c31d4449a53768016ede3f8776a06e2a08708a3ce3f64b6624
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: error execution phase control-plane-prepare/download-certs: error downloading certs: error decoding secret data with provided key: cipher: message authentication failed
kubernetes-bootstrap: To see the stack trace of this error execute with --v=5 or higher
kubernetes-bootstrap: I0604 22:18:10.135799    8977 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16
kubernetes-bootstrap: [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [upload-certs] Using certificate key:
kubernetes-bootstrap: 8dda5d67def40339eca4e2897044add56cd9fbf20dc8d146b561ac51a7a17ff0
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
kubernetes-bootstrap: [preflight] Running pre-flight checks
kubernetes-bootstrap: [preflight] Reading configuration from the cluster...
kubernetes-bootstrap: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
kubernetes-bootstrap: [preflight] Running pre-flight checks before initializing the new control plane instance
kubernetes-bootstrap: [preflight] Pulling images required for setting up a Kubernetes cluster
kubernetes-bootstrap: [preflight] This might take a minute or two, depending on the speed of your internet connection
kubernetes-bootstrap: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
kubernetes-bootstrap: [certs] Using certificateDir folder "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Generating "etcd/healthcheck-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver-etcd-client" certificate and key
kubernetes-bootstrap: [certs] Generating "etcd/peer" certificate and key
kubernetes-bootstrap: [certs] etcd/peer serving cert is signed for DNS names [rpi-kube-master-01 localhost] and IPs [192.168.0.151 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "etcd/server" certificate and key
kubernetes-bootstrap: [certs] etcd/server serving cert is signed for DNS names [rpi-kube-master-01 localhost] and IPs [192.168.0.151 127.0.0.1 ::1]
kubernetes-bootstrap: [certs] Generating "apiserver-kubelet-client" certificate and key
kubernetes-bootstrap: [certs] Generating "apiserver" certificate and key
kubernetes-bootstrap: [certs] apiserver serving cert is signed for DNS names [rpi-kube-master-01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.151 192.168.0.150 192.168.0.150]
kubernetes-bootstrap: [certs] Generating "front-proxy-client" certificate and key
kubernetes-bootstrap: [certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
kubernetes-bootstrap: [certs] Using the existing "sa" key
kubernetes-bootstrap: [kubeconfig] Generating kubeconfig files
kubernetes-bootstrap: [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
kubernetes-bootstrap: [kubeconfig] Writing "admin.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "controller-manager.conf" kubeconfig file
kubernetes-bootstrap: [kubeconfig] Writing "scheduler.conf" kubeconfig file
kubernetes-bootstrap: [control-plane] Using manifest folder "/etc/kubernetes/manifests"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-apiserver"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-controller-manager"
kubernetes-bootstrap: [control-plane] Creating static Pod manifest for "kube-scheduler"
kubernetes-bootstrap: [check-etcd] Checking that the etcd cluster is healthy
kubernetes-bootstrap: [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
kubernetes-bootstrap: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
kubernetes-bootstrap: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
kubernetes-bootstrap: [kubelet-start] Activating the kubelet service
kubernetes-bootstrap: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
kubernetes-bootstrap: [kubelet-check] Initial timeout of 40s passed.
kubernetes-bootstrap: error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition
kubernetes-bootstrap: To see the stack trace of this error execute with --v=5 or higher

In these error messages you see again the message:

kubernetes-bootstrap: I0604 22:17:52.181389    8554 version.go:251] remote version is much newer: v1.18.3; falling back to: stable-1.16

but also this error:

kubernetes-bootstrap: error execution phase control-plane-prepare/download-certs: error downloading certs: error decoding secret data with provided key: cipher: message authentication failed

That is quite weird why it would throw that error message...
What I found is an old post that people had the same issue and it was fixed by adding the --config param too:
kubernetes/kubeadm#1886 (comment)

What do you think?

@blabla1337
Copy link
Contributor Author

I really have no clue anymore, I tried on other thing some people had issues with the kubeadm and for them it got fixed when running with sudo. So I tried it and now its booting again, all masters are up

init_master() {
  # initialise kubernetes cluster
  sudo kubeadm init \

kubernetes/kubeadm#1438 (comment)

@lucasteligioridis
Copy link
Owner

What the?
The bootstrap scripts are running as root?

Was that with the new version of docker or old?

@lucasteligioridis
Copy link
Owner

Sounds like a bug somewhere in kubeadm!

@Bujail
Copy link

Bujail commented Jun 5, 2020

Facing similar issue.

W0605 06:23:12.163394 570 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... [kubelet-check] Initial timeout of 40s passed. error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition To see the stack trace of this error execute with --v=5 or higher

Docker version:

Client:
 Version:           18.09.7
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        2d0083d
 Built:             Thu Jun 27 17:56:06 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:26:28 2019
  OS/Arch:          linux/amd64
  Experimental:     false

kubeadm version

kubeadm version: &version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.5", GitCommit:"e0fccafd69541e3750d460ba0f9743b90336f24f", GitTreeState:"clean", BuildDate:"2020-04-16T11:41:38Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

But kubelet is running and its joining the cluster.

@blabla1337
Copy link
Contributor Author

What the?
The bootstrap scripts are running as root?

Was that with the new version of docker or old?

For me I used the old one with the export docker api fix and running the sudo kubeadm init

@lucasteligioridis
Copy link
Owner

I still have no idea how the sudo made any difference.
If we're running as root that should have been a negligible change.

Super super interesting. I mean we can patch it with that and see if that works, but really weird.

@blabla1337
Copy link
Contributor Author

well indeed I also didn't get it but gave it a go and worked, so for the old version of docker + the docker env fix API + making the config sudo will now work again.

For using the new docker version, it raises other errors and I didn't know why and how to solve that...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants