gen2-v20230915.0

• Support VXLAN on GCP (as well as on AWS, Azure, etc.) (#300)
• Support Rocky Linux 9 and AlmaLinux 9 hosts (#301)
• Move init-host scripts out of the hack directory (#302)
• Add make kubectl (#303)

gen2-v20230906.0

This is the first release of "Usernetes Generation 2" (#287)

---

Usernetes: Kubernetes without the root privileges (Generation 2)

Usernetes (Gen2) deploys a Kubernetes cluster inside Rootless Docker,
so as to mitigate potential container-breakout vulnerabilities.

Note

Usernetes (Gen2) has significantly diverged from the original Usernetes (Gen1),
which did not require Rootless Docker to be installed on hosts.

See the gen1 branch for
the original Usernetes (Gen1).

Usernetes (Gen2) is similar to Rootless kind and Rootless minikube,
but Usernetes (Gen 2) supports creating a cluster with multiple hosts.

Components

• Cluster configuration: kubeadm
• CRI: containerd
• OCI: runc
• CNI: Flannel

Requirements

Note

Using Ubuntu 22.04 hosts is recommended.

• Rootless Docker:

curl -o install.sh -fsSL https://get.docker.com
sudo sh install.sh
dockerd-rootless-setuptool.sh install

• systemd lingering:

sudo loginctl enable-linger $(whoami)

• cgroup v2 delegation:

sudo mkdir -p /etc/systemd/system/user@.service.d

cat <<EOF | sudo tee /etc/systemd/system/user@.service.d/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF

sudo systemctl daemon-reload

• Kernel modules:

sudo modprobe vxlan

Usage

See make help.

# Bootstrap a cluster
make up
make kubeadm-init
make install-flannel

# Enable kubectl
make kubeconfig
export KUBECONFIG=$(pwd)/kubeconfig
kubectl get pods -A

# Multi-host
make join-command
scp join-command another-host:~/usernetes
ssh another-host make -C ~/usernetes up kubeadm-join

# Debug
make logs
make shell
make down-v
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Limitations

• Node ports cannot be exposed automatically. Edit docker-compose.yaml for exposing additional node ports.
• Most of host files are not visible with hostPath mounts. Edit docker-compose.yaml for mounting additional files.
• Some volume drivers such as nfs do not work.

Advanced topics

• Although Usernetes (Gen2) is designed to be used with Rootless Docker, it should work with the regular "rootful" Docker too.
  This might be useful for some people who are looking for "multi-host" version of kind and minikube. (Support for "rootful" is currently broken: #297)

v20230816.0

Kubernetes version: v1.28.0

---

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/actions/runs/5874550853/job/15929481770

v20230518.0

Kubernetes version: v1.27.2

Thanks to @cloud-66 for #279

---

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/actions/runs/5009284196/jobs/8978057689

v20221007.0

Kubernetes version: v1.25.2

---

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/actions/runs/3204008677/jobs/5234781877

v20220506.0

Kubernetes version: v1.24.0

---

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/runs/6316210320

v20211108.0

• Dropped support for cgroup v1. Now cgroup v2 is an essential requirement. (#237)
• kube-proxy: changed from userspace mode to iptables mode. Requires several kernel modules such as br_netfilter to be loaded. (#238)
• Install CoreDNS by default (#247)

---

Kubernetes version: v1.23.0-alpha.4

---

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/runs/4138131870

v20210708.0

Notable changes:

• Remove most of the patches + deprecate "none" cgroup driver (#234)

---

⚠️ The support for cgroup v1 will be removed in the next release. Upgrading to cgroup v2 is highly recommended.

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/runs/3017128248

v20210624.0

Notable changes:

• Moved the image from Docker Hub to GHCR: ghcr.io/rootless-containers/usernetes

---

⚠️ The rootlesscontainers/usernetes image on Docker Hub will no longer be updated, due to the latest policy change of Docker Hub: https://www.docker.com/blog/changes-to-docker-hub-autobuilds/

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/runs/2903580251

v20210601.0

Notable changes:

• Updated the patch set to follow the latest KEP discussion (introduced KubeletInUserNamespace feature gate): #228 #229
• Improve 'hostname' compatibility : #224, thanks to @Silvanoc

Build logs (available for 90 days): https://github.com/rootless-containers/usernetes/runs/2715722221