Skip to content

Latest commit

 

History

History
31 lines (27 loc) · 2.05 KB

README.md

File metadata and controls

31 lines (27 loc) · 2.05 KB
Raw

borg-k8s-volume-backup

Backing up kubernetes volumes on a node using borg backup

This repo will soon contain a docker image and kubernetes resource YAMLs for backing up kubernetes volumes in a cluster

Basic concept

  • Kubelet mounts pod volumes as /var/lib/kubelet/pods/<pod_uuid>/volumes/<volume_type>/<volume_name>
    • NOTE! volume_name here refers to the "VOLUME" attribute listed in kubectl get pvc output, NOT the "NAME" attribute
  • By iterating over /var/lib/kubelet/pods/*/volumes/*/*, we can run operations on each mounted volume on a k8s node
    • For example, by iterating over /var/lib/kubelet/pods/*/volumes/kubernetes.io~iscsi/*, we can back up each mounted iSCSI volume.

Short-term Goals

  • Done, see k8s-volume-backup.py - requires python3
    • Generic script that uses env variables for volume type, borg repo location, and borg passphrase
    • Uses kubectl get pvc in backup script to get all PVCs, and build a volume name to PVC name dictionary
    • Uses this dictionary to save backups using the PVC name rather than the volume name
      • The PVC name is typically specified by the user in a PVC resource YAML
      • The volume name may be automatically generated by a dynamic volume provisioner
      • Goal is to have archives in borg repo named something like sonarr-2020-03-01 rather than pvc-b1397318-8fa4-4216-aa6b-6568d1394e89-2020-03-01
      • Archive contents will be a 1:1 reproduction of the PVC storage contents at time of archive creation
  • TBD:
    • Generic script to restore a borg backup to a k8s volume

End goals

  • Create a docker image that can do backup/restory operations
    • Mount /var/lib/kubelet/pods/ using hostPath
    • Run backup script in container to back up all mounted PVCs on the kubernetes node
  • Create a DaemonSet, so that one can easily back up all PVCs in a cluster by running the container on each node
    • Maybe even a DaemonJob, if that gets merged.
  • Create a docker container that can do restore operations
  • Create a Job template file that can be used to easily restore a borg backup archive to a PVC