From 8ed2daaf98d1a98873fe4a7982adf145b83affad Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Fri, 17 May 2024 10:46:39 -0600 Subject: [PATCH 1/6] Add user guide on how to schedule backups with vtop Signed-off-by: Florent Poinsard --- .../configuration-advanced/resharding.md | 5 + .../backup-and-restore/scheduled-backups.md | 98 +++++++++++++++++++ 2 files changed, 103 insertions(+) create mode 100644 content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md diff --git a/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md b/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md index 0b23ddb33..5fc5801c1 100644 --- a/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md +++ b/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md @@ -361,3 +361,8 @@ Beyond this, you will also want to manually delete the on-disk directories assoc ```bash rm -rf ${VTDATAROOT}/vt_000000020{0,1,2}/ ``` + +## Next Steps + +Congratulations, you have successfully resharded your `customer` keyspace in two shards. +Now, let's learn [how to schedule backups](../../operating-vitess/backup-and-restore/scheduled-backups) of your Vitess custer. diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md new file mode 100644 index 000000000..0b3a260be --- /dev/null +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -0,0 +1,98 @@ +--- +title: Scheduled Backups +weight: 5 +aliases: ['/docs/user-guides/backup-and-restore/'] +--- + +{{< info >}} +This guide follows on from the Get Started guides. Please make sure that you have +an [Operator](../../../../get-started/operator) or [local](../../../../get-started/local) installation ready. It also assumes +that the [MoveTables](../../../migration/move-tables/) and [Resharding](../../../configuration-advanced/resharding) user guides have been followed (which take you through +steps `101` to `306`). + +This step is doable only if you are using the vitess-operator. +{{< /info >}} + +## Backups + +If you are not familiar with how backups in Vitess work we suggest you familiarize yourself with it. +An [entire guide](../overview/) is available in this section. + +## Scheduling backups + +### Adding the schedule + +For this example we are going to create two schedules: each will be executed every minute, the first one will backup +the two `customer` shards, and the second one will backup the `commerce` keyspace. +The backups will be stored directly inside the Minikube node, but if you want to backup to for instance S3, you can +change the `location` of the backup in `401_scheduled_backups.yaml`. + +``` +kubectl apply -f 401_scheduled_backups.yaml +``` + +After at least a minute, we should appear at least two pods that are created by the `VitessBackupSchedule` object. +These pods will take a backup of vitess. + +``` +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +... +example-vbsc-every-minute-commerce-ac6ff735-1715963880-4rt6r 0/1 Completed 0 31s +example-vbsc-every-minute-customer-8aaaa771-1715963880-n7cm7 0/1 Completed 0 31s +... +``` + +### Listing backups + +Now we can list the available backups, by getting the `vtb` (`VitessBackup`) objects in our Kubernetes cluster. +We can see we have three backups, that is because the schedule `every-minute-customer` takes two backups (one for each shard). + +``` +$ kubectl get vtb --no-headers +example-commerce-x-x-20240517-163802-97ece60f-8f2a3d47 111s +example-customer-80-x-20240517-163812-70e40ad-78a0d60b 58s +example-customer-x-80-20240517-163803-729301e-02e8899b 111s +``` + +Since we are running on Minikube, our backups are stored locally on the Minikube node, we can take a look at them like so: + +``` +$ minikube ssh +docker@minikube:~$ cd /tmp/example/ +docker@minikube:/tmp/example$ ls +commerce customer +docker@minikube:/tmp/example$ ls commerce/-/ customer/ +-80/ 80-/ +docker@minikube:/tmp/example$ ls -l commerce/-/ customer/-80/ customer/80-/ +commerce/-/: +total 0 +drwxr-xr-x 2 999 docker 220 May 17 16:38 2024-05-17.163802.zone1-2548885007 +drwxr-xr-x 2 999 docker 220 May 17 16:39 2024-05-17.163903.zone1-2548885007 +drwxr-xr-x 2 999 docker 220 May 17 16:40 2024-05-17.164003.zone1-2548885007 +drwxr-xr-x 2 999 docker 220 May 17 16:41 2024-05-17.164102.zone1-2548885007 +drwxr-xr-x 2 999 docker 220 May 17 16:42 2024-05-17.164202.zone1-2548885007 + +customer/-80/: +total 0 +drwxr-xr-x 2 999 docker 220 May 17 16:38 2024-05-17.163803.zone1-0120139806 +drwxr-xr-x 2 999 docker 220 May 17 16:39 2024-05-17.163902.zone1-2289928654 +drwxr-xr-x 2 999 docker 220 May 17 16:40 2024-05-17.164003.zone1-2289928654 +drwxr-xr-x 2 999 docker 220 May 17 16:41 2024-05-17.164102.zone1-2289928654 +drwxr-xr-x 2 999 docker 220 May 17 16:42 2024-05-17.164202.zone1-2289928654 + +customer/80-/: +total 0 +drwxr-xr-x 2 999 docker 220 May 17 16:38 2024-05-17.163812.zone1-0118374573 +drwxr-xr-x 2 999 docker 220 May 17 16:39 2024-05-17.163911.zone1-4277914223 +drwxr-xr-x 2 999 docker 220 May 17 16:40 2024-05-17.164010.zone1-4277914223 +drwxr-xr-x 2 999 docker 220 May 17 16:41 2024-05-17.164111.zone1-4277914223 +drwxr-xr-x 2 999 docker 220 May 17 16:42 2024-05-17.164208.zone1-4277914223 +docker@minikube:/tmp/example$ +``` + +### Cleanup + +Congratulations, you have correctly scheduled recurrent backups of your Vitess cluster. + +If you want, you can now clean up the entire cluster by running: `minikube delete`. From 2d1c63c641756c50570af7c369e093b808b8f787 Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Fri, 17 May 2024 11:07:53 -0600 Subject: [PATCH 2/6] Add bash instruction to code blocks Signed-off-by: Florent Poinsard --- .../backup-and-restore/scheduled-backups.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md index 0b3a260be..d29e4fb21 100644 --- a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -27,14 +27,14 @@ the two `customer` shards, and the second one will backup the `commerce` keyspac The backups will be stored directly inside the Minikube node, but if you want to backup to for instance S3, you can change the `location` of the backup in `401_scheduled_backups.yaml`. -``` +```bash kubectl apply -f 401_scheduled_backups.yaml ``` After at least a minute, we should appear at least two pods that are created by the `VitessBackupSchedule` object. These pods will take a backup of vitess. -``` +```bash $ kubectl get pods NAME READY STATUS RESTARTS AGE ... @@ -48,7 +48,7 @@ example-vbsc-every-minute-customer-8aaaa771-1715963880-n7cm7 0/1 Completed Now we can list the available backups, by getting the `vtb` (`VitessBackup`) objects in our Kubernetes cluster. We can see we have three backups, that is because the schedule `every-minute-customer` takes two backups (one for each shard). -``` +```bash $ kubectl get vtb --no-headers example-commerce-x-x-20240517-163802-97ece60f-8f2a3d47 111s example-customer-80-x-20240517-163812-70e40ad-78a0d60b 58s @@ -57,7 +57,7 @@ example-customer-x-80-20240517-163803-729301e-02e8899b 111s Since we are running on Minikube, our backups are stored locally on the Minikube node, we can take a look at them like so: -``` +```bash $ minikube ssh docker@minikube:~$ cd /tmp/example/ docker@minikube:/tmp/example$ ls From c79664809db6da3c8a7ea3a61ab719850eec226f Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Fri, 17 May 2024 11:10:04 -0600 Subject: [PATCH 3/6] Be more clear about why we are expecting to see new pods Signed-off-by: Florent Poinsard --- .../operating-vitess/backup-and-restore/scheduled-backups.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md index d29e4fb21..bad4dcd80 100644 --- a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -31,8 +31,9 @@ change the `location` of the backup in `401_scheduled_backups.yaml`. kubectl apply -f 401_scheduled_backups.yaml ``` -After at least a minute, we should appear at least two pods that are created by the `VitessBackupSchedule` object. -These pods will take a backup of vitess. +After at least a minute, we should see two new pods that were created by the operator. Under the hood, these pods +are managed by a Kubernetes Job, and their goal is to take a backup of Vitess, as we defined in the `strategies` field +of the `401_scheduled_backups.yaml` file. ```bash $ kubectl get pods From 117d61270d81a4151d50b9bb2528537b8f72414a Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Fri, 17 May 2024 11:11:27 -0600 Subject: [PATCH 4/6] Rephrase the listing backups section Signed-off-by: Florent Poinsard --- .../operating-vitess/backup-and-restore/scheduled-backups.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md index bad4dcd80..650801657 100644 --- a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -47,7 +47,8 @@ example-vbsc-every-minute-customer-8aaaa771-1715963880-n7cm7 0/1 Completed ### Listing backups Now we can list the available backups, by getting the `vtb` (`VitessBackup`) objects in our Kubernetes cluster. -We can see we have three backups, that is because the schedule `every-minute-customer` takes two backups (one for each shard). +We can see we have three backups, that is because the schedule `every-minute-customer` takes two backups (one for each shard, `-80` and `80-`), +and the other schedule (`every-minute-commerce`) takes only one backup (for the shard `-`). ```bash $ kubectl get vtb --no-headers From 4d34010c2ec076b9a26ca5b068a2695d0ee527bb Mon Sep 17 00:00:00 2001 From: Florent Poinsard <35779988+frouioui@users.noreply.github.com> Date: Mon, 3 Jun 2024 14:59:32 -0600 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Deepthi Sigireddi Signed-off-by: Florent Poinsard <35779988+frouioui@users.noreply.github.com> --- .../user-guides/configuration-advanced/resharding.md | 4 ++-- .../backup-and-restore/scheduled-backups.md | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md b/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md index 5fc5801c1..1c68be9e7 100644 --- a/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md +++ b/content/en/docs/20.0/user-guides/configuration-advanced/resharding.md @@ -364,5 +364,5 @@ rm -rf ${VTDATAROOT}/vt_000000020{0,1,2}/ ## Next Steps -Congratulations, you have successfully resharded your `customer` keyspace in two shards. -Now, let's learn [how to schedule backups](../../operating-vitess/backup-and-restore/scheduled-backups) of your Vitess custer. +Congratulations, you have successfully resharded your `customer` keyspace into two shards. +Now, let's learn [how to schedule backups](../../operating-vitess/backup-and-restore/scheduled-backups) of your Vitess cluster. diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md index 650801657..22db89430 100644 --- a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -10,12 +10,12 @@ an [Operator](../../../../get-started/operator) or [local](../../../../get-start that the [MoveTables](../../../migration/move-tables/) and [Resharding](../../../configuration-advanced/resharding) user guides have been followed (which take you through steps `101` to `306`). -This step is doable only if you are using the vitess-operator. +This guide is useful only if you are using the vitess-operator. {{< /info >}} ## Backups -If you are not familiar with how backups in Vitess work we suggest you familiarize yourself with it. +If you are not already familiar with how backups work in Vitess we suggest you familiarize yourself with them first. An [entire guide](../overview/) is available in this section. ## Scheduling backups @@ -24,14 +24,14 @@ An [entire guide](../overview/) is available in this section. For this example we are going to create two schedules: each will be executed every minute, the first one will backup the two `customer` shards, and the second one will backup the `commerce` keyspace. -The backups will be stored directly inside the Minikube node, but if you want to backup to for instance S3, you can +The backups will be stored directly inside the Minikube node, but if you want to backup to a cloud storage provider like S3, you can change the `location` of the backup in `401_scheduled_backups.yaml`. ```bash kubectl apply -f 401_scheduled_backups.yaml ``` -After at least a minute, we should see two new pods that were created by the operator. Under the hood, these pods +After a minute or so, we should see two new pods that were created by the operator. Under the hood, these pods are managed by a Kubernetes Job, and their goal is to take a backup of Vitess, as we defined in the `strategies` field of the `401_scheduled_backups.yaml` file. @@ -95,6 +95,6 @@ docker@minikube:/tmp/example$ ### Cleanup -Congratulations, you have correctly scheduled recurrent backups of your Vitess cluster. +Congratulations, you have correctly scheduled recurring backups of your Vitess cluster. If you want, you can now clean up the entire cluster by running: `minikube delete`. From 5a977a20e0b863ab5bb88df8123e6e7fb106cc24 Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Mon, 3 Jun 2024 15:02:42 -0600 Subject: [PATCH 6/6] Apply review suggestions Signed-off-by: Florent Poinsard --- .../backup-and-restore/scheduled-backups.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md index 22db89430..13caee869 100644 --- a/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md +++ b/content/en/docs/20.0/user-guides/operating-vitess/backup-and-restore/scheduled-backups.md @@ -6,7 +6,7 @@ aliases: ['/docs/user-guides/backup-and-restore/'] {{< info >}} This guide follows on from the Get Started guides. Please make sure that you have -an [Operator](../../../../get-started/operator) or [local](../../../../get-started/local) installation ready. It also assumes +an [Operator](../../../../get-started/operator) installation ready. It also assumes that the [MoveTables](../../../migration/move-tables/) and [Resharding](../../../configuration-advanced/resharding) user guides have been followed (which take you through steps `101` to `306`). @@ -15,13 +15,16 @@ This guide is useful only if you are using the vitess-operator. ## Backups -If you are not already familiar with how backups work in Vitess we suggest you familiarize yourself with them first. -An [entire guide](../overview/) is available in this section. +If you are not already familiar with [how backups work](../overview/) in Vitess we suggest you familiarize yourself with them first. ## Scheduling backups ### Adding the schedule +{{< warning >}} +Please note that is not recommended to run production backups every minute. These schedules are only an example. +{{}} + For this example we are going to create two schedules: each will be executed every minute, the first one will backup the two `customer` shards, and the second one will backup the `commerce` keyspace. The backups will be stored directly inside the Minikube node, but if you want to backup to a cloud storage provider like S3, you can