Skip to content

Commit

Permalink
Merge pull request karmada-io#4131 from XiShanYongYe-Chang/refine-res…
Browse files Browse the repository at this point in the history
…ource-deletion-protection-proposal

Refine resource deletion protection proposal
  • Loading branch information
karmada-bot authored Oct 16, 2023
2 parents f56af66 + dcc0577 commit 575d4d9
Showing 1 changed file with 80 additions and 48 deletions.
128 changes: 80 additions & 48 deletions docs/proposals/resource-deletion-protection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,43 +19,112 @@ creation-date: 2023-07-17

## Summary

In the Karmada control plane, we require a resource protection feature. Once users specify the resources to be protected (such as Deployment, StatefulSet, Service, Namespace, and so on), if any of these protected resources are deleted, we need to prevent this operation.
In the Karmada control plane, we require a resource protection feature. Once the user specify that the resources (such as Deployment, StatefulSet, Service, Namespace, and so on) needs to be protected, the request will be rejected when the user attempts to delete them.

## Motivation

This feature is highly necessary, as the mistaken deletion of a resource in the control plane could potentially lead to severe consequences.
In order to prevent catastrophic effects when users delete resources, such as Namespace or Cluster resources, on the Karmada control plane due to misuse, we need to provide a resource deletion protection mechanism.

### Goals

- Prevent the user's deletion operation if the resource is protected.

## Protected resources

- CRD
- CustomResourceDefinitions(CRD)
- Kubernetes resources
- Resource extend by [Aggregate Layer](https://kubernetes.io/zh-cn/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)
- Resource extend by [Kubernetes API Aggregate Layer](https://kubernetes.io/zh-cn/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)

## Proposal

### User Story

#### Story 1: Protect the namespace

As a manager, I want to protect my `minio` namespace, so that they won't be accidentally deleted.
As an end user, I want to protect my `minio` namespace, so that they won't be accidentally deleted.

![user-story-1](statics/user-story-1.drawio.svg)

#### Story 2: Protect the whole application

As a manager, I want to protect my `MySQL` application, so that they won't be accidentally deleted.
As an end user, I want to protect my `MySQL` application, so that they won't be accidentally deleted.

Our users may need to provide further protection for certain resources, not limited to just `Namespace`. They might want to secure resources such as `Deployment`,`StatefulSet`,`Service` and so on, for example, **all resources related to `MySQL` application**.

### Design Details

We can add a CRD `ResourceDeletionProtection` to specify the resources which should be protected.
Add a new label `resourcetemplate.karmada.io/deletion-protected` with enumerated values including `Always`. Users can set this label for the resource to be protected. When the label value is set to `Always`, the user performs a delete operation, and `karmada-webhook` will reject the request, thus preventing the target resource form being deleted.

#### CRD Example
#### New label added

```go
// ResourceDeletionProtectionLabelKey is the label key used to add to the resource to
// indicate the deletion protection policy of the resource.
ResourceDeletionProtectionLabelKey = "resourcetemplate.karmada.io/deletion-protected"
// ResourceDeletionProtectionAlways indicate that deletion protection is always performed.
ResourceDeletionProtectionAlways = "Always"
```

#### Webhook Configuration

The ValidatingWebhookConfiguration resource can be configured so the karmada-webhook only focuses on resource deletion operations and the resource labels include `resourcetemplate.karmada.io/deletion-protected`.

```yaml
rules:
- operations: ["DELETE"]
apiGroups: ["*"]
apiVersions: ["*"]
resources: ["*"]
scope: "*"
objectSelector:
matchExpressions:
- { key: "resourcetemplate.karmada.io/deletion-protected", operator: "Exists" }
```
In the karmada-webhook implementation of the corresponding route, we can directly get the value of the key `resourcetemplate.karmada.io/deletion-protected` label in the resource without decoding the request object, and if the value is `Always`, the user's delete operation is rejected, otherwise it will be passed.

#### What users need to do

Users can add this label to a resources to set deletion protection on the target resource with the following command:

```
kubectl label <resource-type> <resource-name> resourcetemplate.karmada.io/deletion-protected=Always
```

When users no longer wants to protect the protected resource, the following command is available:

```
kubectl label <resource-type> <resource-name> resourcetemplate.karmada.io/deletion-protected-
```

Alternatively, users can modify the resource directly using the `kubectl edit` command.

#### Pros and Cons

##### Pros

- Simplicity of solution, simplicity of realization.

##### Cons

- Users need to process the label one by one for the resources to be protected.
- The permission management is scattered, unable to follow the resources for permission control and unified handling.

### Q&A

#### Why choose Label over Annotations to tag resources?

Annotations are typically used to store metadata that is not used for identification and selection, such as descriptive information or the status information of management tools. In this case, labels like "cannot be deleted" or "protected status" are not suitable as annotations.

#### Whether resource deletion protection needs to be enabled by default

If subsequently needed, we can add a feature gate to enable deletion protection by default for all resources.

## Alternatives

Add a new CRD `ResourceDeletionProtection` to specify the resources which should be protected.

### CRD Example

Resource Scope: Cluster

Expand Down Expand Up @@ -83,13 +152,14 @@ spec:
- "service-1"
```

#### Webhook Implement Detail
### Webhook Implement Detail

Whenever we receive a `DELETE` request from a user/manager, our webhook will use an informer to list all `ResourceDeletionProtection` resources to check if the resource is protected. If it is, the operation will be prevented.

### Pros and Cons

#### Pros

- No need for add label to resources.
- Centralized permission control.
- More comprehensive presentation to users.

Expand All @@ -112,44 +182,6 @@ I believe it's unnecessary. If `ResourceDeletionProtection` is not created, it c

We may not be able to distinguish between the force delete and normal delete operations, so for force delete, we will still follow the original logic and prevent users from deleting the resource.

## Alternatives

### Implemented by labels

**User** can add a label `karmada.io/deletion-protected=always` to the resource that needs protection.

If a resource that a user attempts to delete with the label `karmada.io/deletion-protected=always` with a value of `always`, then we will prevent its operation through the webhook.

#### Add the protected tag

`kubectl label <resource-type> <resource-name> karmada.io/deletion-protected=always`

`karmadactl protect <resource-type> <resource-name>`

#### Remove the protected tag

`kubectl label <resource-type> <resource-name> karmada.io/deletion-protected=none`

`karmadactl unprotect <resource-type> <resource-name>`

#### Why choose Label over Annotations to tag resources?

Annotations are typically used to store metadata that is not used for identification and selection, such as descriptive information or the status information of management tools. In this case, labels like "cannot be deleted" or "protected status" are not suitable as annotations.

#### Enable protection by default (imagine)

We can add a switch to enable deletion protection by default for all resources, if user want to delete the resources, he must remove the label first.

#### Pros

- Easy to implement

#### Cons

- The user needs to add/remove tags to resources one by one.
- The permission management is scattered, unable to follow the resources for permission control and unified handling.
- A label is not a runtime state, and in certain situations, labels might be removed (such as when a controller is deleted and then the resource is re-created).

## Test Plan

### UT
Expand Down

0 comments on commit 575d4d9

Please sign in to comment.