Kubernetes Operator Support

[vpatil-tibco]

Context
Need support for Kubernetes Operator. Following are the highlights around what this operator should support at the minimum,

1. Ability to dynamically scale BE nodes (cache/inference) based on resource (memory/cpu) thresholds.
2. Ability to distribute BE nodes evenly across resources (hardware/boxes/etc). Should apply to both scale up and scale down.
3. Ability to replace dead/unrecoverable nodes.
4. Ability to update Replication count.
5. Ability to perform Back/Restore via Velero.

Alternatives
No alternatives/workarounds today.

Has the feature been requested before?
No

If the feature request is approved, would you be willing to submit a PR?
Yes

[rameshpolishetti]

All listed asks are feasible using Kubernetes OOB capabilities and currently made available using helm chart.

Details:

• Point 1: Made available using helm chart using K8S Horizontal Pod Auto scale feature. It can be enabled/disabled through values.yaml
• Point 2: Made available using helm chart using K8S Affinity/Anti Affinity feature. It can be enabled/disabled through values.yaml
• Point 3 & 4: Made available using helm chart using K8S StatefulSet replica count. It can be controlled through values.yaml (Agents replicas)
• Point 5: User can always perform Backup/Restore using Velero directly.

Operator can be added in the future as required.