diff --git a/README.md b/README.md index 35c580a9..da503197 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,6 @@ * [Developer Preview](#developer-preview) * [Introduction](#introduction) * [Environment Preparation](#environment-preparation) - * [Create a Local Kubernetes Cluster](#create-a-local-kubernetes-cluster) * [Install Helm on Kubernetes](#install-helm-on-kubernetes) * [Run Confluent Platform](#run-confluent-platform) * [Install cp-helm-charts](#install-cp-helm-charts) @@ -14,6 +13,8 @@ * [Monitoring](#monitoring) * [Teardown](#teardown) * [Thanks](#thanks) +* [Appendix](#appendix) + * [Create a Local Kubernetes Cluster](#create-a-local-kubernetes-cluster) ## Developer Preview @@ -49,82 +50,9 @@ These Helm charts have been tested with the following software versions: * [Kubernetes](https://kubernetes.io/) 1.9.2+ * [Helm](https://helm.sh/) 2.8.2+ -* [Confluent Platform Open Source Docker Images](https://hub.docker.com/u/confluentinc/) 4.1.1 - -### Create a Local Kubernetes Cluster - -There are many deployment options to get set up with a Kubernetes cluster, and this document provides instructions for using [Minikube](https://kubernetes.io/docs/setup/minikube/) to set up a local Kubernetes cluster. -Minikube runs a single-node Kubernetes cluster inside a VM on your laptop. - -You may alternatively set up a Kubernetes cluster in the cloud using other providers such as [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/quickstart). - -#### Install Minikube and Drivers - -Minikube version 0.23.0 or higher is required for docker server https://github.com/moby/moby/pull/31352[17.05], which adds support for using `ARG` in `FROM` in your `Dockerfile`. - -First follow the basic [Minikube installation instructions](https://github.com/kubernetes/minikube). - -Then install the [Minikube drivers](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md). Minikube uses Docker Machine to manage the Kubernetes VM so it benefits from the driver plugin architecture that Docker Machine uses to provide a consistent way to manage various VM providers. -Minikube embeds VirtualBox and VMware Fusion drivers so there are no additional steps to use them. -However, other drivers require an extra binary to be present in the host `PATH`. - -If you are running on macOS, in particular make sure to install the `xhyve` drivers for the native OS X hypervisor: - -```sh -brew install docker-machine-driver-xhyve - -$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve - -$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve -``` - -#### Start Minikube +* [Confluent Platform Open Source Docker Images](https://hub.docker.com/u/confluentinc/) 5.0.0 -1. Start Minikube. - -In the command below, note that memory has been increased to 6096 MB and it uses the xhyve driver for the native OS X hypervisor. - -```sh -minikube start --kubernetes-version v1.8.0 --cpus 4 --memory 6096 --vm-driver=xhyve --v=8 -``` - -2. Continue to check status of your local Kubernetes cluster until both minikube and cluster are in Running state - -```sh -$ minikube status -minikube: Running -cluster: Running -kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.106 -``` - -3. Work around Minikube [issue #1568](https://github.com/kubernetes/minikube/issues/1568). - -```sh -minikube ssh -- sudo ip link set docker0 promisc on -``` - -. Set the context. - -```sh -$ eval $(minikube docker-env) - -$ kubectl config set-context minikube.internal --cluster=minikube --user=minikube -Context "minikube.internal" modified. -o -$ kubectl config use-context minikube.internal -Switched to context "minikube.internal". -``` - -#### Verify Minikube Local Kubernetes Environment - -```sh -$ kubectl config current-context -minikube.internal - -$ kubectl cluster-info -Kubernetes master is running at https://192.168.99.106:8443 -KubeDNS is running at https://192.168.99.106:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy -``` +For local Kubernetes installation with Minikube please refer to [Create a Local Kubernetes Cluster](#local-cluster-with-minikube) ### Install Helm on Kubernetes @@ -132,21 +60,21 @@ Follow the directions to [install and deploy Helm](https://docs.helm.sh/using_he View a list of all deployed releases in releases in the local installation. -```sh -$ helm init -$ helm repo update -$ helm list +```console +helm init +helm repo update +helm list ``` NOTE: For Helm versions prior to 2.9.1, you may see `"connect: connection refused"`, and will need to fix up the deployment before proceeding. -```sh -$ kubectl delete --namespace kube-system svc tiller-deploy -$ kubectl delete --namespace kube-system deploy tiller-deploy -$ kubectl create serviceaccount --namespace kube-system tiller -$ kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller -$ kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}' -$ helm init --service-account tiller --upgrade +```console +kubectl delete --namespace kube-system svc tiller-deploy +kubectl delete --namespace kube-system deploy tiller-deploy +kubectl create serviceaccount --namespace kube-system tiller +kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller +kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}' +helm init --service-account tiller --upgrade ``` ## Run Confluent Platform @@ -155,25 +83,25 @@ $ helm init --service-account tiller --upgrade Clone the Confluent Helm Chart repo -```sh +```console git clone https://github.com/confluentinc/cp-helm-charts.git ``` Install a 3 node ZooKeeper ensemble, a Kafka cluster of 3 brokers, 1 Confluent Schema Registry instance, 1 REST Proxy instance, and 1 Kafka Connect worker in your Kubernetes environment. Naming the chart `--name my-confluent-oss` is optional, but we assume this is the name in the remainder of the documentation. -```sh +```console helm install --name my-confluent-oss cp-helm-charts ``` If you want to install without the Confluent Schema Registry instance, the REST Proxy instance, and the Kafka Connect worker: -```sh +```console helm install --set cp-schema-registry.enabled=false,cp-kafka-rest.enabled=false,cp-kafka-connect.enabled=false cp-helm-charts ``` View the installed Helm releases: -```sh +```console $ helm list NAME REVISION UPDATED STATUS CHART NAMESPACE my-confluent-oss 1 Tue Jun 12 16:56:39 2018 DEPLOYED cp-helm-charts-0.1.0 default @@ -185,7 +113,7 @@ my-confluent-oss 1 Tue Jun 12 16:56:39 2018 DEPLOYED cp-helm- This step is optional: run the embedded test pod in each sub-chart to verify installation: -```sh +```console helm test my-confluent-oss ``` @@ -195,19 +123,19 @@ This step is optional: to verify that Kafka is working as expected, connect to o 1. List your pods and wait until they are all in `Running` state. -```sh +```console kubectl get pods ``` 2. Connect to the container `cp-kafka-broker` in a Kafka broker pod to produce messages to a Kafka topic. If you specified a different release name, substitute `my-confluent-oss` with whatever you named your release. -```sh +```console kubectl exec -c cp-kafka-broker -it my-confluent-oss-cp-kafka-0 -- /bin/bash /usr/bin/kafka-console-producer --broker-list localhost:9092 --topic test ``` Wait for a `>` prompt, and enter some text. -```sh +```console m1 m2 ``` @@ -216,7 +144,7 @@ Press Control-d to close the producer session. 3. Consume the messages from the same Kafka topic as above. -```sh +```console kubectl exec -c cp-kafka-broker -it my-confluent-oss-cp-kafka-0 -- /bin/bash /usr/bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic test --from-beginning ``` @@ -227,13 +155,13 @@ You should see the messages which were published from the console producer. Pres ##### ZooKeeper 1. Deploy a ZooKeeper client pod - ```sh + ```console kubectl apply -f cp-helm-charts/examples/zookeeper-client.yaml ``` 2. Connect to the ZooKeeper client pod and use the `zookeeper-shell` command to explore brokers, topics, etc: - ```sh + ```console kubectl exec -it zookeeper-client -- /bin/bash zookeeper-shell : ls /brokers/ids kubectl exec -it zookeeper-client -- /bin/bash zookeeper-shell : get /brokers/ids/0 kubectl exec -it zookeeper-client -- /bin/bash zookeeper-shell : ls /brokers/topics @@ -242,17 +170,17 @@ You should see the messages which were published from the console producer. Pres ##### Kafka 1. Deploy a Kafka client pod - ```sh + ```console kubectl apply -f cp-helm-charts/examples/kafka-client.yaml ``` 2. Log into the Pod - ```sh + ```console kubectl exec -it kafka-client -- /bin/bash ``` 3. From within the `kafka-client` pod, explore with Kafka commands: - ```sh + ```console ## Setup export RELEASE_NAME= export ZOOKEEPERS=${RELEASE_NAME}-cp-zookeeper:2181 @@ -277,48 +205,48 @@ KSQL is the streaming SQL engine that enables real-time data processing against ### Scaling -NOTE: All scaling operations should be done offline with no producer/consumer connection +> **Note:** All scaling operations should be done offline with no producer/consumer connection #### ZooKeeper Install cp-helm-charts with default 3 node ZooKeeper ensemble -```sh +```console helm install cp-helm-charts ``` Scale ZooKeeper nodes up to 5, change `servers` under `cp-zookeeper` to 5 in [values.yaml](values.yaml) -```sh +```console helm upgrade cp-helm-charts ``` Scale ZooKeeper nodes down to 3, change `servers` under `cp-zookeeper` to 3 in [values.yaml](values.yaml) -```sh +```console helm upgrade cp-helm-charts ``` #### Kafka -NOTE: Scaling Kafka brokers without doing Partition Reassignment will cause data loss!! +> **Note:** Scaling Kafka brokers without doing Partition Reassignment will cause data loss!! Be sure to reassign partitions correctly before [scaling the Kafka cluster](https://kafka.apache.org/documentation/#basic_ops_cluster_expansion). Install cp-helm-charts with default 3 brokers Kafka cluster -```sh +```console helm install cp-helm-charts ``` Scale Kafka brokers up to 5, change `brokers` under `cp-kafka` to 5 in [values.yaml](values.yaml) -```sh +```console helm upgrade cp-helm-charts ``` Scale Kafka brokers down to 3, change `brokers` under `cp-kafka` to 3 in [values.yaml](values.yaml) -```sh +```console helm upgrade cp-helm-charts ``` @@ -328,7 +256,7 @@ JMX Metrics are enabled by default for all components, Prometheus JMX Exporter i 1. Install Prometheus and Grafana in same Kubernetes cluster using helm - ```sh + ```console helm install stable/prometheus helm install stable/grafana ``` @@ -344,7 +272,7 @@ JMX Metrics are enabled by default for all components, Prometheus JMX Exporter i To remove the pods, list the pods with `kubectl get pods` and then delete the pods by name. -```sh +```console kubectl get pods kubectl delete pod ``` @@ -353,20 +281,13 @@ To delete the Helm release, find the Helm release name with `helm list` and dele You may also need to clean up leftover `StatefulSets`, since `helm delete` can leave them behind. Finally, clean up all persisted volume claims (pvc) created by this release. -```sh +```console helm list helm delete $ kubectl delete statefulset -cp-kafka -cp-zookeeper kubectl delete pvc --selector=release= ``` -To stop or delete Minikube: - -```sh -minikube stop -minikube delete -``` - ## Thanks Huge thanks to: @@ -376,3 +297,87 @@ Huge thanks to: * [Schema Registry helm chart](https://github.com/kubernetes/charts/tree/master/incubator/schema-registry) * [kubernetes-kafka](https://github.com/Yolean/kubernetes-kafka) * [docker-kafka](https://github.com/solsson/dockerfiles) + +## Appendix + +### Create a Local Kubernetes Cluster + +There are many deployment options to get set up with a Kubernetes cluster, and this document provides instructions for using [Minikube](https://kubernetes.io/docs/setup/minikube/) to set up a local Kubernetes cluster. +Minikube runs a single-node Kubernetes cluster inside a VM on your laptop. + +You may alternatively set up a Kubernetes cluster in the cloud using other providers such as [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/quickstart). + +#### Install Minikube and Drivers + +Minikube version 0.23.0 or higher is required for docker server https://github.com/moby/moby/pull/31352[17.05], which adds support for using `ARG` in `FROM` in your `Dockerfile`. + +First follow the basic [Minikube installation instructions](https://github.com/kubernetes/minikube). + +Then install the [Minikube drivers](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md). Minikube uses Docker Machine to manage the Kubernetes VM so it benefits from the driver plugin architecture that Docker Machine uses to provide a consistent way to manage various VM providers. +Minikube embeds VirtualBox and VMware Fusion drivers so there are no additional steps to use them. +However, other drivers require an extra binary to be present in the host `PATH`. + +If you are running on macOS, in particular make sure to install the `xhyve` drivers for the native OS X hypervisor: + +```console +brew install docker-machine-driver-xhyve + +$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve + +$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve +``` + +#### Start Minikube + +> **Note**: In the command below, memory has been increased to 6096 MB and it uses the `xhyve` driver for the native OS X hypervisor. + +```console +minikube start --kubernetes-version v1.9.4 --cpus 4 --memory 6096 --vm-driver=xhyve --v=8 +``` + +1. Continue to check status of your local Kubernetes cluster until both minikube and cluster are in Running state + + ```console + $ minikube status + minikube: Running + cluster: Running + kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.106 + ``` + +1. Work around Minikube [issue #1568](https://github.com/kubernetes/minikube/issues/1568). + + ```console + minikube ssh -- sudo ip link set docker0 promisc on + ``` + +1. Set the context. + + ```console + $ eval $(minikube docker-env) + + $ kubectl config set-context minikube.internal --cluster=minikube --user=minikube + Context "minikube.internal" modified. + o + $ kubectl config use-context minikube.internal + Switched to context "minikube.internal". + ``` + +#### Verify Minikube Local Kubernetes Environment + +```console +$ kubectl config current-context +minikube.internal + +$ kubectl cluster-info +Kubernetes master is running at https://192.168.99.106:8443 +KubeDNS is running at https://192.168.99.106:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy +``` + +#### Teardown of local cluster + +To stop or delete Minikube: + +```console +minikube stop +minikube delete +``` diff --git a/charts/cp-kafka-connect/README.md b/charts/cp-kafka-connect/README.md index 5f8da612..0ac1fd9d 100644 --- a/charts/cp-kafka-connect/README.md +++ b/charts/cp-kafka-connect/README.md @@ -1,41 +1,52 @@ # CP-Kafka Connect Helm Chart + This chart bootstraps a deployment of a Confluent Kafka Connect ## Prerequisites + * Kubernetes 1.9.2+ * Helm 2.8.2+ * A healthy and accessible Kafka Cluster -## Developing Environment: -* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) +## Developing Environment: + * [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) +* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) + +## Docker Image Source: -## Docker Image Source: -* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/ +* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/) ## Installing the Chart + ### Install along with cp-helm-charts + ```console -$ git clone https://github.com/confluentinc/cp-helm-charts.git -$ helm install cp-helm-charts +git clone https://github.com/confluentinc/cp-helm-charts.git +helm install cp-helm-charts ``` To install with a specific name, you can do: + ```console -$ helm install --name my-confluent cp-helm-charts +helm install --name my-confluent cp-helm-charts ``` ### Install with a existing CP-Kafka release + ```console -$ helm install --set kafka.bootstrapServers="PLAINTEXT://lolling-chinchilla-cp-kafka-headless:9092",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-connect +helm install --set kafka.bootstrapServers="PLAINTEXT://lolling-chinchilla-cp-kafka-headless:9092",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-connect ``` ### Installed Components + You can use `helm status ` to view all of the installed components. For example: -```console{%raw} + +```console $ helm status kissing-macaw + NAMESPACE: default STATUS: DEPLOYED @@ -52,41 +63,47 @@ kissing-macaw-cp-kafka-connect 1 1 1 1 34m NAME READY STATUS RESTARTS AGE kissing-macaw-cp-kafka-connect-6c77b8f5fd-cqlzq 1/1 Running 0 34m ``` -There are + +There are 1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `kissing-macaw-cp-kafka-connect` which contains 1 Kafka Connect [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `kissing-macaw-cp-kafka-connect-6c77b8f5fd-cqlzq`. -1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `kissing-macaw-cp-kafka-connect` for clients to connect to Kafka Connect REST endpoint. +2. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `kissing-macaw-cp-kafka-connect` for clients to connect to Kafka Connect REST endpoint. ## Configuration + You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```console -$ helm install --name my-kafka-connect -f my-values.yaml ./cp-kafka-connect +helm install --name my-kafka-connect -f my-values.yaml ./cp-kafka-connect ``` > **Tip**: A default [values.yaml](values.yaml) is provided ### Kafka Connect Deployment -The configuration parameters in this section control the resources requested and utilized by the cp-kafka-connect chart. -| Parameter | Description | Default | -| --------- | ----------- | ------- | -| `replicaCount` | The number of Kafka Connect Servers. | `1` | +The configuration parameters in this section control the resources requested and utilized by the `cp-kafka-connect` chart. + +| Parameter | Description | Default | +| ----------------- | ------------------------------------- | --------- | +| `replicaCount` | The number of Kafka Connect Servers. | `1` | ### Image + | Parameter | Description | Default | | --------- | ----------- | ------- | | `image` | Docker Image of Confluent Kafka Connect. | `confluentinc/cp-kafka-connect` | -| `imageTag` | Docker Image Tag of Confluent Kafka Connect. | `4.1.1` | +| `imageTag` | Docker Image Tag of Confluent Kafka Connect. | `5.0.0` | | `imagePullPolicy` | Docker Image Tag of Confluent Kafka Connect. | `IfNotPresent` | ### Port + | Parameter | Description | Default | | --------- | ----------- | ------- | | `servicePort` | The port on which the Kafka Connect will be available and serving requests. | `8082` | ### Kafka Connect Worker Configurations + | Parameter | Description | Default | | --------- | ----------- | ------- | | `connectKeyConverter` | Converter class for key Connect data. This controls the format of the data that will be written to Kafka for source connectors or read from Kafka for sink connectors. Popular formats include Avro and JSON. |`"io.confluent.connect.avro.AvroConverter"`| @@ -96,6 +113,7 @@ The configuration parameters in this section control the resources requested and | `pluginPath` | The comma-separated list of paths to directories that contain Kafka Connect plugins. |`"/usr/share/java"`| ### Resources + | Parameter | Description | Default | | --------- | ----------- | ------- | | `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details | @@ -104,20 +122,24 @@ The configuration parameters in this section control the resources requested and | `resources.requests.limit` | The upper limit memory usage for a Kafka Connect Pod. | see [values.yaml](values.yaml) for details | ### JMX Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` | ### Prometheus JMX Exporter Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` | -| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | +| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | | `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` | | `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` | ## Dependencies + ### Kafka + | Parameter | Description | Default | | --------- | ----------- | ------- | | `kafka.bootstrapServers` | Bootstrap Servers for Kafka Connect | `""` | \ No newline at end of file diff --git a/charts/cp-kafka-connect/values.yaml b/charts/cp-kafka-connect/values.yaml index 94f9c672..8dca1ba1 100644 --- a/charts/cp-kafka-connect/values.yaml +++ b/charts/cp-kafka-connect/values.yaml @@ -7,7 +7,7 @@ replicaCount: 1 ## Image Info ## ref: https://hub.docker.com/r/confluentinc/cp-kafka/ image: confluentinc/cp-kafka-connect -imageTag: 4.1.1 +imageTag: 5.0.0 ## Specify a imagePullPolicy ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images diff --git a/charts/cp-kafka-rest/README.md b/charts/cp-kafka-rest/README.md index 2f853134..89d09104 100644 --- a/charts/cp-kafka-rest/README.md +++ b/charts/cp-kafka-rest/README.md @@ -1,40 +1,50 @@ # CP-REST Proxy Helm Chart + This chart bootstraps a deployment of a Confluent REST Proxy ## Prerequisites + * Kubernetes 1.9.2+ * Helm 2.8.2+ * A healthy and accessible Zookeeper Ensemble of the Kafka Cluster -## Developing Environment: -* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) +## Developing Environment + * [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) +* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) + +## Docker Image Source -## Docker Image Source: -* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/ +* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/) ## Installing the Chart + ### Install along with cp-helm-charts + ```console -$ git clone https://github.com/confluentinc/cp-helm-charts.git -$ helm install cp-helm-charts +git clone https://github.com/confluentinc/cp-helm-charts.git +helm install cp-helm-charts ``` To install with a specific name, you can do: + ```console -$ helm install --name my-confluent cp-helm-charts +helm install --name my-confluent cp-helm-charts ``` ### Install with a existing cp-kafka and cp-schema-registry release + ```console -$ helm install --set cp-zookeeper.url="unhinged-robin-cp-zookeeper:2181",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-rest +helm install --set cp-zookeeper.url="unhinged-robin-cp-zookeeper:2181",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-rest ``` ### Installed Components + You can use `helm status ` to view all of the installed components. For example: -```console{%raw} + +```console $ helm status lolling-chinchilla NAMESPACE: default STATUS: DEPLOYED @@ -56,43 +66,49 @@ hopping-salamander-cp-kafka-rest-67b86cff98-qxrd8 1/1 Running 0 1m NAME DATA AGE hopping-salamander-cp-kafka-rest-jmx-configmap 1 1s ``` -There are + +There are 1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `hopping-salamander-cp-kafka-rest` which contains 1 REST Proxy [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `hopping-salamander-cp-kafka-rest-67b86cff98-qxrd8`. 1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `hopping-salamander-cp-kafka-rest` for clients to connect to REST Proxy. 1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter. 1. (Optional) A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `hopping-salamander-cp-kafka-restproxy-external` for clients to connect to REST Proxy from outside. - + ## Configuration + You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```console -$ helm install --name my-rest-proxy -f my-values.yaml ./cp-kafka-rest +helm install --name my-rest-proxy -f my-values.yaml ./cp-kafka-rest ``` > **Tip**: A default [values.yaml](values.yaml) is provided ### REST Proxy Deployment -The configuration parameters in this section control the resources requested and utilized by the cp-kafka-rest chart. + +The configuration parameters in this section control the resources requested and utilized by the `cp-kafka-rest` chart. | Parameter | Description | Default | | --------- | ----------- | ------- | | `replicaCount` | The number of REST Proxy Servers. | `1` | ### Image + | Parameter | Description | Default | | --------- | ----------- | ------- | | `image` | Docker Image of Confluent REST Proxy. | `confluentinc/cp-kafka-rest` | -| `imageTag` | Docker Image Tag of Confluent REST Proxy. | `4.1.1` | +| `imageTag` | Docker Image Tag of Confluent REST Proxy. | `5.0.0` | | `imagePullPolicy` | Docker Image Tag of Confluent REST Proxy. | `IfNotPresent` | ### Port + | Parameter | Description | Default | | --------- | ----------- | ------- | | `servicePort` | The port on which the REST Proxy will be available and serving requests. | `8082` | ### Resources + | Parameter | Description | Default | | --------- | ----------- | ------- | | `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details | @@ -101,32 +117,38 @@ The configuration parameters in this section control the resources requested and | `resources.requests.limit` | The upper limit memory usage for a REST Proxy Pod. | see [values.yaml](values.yaml) for details | ### JMX Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` | ### Prometheus JMX Exporter Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` | -| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | +| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | | `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` | | `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` | ### External Access + | Parameter | Description | Default | | --------- | ----------- | ------- | | `external.enabled` | whether or not to allow external access to Kafka REST Proxy | `false` | | `external.type` | `Kubernetes Service Type` to expose Kafka REST Proxy to external | `LoadBalancer` | ## Dependencies + ### Zookeeper + | Parameter | Description | Default | | --------- | ----------- | ------- | | `cp-zookeeper.url` | Service name of Zookeeper cluster (Not needed if this is installed along with cp-kafka chart). | `""` | | `cp-zookeeper.clientPort` | Port of Zookeeper Cluster | `2181` | ### Schema Registry (optional) + | Parameter | Description | Default | | --------- | ----------- | ------- | | `cp-schema-registry.url` | Service name of Schema Registry (Not needed if this is installed along with cp-kafka chart). | `""` | diff --git a/charts/cp-kafka-rest/values.yaml b/charts/cp-kafka-rest/values.yaml index 692eb7ac..417b481f 100644 --- a/charts/cp-kafka-rest/values.yaml +++ b/charts/cp-kafka-rest/values.yaml @@ -7,7 +7,7 @@ replicaCount: 1 ## Image Info ## ref: https://hub.docker.com/r/confluentinc/cp-kafka/ image: confluentinc/cp-kafka-rest -imageTag: 4.1.1 +imageTag: 5.0.0 ## Specify a imagePullPolicy ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images diff --git a/charts/cp-kafka/README.md b/charts/cp-kafka/README.md index 47d24d2a..073ae2ce 100644 --- a/charts/cp-kafka/README.md +++ b/charts/cp-kafka/README.md @@ -1,39 +1,48 @@ # CP-Kafka Helm Chart + This chart bootstraps a cluster of Confluent Kafka ## Prerequisites + * Kubernetes 1.9.2+ * Helm 2.8.2+ -## Developing Environment: -* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) +## Developing Environment: + * [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) +* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) + +## Docker Image Source -## Docker Image Source: -* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/ +* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/) ## Installing the Chart + ### Install along with cp-helm-charts + ```console -$ git clone https://github.com/confluentinc/cp-helm-charts.git -$ helm install cp-helm-charts +git clone https://github.com/confluentinc/cp-helm-charts.git +helm install cp-helm-charts ``` To install with a specific name, you can do: ```console -$ helm install --name my-confluent cp-helm-charts +helm install --name my-confluent cp-helm-charts ``` ### Install with a existing cp-zookeeper + ```console -$ helm install --set cp-zookeeper.enabled=false,cp-zookeeper.url="unhinged-robin-cp-zookeeper-headless:2181" cp-helm-charts/charts/cp-kafka +helm install --set cp-zookeeper.enabled=false,cp-zookeeper.url="unhinged-robin-cp-zookeeper-headless:2181" cp-helm-charts/charts/cp-kafka ``` ### Installed Components + You can use `helm status ` to view all of the installed components. For example: -```console{%raw} + +```console $ helm status garish-cat NAMESPACE: default STATUS: DEPLOYED @@ -72,7 +81,8 @@ boiling-heron-cp-kafka-0 2/2 Running 0 5m boiling-heron-cp-kafka-1 2/2 Running 0 5m boiling-heron-cp-kafka-2 2/2 Running 0 5m ``` -There are + +There are 1. A [Confluent Zookeeper Ensemble](https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-zookeeper) created by cp-zookeeper chart. 1. A [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) `boiling-heron-cp-kafka` which contains 3 Kafka [Pods](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `boiling-heron-cp-kafka-<0|1|2>`. Each Pod has a container running a Kafka Broker and an optional sidecar JMX Exporter Container. 1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `boiling-heron-cp-kafka` for clients to connect to Kafka. @@ -81,17 +91,19 @@ There are 1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter. ## Configuration + You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```console -$ helm install --name my-kafka -f my-values.yaml ./cp-kafka +helm install --name my-kafka -f my-values.yaml ./cp-kafka ``` > **Tip**: A default [values.yaml](values.yaml) is provided ### Kafka Cluster + The configuration parameters in this section control the resources requested and utilized by the cp-kafka chart. | Parameter | Description | Default | @@ -99,24 +111,28 @@ The configuration parameters in this section control the resources requested and | `brokers` | The number of Broker servers. | `3` | ### Image + | Parameter | Description | Default | | --------- | ----------- | ------- | | `image` | Docker Image of Confluent Kafka. | `confluentinc/cp-kafka` | -| `imageTag` | Docker Image Tag of Confluent Kafka. | `4.1.1` | +| `imageTag` | Docker Image Tag of Confluent Kafka. | `5.0.0` | | `imagePullPolicy` | Docker Image Tag of Confluent Kafka. | `IfNotPresent` | ### StatefulSet Configurations + | Parameter | Description | Default | | --------- | ----------- | ------- | | `podManagementPolicy` | The Kafka StatefulSet Pod Management Policy: `Parallel` or `OrderedReady`. | `OrderedReady` | | `updateStrategy` | The Kafka StatefulSet update strategy: `RollingUpdate` or `OnDelete`. | `OnDelete` | ### Confluent Kafka Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `configurationOverrides` | Kafka [configuration](https://kafka.apache.org/documentation/#brokerconfigs) overrides in the dictionary format | `{}` | ### Persistence + | Parameter | Description | Default | | --------- | ----------- | ------- | | `persistence.enabled` | Whether to create a PVC. If `false`, an `emptyDir` on the host will be used. | `true` | @@ -124,11 +140,13 @@ The configuration parameters in this section control the resources requested and | `persistence.storageClass` | Valid options: `nil`, `"-"`, or storage class name. | `nil` | ### Kafka JVM Heap Options + | Parameter | Description | Default | | --------- | ----------- | ------- | | `heapOptions` | The JVM Heap Options for Kafka | `"-Xms1G -Xmx1G"` | ### Resources + | Parameter | Description | Default | | --------- | ----------- | ------- | | `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details | @@ -137,27 +155,32 @@ The configuration parameters in this section control the resources requested and | `resources.requests.limit` | The upper limit memory usage for a Kafka Pod. | see [values.yaml](values.yaml) for details | ### JMX Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` | ### Prometheus JMX Exporter Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` | -| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | +| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | | `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` | | `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` | ### External Access + | Parameter | Description | Default | | --------- | ----------- | ------- | | `nodeport.enabled` | Whether or not to allow access to kafka cluster from outside k8s through NodePort. | `false` | -| `nodeport.servicePort` | The Port broker will advertise to external producers and consumers. | `19092` | +| `nodeport.servicePort` | The Port broker will advertise to external producers and consumers. | `19092` | | `nodeport.firstListenerPort` | The first NodePort that Kafka Broker will use for advertising to external producers and consumers. For each broker, advertise.listeners port for external will be set to `31090 + {index of broker pod}`. | `31090` | ## Dependencies + ### Zookeeper + | Parameter | Description | Default | | --------- | ----------- | ------- | | `cp-zookeeper.enabled` | Whether or not to install cp-zookeeper chart alongside cp-kafka chart | `true` | diff --git a/charts/cp-kafka/values.yaml b/charts/cp-kafka/values.yaml index 628137ee..4443bd95 100644 --- a/charts/cp-kafka/values.yaml +++ b/charts/cp-kafka/values.yaml @@ -12,7 +12,7 @@ brokers: 3 ## Image Info ## ref: https://hub.docker.com/r/confluentinc/cp-kafka/ image: confluentinc/cp-kafka -imageTag: 4.1.1 +imageTag: 5.0.0 ## Specify a imagePullPolicy ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images diff --git a/charts/cp-schema-registry/README.md b/charts/cp-schema-registry/README.md index 5de66e66..f23b0236 100644 --- a/charts/cp-schema-registry/README.md +++ b/charts/cp-schema-registry/README.md @@ -1,33 +1,41 @@ # CP-Schema Registry Helm Chart + This chart bootstraps a deployment of a Confluent Schema Registry ## Prerequisites + * Kubernetes 1.9.2+ * Helm 2.8.2+ * A healthy and accessible Kafka Cluster -## Developing Environment: -* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) +## Developing Environment + * [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) +* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) -## Docker Image Source: -* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/ +## Docker Image Source + +* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/) ## Installing the Chart + ### Install along with cp-helm-charts + ```console -$ git clone https://github.com/confluentinc/cp-helm-charts.git -$ helm install cp-helm-charts +git clone https://github.com/confluentinc/cp-helm-charts.git +helm install cp-helm-charts ``` To install with a specific name, you can do: + ```console -$ helm install --name my-confluent cp-helm-charts +helm install --name my-confluent cp-helm-charts ``` ### Install with a existing cp-kafka release + ```console -$ helm install --set kafka.bootstrapServers="PLAINTEXT://unhinged-robin-cp-kafka-headless:9092" cp-helm-charts/charts/cp-schema-registry +helm install --set kafka.bootstrapServers="PLAINTEXT://unhinged-robin-cp-kafka-headless:9092" cp-helm-charts/charts/cp-schema-registry ``` ### Installed Components @@ -56,23 +64,26 @@ lolling-chinchilla-cp-schema-registry ClusterIP 10.19.245.11 808 NAME DATA AGE lolling-chinchilla-cp-schema-registry-jmx-configmap 1 1s ``` -There are + +There are 1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `lolling-chinchilla-cp-schema-registry` which contains 1 Schema Registry [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `lolling-chinchilla-cp-schema-registry-58f854bd47-jxrcj`. 1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `lolling-chinchilla-cp-schema-registry` for clients to connect to Schema Registry. 1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter. ## Configuration + You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```console -$ helm install --name my-schema-registry -f my-values.yaml ./cp-schema-registry +helm install --name my-schema-registry -f my-values.yaml ./cp-schema-registry ``` > **Tip**: A default [values.yaml](values.yaml) is provided ### Schema Registry Deployment + The configuration parameters in this section control the resources requested and utilized by the cp-schema-registry chart. | Parameter | Description | Default | @@ -80,28 +91,33 @@ The configuration parameters in this section control the resources requested and | `replicaCount` | The number of Schema Registry Servers. | `1` | ### Image + | Parameter | Description | Default | | --------- | ----------- | ------- | | `image` | Docker Image of Confluent Schema Registry. | `confluentinc/cp-schema-registry` | -| `imageTag` | Docker Image Tag of Confluent Schema Registry. | `4.1.1` | +| `imageTag` | Docker Image Tag of Confluent Schema Registry. | `5.0.0` | | `imagePullPolicy` | Docker Image Tag of Confluent Schema Registry. | `IfNotPresent` | ### Confluent Schema Registry Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `configurationOverrides` | Schema Registry [configuration](https://docs.confluent.io/current/schema-registry/docs/config.html) overrides in the dictionary format. | `{}` | ### Port + | Parameter | Description | Default | | --------- | ----------- | ------- | | `servicePort` | The port on which the Schema Registry will be available and serving requests. | `8081` | ### Kafka + | Parameter | Description | Default | | --------- | ----------- | ------- | | `kafka.bootstrapServers` | Bootstrap Servers for Schema Registry | `""` | ### Resources + | Parameter | Description | Default | | --------- | ----------- | ------- | | `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details | @@ -110,14 +126,16 @@ The configuration parameters in this section control the resources requested and | `resources.requests.limit` | The upper limit memory usage for a Schema Registry Pod. | see [values.yaml](values.yaml) for details | ### JMX Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` | ### Prometheus JMX Exporter Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` | -| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | +| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | | `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` | | `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` | \ No newline at end of file diff --git a/charts/cp-schema-registry/values.yaml b/charts/cp-schema-registry/values.yaml index d12550a4..290014bd 100644 --- a/charts/cp-schema-registry/values.yaml +++ b/charts/cp-schema-registry/values.yaml @@ -12,7 +12,7 @@ replicaCount: 1 ## Image Info ## ref: https://hub.docker.com/r/confluentinc/cp-schema-registry/ image: confluentinc/cp-schema-registry -imageTag: 4.1.1 +imageTag: 5.0.0 ## Specify a imagePullPolicy ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images diff --git a/charts/cp-zookeeper/README.md b/charts/cp-zookeeper/README.md index 9f1ba751..7244669b 100644 --- a/charts/cp-zookeeper/README.md +++ b/charts/cp-zookeeper/README.md @@ -1,38 +1,48 @@ # CP-Zookeeper Helm Chart + This chart bootstraps an ensemble of Confluent Zookeeper ## Prerequisites + * Kubernetes 1.9.2+ * Helm 2.8.2+ -## Developing Environment: -* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) +## Developing Environment + * [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) +* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) + +## Docker Image Source -## Docker Image Source: -* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/ +* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/) ## Installing the Chart + ### Install along with cp-helm-charts + ```console -$ git clone https://github.com/confluentinc/cp-helm-charts.git -$ helm install cp-helm-charts +git clone https://github.com/confluentinc/cp-helm-charts.git +helm install cp-helm-charts ``` To install with a specific name, you can do: + ```console -$ helm install --name my-zookeeper ./cp-zookeeper +helm install --name my-zookeeper ./cp-zookeeper ``` ### Install with cp-zookeeper alone + ```console -$ helm install cp-helm-charts/charts/cp-zookeeper +helm install cp-helm-charts/charts/cp-zookeeper ``` ### Installed Components + You can use `helm status ` to view all of the installed components. For example: + ```console{%raw} $ helm status unsung-salamander @@ -60,24 +70,28 @@ unsung-salamander-cp-zookeeper-2 1/1 Running 0 6m NAME DATA AGE unsung-salamander-cp-zookeeper-jmx-configmap 1 6m ``` -There are + +There are 1. A [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) `unsung-salamander-cp-zookeeper` which contains 3 Zookeeper [Pods](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `unsung-salamander-cp-zookeeper-<0|1|2>`. Each Pod has a container running a ZooKeeper server and an optional sidecar JMX Exporter Container. 1. A [PodDisruptionBudget](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) `unsung-salamander-cp-zookeeper-pdb` to ensure service availability during planned maintenance. 1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `unsung-salamander-cp-zookeeper` for clients to connect to Zookeeper. 1. A [Headless Service](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services) `unsung-salamander-cp-zookeeper-headless` to control the network domain for the ZooKeeper processes. 1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter. + ## Configuration + You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```console -$ helm install --name my-zookeeper -f my-values.yaml ./cp-zookeeper +helm install --name my-zookeeper -f my-values.yaml ./cp-zookeeper ``` > **Tip**: A default [values.yaml](values.yaml) is provided ### Zookeeper Ensemble + The configuration parameters in this section control the resources requested and utilized by the cp-zookeeper chart. | Parameter | Description | Default | @@ -85,24 +99,28 @@ The configuration parameters in this section control the resources requested and | `servers` | The number of ZooKeeper servers. This should always be (1,3,5, or 7). | `3` | ### PodDisruptionBudget + | Parameter | Description | Default | | --------- | ----------- | ------- | | `minAvailable` | The minimum number of servers that must be available during evictions. This should in the interval `[(servers/2) + 1,(servers - 1)]`. If not set, `maxUnavailable: 1` will be applied. | `servers-1` | ### Image + | Parameter | Description | Default | | --------- | ----------- | ------- | | `image` | Docker Image of Confluent Zookeeper. | `confluentinc/cp-zookeeper` | -| `imageTag` | Docker Image Tag of Confluent Zookeeper. | `4.1.1` | +| `imageTag` | Docker Image Tag of Confluent Zookeeper. | `5.0.0` | | `imagePullPolicy` | Docker Image Tag of Confluent Zookeeper. | `IfNotPresent` | ### StatefulSet Configurations + | Parameter | Description | Default | | --------- | ----------- | ------- | | `podManagementPolicy` | The Zookeeper StatefulSet Pod Management Policy: `Parallel` or `OrderedReady`. | `OrderedReady` | | `updateStrategy` | The ZooKeeper StatefulSet update strategy: `RollingUpdate` or `OnDelete`. | `RollingUpdate` | ### Confluent Zookeeper Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `tickTime` | The length of a single tick, which is the basic time unit used by ZooKeeper, as measured in milliseconds. It is used to regulate heartbeats, and timeouts. For example, the minimum session timeout will be two ticks. | `2000` | @@ -113,11 +131,13 @@ The configuration parameters in this section control the resources requested and | `autoPurgePurgeInterval` | The time interval in hours for which the purge task has to be triggered. Set to a positive integer (1 and above) to enable the auto purging. | `72` | ### Zookeeper JVM Heap Options + | Parameter | Description | Default | | --------- | ----------- | ------- | | `heapOptions` | The JVM Heap Options for Zookeeper | `"-Xms512M -Xmx512M"` | ### Port + | Parameter | Description | Default | | --------- | ----------- | ------- | | `serverPort` | The port on which the ZooKeeper servers listen for requests from other servers in the ensemble. | `2888` | @@ -125,6 +145,7 @@ The configuration parameters in this section control the resources requested and | `clientPort` | The port to listen for client connections; that is, the port that clients attempt to connect to. | `2181` | ### Persistence + | Parameter | Description | Default | | --------- | ----------- | ------- | | `persistence.enabled` | Whether to create a PVC. If `false`, an `emptyDir` on the host will be used. | `true` | @@ -134,6 +155,7 @@ The configuration parameters in this section control the resources requested and | `persistence.dataLogDirStorageClass` | Valid options: `nil`, `"-"`, or storage class name. | `nil` | ### Resources + | Parameter | Description | Default | | --------- | ----------- | ------- | | `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details | @@ -142,14 +164,16 @@ The configuration parameters in this section control the resources requested and | `resources.requests.limit` | The upper limit memory usage for a Zookeeper Pod. | see [values.yaml](values.yaml) for details | ### JMX Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` | ### Prometheus JMX Exporter Configuration + | Parameter | Description | Default | | --------- | ----------- | ------- | | `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` | -| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | +| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` | | `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` | | `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` | diff --git a/charts/cp-zookeeper/values.yaml b/charts/cp-zookeeper/values.yaml index 0a8d28d6..33adb2bb 100644 --- a/charts/cp-zookeeper/values.yaml +++ b/charts/cp-zookeeper/values.yaml @@ -15,7 +15,7 @@ servers: 3 ## Images Info ## ref: https://hub.docker.com/r/confluentinc/cp-zookeeper/ image: confluentinc/cp-zookeeper -imageTag: 4.1.1 +imageTag: 5.0.0 ## Specify a imagePullPolicy ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images diff --git a/examples/avro-client.yaml b/examples/avro-client.yaml index 96ae0859..5b71aee6 100644 --- a/examples/avro-client.yaml +++ b/examples/avro-client.yaml @@ -6,7 +6,7 @@ metadata: spec: containers: - name: avro-client - image: confluentinc/cp-schema-registry:4.1.1 + image: confluentinc/cp-schema-registry:5.0.0 command: - sh - -c diff --git a/examples/kafka-client.yaml b/examples/kafka-client.yaml index 24ea5b4e..156aba2b 100644 --- a/examples/kafka-client.yaml +++ b/examples/kafka-client.yaml @@ -6,7 +6,7 @@ metadata: spec: containers: - name: kafka-client - image: confluentinc/cp-kafka:4.1.1 + image: confluentinc/cp-kafka:5.0.0 command: - sh - -c diff --git a/examples/zookeeper-client.yaml b/examples/zookeeper-client.yaml index 21de7fc9..8f89fee2 100644 --- a/examples/zookeeper-client.yaml +++ b/examples/zookeeper-client.yaml @@ -6,7 +6,7 @@ metadata: spec: containers: - name: zookeeper-client - image: confluentinc/cp-zookeeper:4.1.1 + image: confluentinc/cp-zookeeper:5.0.0 command: - sh - -c diff --git a/grafana-dashboard/confluent-open-source-grafana-dashboard.json b/grafana-dashboard/confluent-open-source-grafana-dashboard.json index 41b9e771..4d3882fd 100644 --- a/grafana-dashboard/confluent-open-source-grafana-dashboard.json +++ b/grafana-dashboard/confluent-open-source-grafana-dashboard.json @@ -3824,7 +3824,7 @@ "schemaVersion": 16, "style": "dark", "tags": [ - "4.1.1" + "5.0.0" ], "templating": { "list": [ diff --git a/values.yaml b/values.yaml index 622e0cfd..1b26c99a 100644 --- a/values.yaml +++ b/values.yaml @@ -5,7 +5,7 @@ cp-zookeeper: enabled: true servers: 3 image: confluentinc/cp-zookeeper - imageTag: 4.1.1 + imageTag: 5.0.0 heapOptions: "-Xms512M -Xmx512M" persistence: enabled: true @@ -36,7 +36,7 @@ cp-kafka: enabled: true brokers: 3 image: confluentinc/cp-kafka - imageTag: 4.1.1 + imageTag: 5.0.0 heapOptions: "-Xms512M -Xmx512M" persistence: enabled: true @@ -58,7 +58,7 @@ cp-kafka: cp-schema-registry: enabled: true image: confluentinc/cp-schema-registry - imageTag: 4.1.1 + imageTag: 5.0.0 heapOptions: "-Xms512M -Xmx512M" resources: {} ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, @@ -76,7 +76,7 @@ cp-schema-registry: cp-kafka-rest: enabled: true image: confluentinc/cp-kafka-rest - imageTag: 4.1.1 + imageTag: 5.0.0 resources: {} ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, ## and remove the curly braces after 'resources:' @@ -93,7 +93,7 @@ cp-kafka-rest: cp-kafka-connect: enabled: true image: confluentinc/cp-kafka-connect - imageTag: 4.1.1 + imageTag: 5.0.0 resources: {} ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, ## and remove the curly braces after 'resources:'