Skip to content
This repository has been archived by the owner on Feb 20, 2024. It is now read-only.

Commit

Permalink
Upgrade CP version to 5.0.0 (#115)
Browse files Browse the repository at this point in the history 
* bumped cp version to 5.0.0

* Restructured README

Moved minikube setup to the bottom
  • Loading branch information
@gAmUssA
gAmUssA authored Aug 1, 2018
1 parent 93afb83 commit e62d1ad
Show file tree
Hide file tree
Showing 16 changed files with 318 additions and 204 deletions.
243 changes: 124 additions & 119 deletions README.md
View file

Large diffs are not rendered by default.

58 changes: 40 additions & 18 deletions charts/cp-kafka-connect/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,41 +1,52 @@
# CP-Kafka Connect Helm Chart

This chart bootstraps a deployment of a Confluent Kafka Connect

## Prerequisites

* Kubernetes 1.9.2+
* Helm 2.8.2+
* A healthy and accessible Kafka Cluster

## Developing Environment:
* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/)
## Developing Environment:

* [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service)
* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/)

## Docker Image Source:

## Docker Image Source:
* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/
* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/)

## Installing the Chart

### Install along with cp-helm-charts

```console
$ git clone https://github.com/confluentinc/cp-helm-charts.git
$ helm install cp-helm-charts
git clone https://github.com/confluentinc/cp-helm-charts.git
helm install cp-helm-charts
```

To install with a specific name, you can do:

```console
$ helm install --name my-confluent cp-helm-charts
helm install --name my-confluent cp-helm-charts
```

### Install with a existing CP-Kafka release

```console
$ helm install --set kafka.bootstrapServers="PLAINTEXT://lolling-chinchilla-cp-kafka-headless:9092",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-connect
helm install --set kafka.bootstrapServers="PLAINTEXT://lolling-chinchilla-cp-kafka-headless:9092",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-connect
```

### Installed Components

You can use `helm status <release name>` to view all of the installed components.

For example:
```console{%raw}

```console
$ helm status kissing-macaw

NAMESPACE: default
STATUS: DEPLOYED

Expand All @@ -52,41 +63,47 @@ kissing-macaw-cp-kafka-connect 1 1 1 1 34m
NAME READY STATUS RESTARTS AGE
kissing-macaw-cp-kafka-connect-6c77b8f5fd-cqlzq 1/1 Running 0 34m
```
There are

There are
1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `kissing-macaw-cp-kafka-connect` which contains 1 Kafka Connect [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `kissing-macaw-cp-kafka-connect-6c77b8f5fd-cqlzq`.
1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `kissing-macaw-cp-kafka-connect` for clients to connect to Kafka Connect REST endpoint.
2. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `kissing-macaw-cp-kafka-connect` for clients to connect to Kafka Connect REST endpoint.

## Configuration

You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

```console
$ helm install --name my-kafka-connect -f my-values.yaml ./cp-kafka-connect
helm install --name my-kafka-connect -f my-values.yaml ./cp-kafka-connect
```

> **Tip**: A default [values.yaml](values.yaml) is provided
### Kafka Connect Deployment
The configuration parameters in this section control the resources requested and utilized by the cp-kafka-connect chart.

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `replicaCount` | The number of Kafka Connect Servers. | `1` |
The configuration parameters in this section control the resources requested and utilized by the `cp-kafka-connect` chart.

| Parameter | Description | Default |
| ----------------- | ------------------------------------- | --------- |
| `replicaCount` | The number of Kafka Connect Servers. | `1` |

### Image

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `image` | Docker Image of Confluent Kafka Connect. | `confluentinc/cp-kafka-connect` |
| `imageTag` | Docker Image Tag of Confluent Kafka Connect. | `4.1.1` |
| `imageTag` | Docker Image Tag of Confluent Kafka Connect. | `5.0.0` |
| `imagePullPolicy` | Docker Image Tag of Confluent Kafka Connect. | `IfNotPresent` |

### Port

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `servicePort` | The port on which the Kafka Connect will be available and serving requests. | `8082` |

### Kafka Connect Worker Configurations

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `connectKeyConverter` | Converter class for key Connect data. This controls the format of the data that will be written to Kafka for source connectors or read from Kafka for sink connectors. Popular formats include Avro and JSON. |`"io.confluent.connect.avro.AvroConverter"`|
Expand All @@ -96,6 +113,7 @@ The configuration parameters in this section control the resources requested and
| `pluginPath` | The comma-separated list of paths to directories that contain Kafka Connect plugins. |`"/usr/share/java"`|

### Resources

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details |
Expand All @@ -104,20 +122,24 @@ The configuration parameters in this section control the resources requested and
| `resources.requests.limit` | The upper limit memory usage for a Kafka Connect Pod. | see [values.yaml](values.yaml) for details |

### JMX Configuration

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` |

### Prometheus JMX Exporter Configuration

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` |
| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` |
| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` |
| `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` |
| `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` |

## Dependencies

### Kafka

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `kafka.bootstrapServers` | Bootstrap Servers for Kafka Connect | `""` |
2 changes: 1 addition & 1 deletion charts/cp-kafka-connect/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ replicaCount: 1
## Image Info
## ref: https://hub.docker.com/r/confluentinc/cp-kafka/
image: confluentinc/cp-kafka-connect
imageTag: 4.1.1
imageTag: 5.0.0

## Specify a imagePullPolicy
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
Expand Down
52 changes: 37 additions & 15 deletions charts/cp-kafka-rest/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,40 +1,50 @@
# CP-REST Proxy Helm Chart

This chart bootstraps a deployment of a Confluent REST Proxy

## Prerequisites

* Kubernetes 1.9.2+
* Helm 2.8.2+
* A healthy and accessible Zookeeper Ensemble of the Kafka Cluster

## Developing Environment:
* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/)
## Developing Environment

* [Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service)
* [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/)

## Docker Image Source

## Docker Image Source:
* DockerHub - ConfluentInc: https://hub.docker.com/u/confluentinc/
* [DockerHub -> ConfluentInc](https://hub.docker.com/u/confluentinc/)

## Installing the Chart

### Install along with cp-helm-charts

```console
$ git clone https://github.com/confluentinc/cp-helm-charts.git
$ helm install cp-helm-charts
git clone https://github.com/confluentinc/cp-helm-charts.git
helm install cp-helm-charts
```

To install with a specific name, you can do:

```console
$ helm install --name my-confluent cp-helm-charts
helm install --name my-confluent cp-helm-charts
```

### Install with a existing cp-kafka and cp-schema-registry release

```console
$ helm install --set cp-zookeeper.url="unhinged-robin-cp-zookeeper:2181",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-rest
helm install --set cp-zookeeper.url="unhinged-robin-cp-zookeeper:2181",cp-schema-registry.url="lolling-chinchilla-cp-schema-registry:8081" cp-helm-charts/charts/cp-kafka-rest
```

### Installed Components

You can use `helm status <release name>` to view all of the installed components.

For example:
```console{%raw}

```console
$ helm status lolling-chinchilla
NAMESPACE: default
STATUS: DEPLOYED
Expand All @@ -56,43 +66,49 @@ hopping-salamander-cp-kafka-rest-67b86cff98-qxrd8 1/1 Running 0 1m
NAME DATA AGE
hopping-salamander-cp-kafka-rest-jmx-configmap 1 1s
```
There are

There are
1. A [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) `hopping-salamander-cp-kafka-rest` which contains 1 REST Proxy [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/): `hopping-salamander-cp-kafka-rest-67b86cff98-qxrd8`.
1. A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `hopping-salamander-cp-kafka-rest` for clients to connect to REST Proxy.
1. A [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) which contains configuration for Prometheus JMX Exporter.
1. (Optional) A [Service](https://kubernetes.io/docs/concepts/services-networking/service/) `hopping-salamander-cp-kafka-restproxy-external` for clients to connect to REST Proxy from outside.

## Configuration

You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

```console
$ helm install --name my-rest-proxy -f my-values.yaml ./cp-kafka-rest
helm install --name my-rest-proxy -f my-values.yaml ./cp-kafka-rest
```

> **Tip**: A default [values.yaml](values.yaml) is provided
### REST Proxy Deployment
The configuration parameters in this section control the resources requested and utilized by the cp-kafka-rest chart.

The configuration parameters in this section control the resources requested and utilized by the `cp-kafka-rest` chart.

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `replicaCount` | The number of REST Proxy Servers. | `1` |

### Image

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `image` | Docker Image of Confluent REST Proxy. | `confluentinc/cp-kafka-rest` |
| `imageTag` | Docker Image Tag of Confluent REST Proxy. | `4.1.1` |
| `imageTag` | Docker Image Tag of Confluent REST Proxy. | `5.0.0` |
| `imagePullPolicy` | Docker Image Tag of Confluent REST Proxy. | `IfNotPresent` |

### Port

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `servicePort` | The port on which the REST Proxy will be available and serving requests. | `8082` |

### Resources

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `resources.requests.cpu` | The amount of CPU to request. | see [values.yaml](values.yaml) for details |
Expand All @@ -101,32 +117,38 @@ The configuration parameters in this section control the resources requested and
| `resources.requests.limit` | The upper limit memory usage for a REST Proxy Pod. | see [values.yaml](values.yaml) for details |

### JMX Configuration

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `jmx.port` | The jmx port which JMX style metrics are exposed. | `5555` |

### Prometheus JMX Exporter Configuration

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `prometheus.jmx.enabled` | Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus. | `true` |
| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` |
| `prometheus.jmx.image` | Docker Image for Prometheus JMX Exporter container. | `solsson/kafka-prometheus-jmx-exporter@sha256` |
| `prometheus.jmx.imageTag` | Docker Image Tag for Prometheus JMX Exporter container. | `a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8` |
| `prometheus.jmx.port` | JMX Exporter Port which exposes metrics in Prometheus format for scraping. | `5556` |

### External Access

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `external.enabled` | whether or not to allow external access to Kafka REST Proxy | `false` |
| `external.type` | `Kubernetes Service Type` to expose Kafka REST Proxy to external | `LoadBalancer` |

## Dependencies

### Zookeeper

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `cp-zookeeper.url` | Service name of Zookeeper cluster (Not needed if this is installed along with cp-kafka chart). | `""` |
| `cp-zookeeper.clientPort` | Port of Zookeeper Cluster | `2181` |

### Schema Registry (optional)

| Parameter | Description | Default |
| --------- | ----------- | ------- |
| `cp-schema-registry.url` | Service name of Schema Registry (Not needed if this is installed along with cp-kafka chart). | `""` |
Expand Down
2 changes: 1 addition & 1 deletion charts/cp-kafka-rest/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ replicaCount: 1
## Image Info
## ref: https://hub.docker.com/r/confluentinc/cp-kafka/
image: confluentinc/cp-kafka-rest
imageTag: 4.1.1
imageTag: 5.0.0

## Specify a imagePullPolicy
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
Expand Down
Loading

0 comments on commit e62d1ad

Please sign in to comment.