-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* add kafka connector doc * modify punctuation
- Loading branch information
Showing
4 changed files
with
119 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Kafka | ||
|
|
||
| ## KafkaSinkConnector：From EventMesh to Kafka | ||
|
|
||
| 1. Start your EventMesh Runtime. | ||
| 2. Enable sinkConnector and check `sink-config.yml`. | ||
| 3. Start your KafkaConnectServer, which will subscribe to the topic defined in `pubSubConfig.subject` in the EventMesh Runtime, The received data is published to `connectorConfig.topic` in Kafka. | ||
| 4. Using the Topic specified in `pubSubConfig.subject`, send a message to EventMesh, which you will then see in Kafka. | ||
|
|
||
| ```yaml | ||
| # Common configuration | ||
| pubSubConfig: | ||
| meshAddress: 127.0.0.1:10000 | ||
| subject: TopicTest | ||
| idc: FT | ||
| env: PRD | ||
| group: kafkaSink | ||
| appId: 5031 | ||
| userName: kafkaSinkUser | ||
| passWord: kafkaPassWord | ||
| connectorConfig: | ||
| connectorName: kafkaSink | ||
| # kafka connection parameters ↓ | ||
| bootstrapServers: 127.0.0.1:9092 | ||
| topic: TopicTest | ||
| keyConverter: org.apache.kafka.common.serialization.StringSerializer | ||
| valueConverter: org.apache.kafka.common.serialization.StringSerializer | ||
| ``` | ||
| ## KafkaSourceConnector：From Kafka to EventMesh | ||
| 1. Start your EventMesh Runtime. | ||
| 2. Enable sourceConnector and check `source-config.yml`. | ||
| 3. Start your KafkaConnectServer, which will subscribe to Kafka's `connectorConfig.topic` and send the read data to `pubSubConfig.subject` in the EventMesh Runtime. | ||
| 4. Send a message to Kafka, and you'll receive it in EventMesh. | ||
|
|
||
| ```yaml | ||
| # Common configuration | ||
| pubSubConfig: | ||
| meshAddress: 127.0.0.1:10000 | ||
| subject: TopicTest | ||
| idc: FT | ||
| env: PRD | ||
| group: kafkaSource | ||
| appId: 5032 | ||
| userName: kafkaSourceUser | ||
| passWord: kafkaPassWord | ||
| connectorConfig: | ||
| connectorName: kafkaSource | ||
| # kafka connection parameters ↓ | ||
| bootstrapServers: 127.0.0.1:9092 | ||
| topic: TopicTest | ||
| groupID: kafkaSource | ||
| sessionTimeoutMS: 10000 | ||
| maxPollRecords: 1000 | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
56 changes: 56 additions & 0 deletions
56
...us-plugin-content-docs/current/design-document/03-connect/11-kafka-connector.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Kafka | ||
|
|
||
| ## KafkaSinkConnector：从 EventMesh 到 Kafka | ||
|
|
||
| 1. 启动你的 EventMesh Runtime。 | ||
| 2. 启用 sinkConnector 并检查 `sink-config.yml`。 | ||
| 3. 启动你的 KafkaConnectServer，它将订阅到 EventMesh Runtime 中 `pubSubConfig.subject` 中定义的主题，并将收到的数据发布到 Kafka 中的 `connectorConfig.topic`。 | ||
| 4. 使用在 `pubSubConfig.subject` 中指定的 Topic，向 EventMesh 发送消息，然后你将在 Kafka 看到该消息。 | ||
|
|
||
| ```yaml | ||
| # 公共配置 | ||
| pubSubConfig: | ||
| meshAddress: 127.0.0.1:10000 | ||
| subject: TopicTest | ||
| idc: FT | ||
| env: PRD | ||
| group: kafkaSink | ||
| appId: 5031 | ||
| userName: kafkaSinkUser | ||
| passWord: kafkaPassWord | ||
| connectorConfig: | ||
| connectorName: kafkaSink | ||
| # kafka连接参数 ↓ | ||
| bootstrapServers: 127.0.0.1:9092 | ||
| topic: TopicTest | ||
| keyConverter: org.apache.kafka.common.serialization.StringSerializer | ||
| valueConverter: org.apache.kafka.common.serialization.StringSerializer | ||
| ``` | ||
| ## KafkaSourceConnector：从 Kafka 到 EventMesh | ||
| 1. 启动你的 EventMesh Runtime。 | ||
| 2. 启用 sourceConnector 并检查 `source-config.yml`。 | ||
| 3. 启动你的 KafkaConnectServer，它将订阅 Kafka的 `connectorConfig.topic `，并将读取的数据发送到 EventMesh Runtime 中的 `pubSubConfig.subject`。 | ||
| 4. 向 Kafka 发送一个消息，然后你将在 EventMesh 中接收到该消息。 | ||
|
|
||
| ```yaml | ||
| # 公共配置 | ||
| pubSubConfig: | ||
| meshAddress: 127.0.0.1:10000 | ||
| subject: TopicTest | ||
| idc: FT | ||
| env: PRD | ||
| group: kafkaSource | ||
| appId: 5032 | ||
| userName: kafkaSourceUser | ||
| passWord: kafkaPassWord | ||
| connectorConfig: | ||
| connectorName: kafkaSource | ||
| # kafka连接参数 ↓ | ||
| bootstrapServers: 127.0.0.1:9092 | ||
| topic: TopicTest | ||
| groupID: kafkaSource | ||
| sessionTimeoutMS: 10000 | ||
| maxPollRecords: 1000 | ||
| ``` |