-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ISSUE #146] Sync and translate connetor documents from source modules (
#147) * Sync optimized documents * Translate connectors document * Optimize connectors/connector briefing and more translation * Optimize titles and sequence * Mark knative doc as outdated * Remove outdated mark
- Loading branch information
Showing
6 changed files
with
174 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Connectors | ||
|
||
## Connector | ||
|
||
A connector is a bridge that interacts with a specific external service or underlying data source (e.g., Databases) on behalf of user applications. A connector is either a Source or a Sink. | ||
|
||
## Source | ||
|
||
A source connector obtains data from an underlying data producer, and delivers it to targets after original data has been transformed into CloudEvents. It doesn't limit the way how a source retrieves data. (e.g., A source may pull data from a message queue or act as an HTTP server waiting for data sent to it). | ||
|
||
CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems. | ||
|
||
## Sink | ||
|
||
A sink connector receives CloudEvents and does some specific business logics. (e.g., A MySQL Sink extracts useful data from CloudEvents and writes them to a MySQL database). | ||
|
||
## Implements | ||
|
||
Add a new connector by implementing the source/sink interface using [eventmesh-openconnect-java](https://github.com/apache/eventmesh/tree/master/eventmesh-openconnect/eventmesh-openconnect-java). | ||
|
||
## Connector Status | ||
|
||
| Connector Name | Source | Sink | | ||
|:------------------------------------------------:|:-----------:|:-------:| | ||
| [RocketMQ](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-rocketmq) | ✅ | ✅ | | ||
| ChatGPT | ⬜ | ⬜ | | ||
| ClickHouse | ⬜ | ⬜ | | ||
| [DingTalk](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-dingtalk) | ⬜ | ✅ | | ||
| Email | ⬜ | ⬜ | | ||
| [Feishu/Lark](./lark-connector) | ⬜ | ✅ | | ||
| [File](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-file) | ✅ | ✅ | | ||
| GitHub | ⬜ | ⬜ | | ||
| [HTTP](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-http) | ✅ | ⬜ | | ||
| [Jdbc](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-jdbc) | ⬜ | ✅ | | ||
| [Kafka](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-kafka) | ✅ | ✅ | | ||
| [Knative](./knative-connector) | ✅ | ✅ | | ||
| [MongoDB](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-mongodb) | ✅ | ✅ | | ||
| [OpenFunction](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-openfunction) | ✅ | ✅ | | ||
| [Pravega](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-pravega) | ✅ | ✅ | | ||
| [Prometheus](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-prometheus) | ✅ | ⬜ | | ||
| [Pulsar](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-pulsar) | ✅ | ✅ | | ||
| [RabbitMQ](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-rabbitmq) | ✅ | ✅ | | ||
| [Redis](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-redis) | ✅ | ✅ | | ||
| [S3 File](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-s3) | ⬜ | ✅ | | ||
| [Slack](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-slack) | ⬜ | ✅ | | ||
| [Spring](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-spring) | ✅ | ✅ | | ||
| [WeCom](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-wecom) | ⬜ | ✅ | | ||
| [WeChat](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-wechat) | ⬜ | ✅ | | ||
| More connectors will be added... | N/A | N/A | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Lark | ||
|
||
## Lark Sink Server Config And Start | ||
|
||
Before using eventmesh-connector-lark to sink events, you need to configure the server. | ||
- Please customize `sinkEnable``=`true`/`false` in `/resource/server-config.yml` to turn on/off the sink function. | ||
- Regarding `/resource/sink-config.yml`, only the configuration under `sinkConnectorConfig` is explained here: | ||
- `connectorName`, specify the connector name | ||
- (required) `appId`, the appId obtained from lark | ||
- (required) `appSecret`, the appSecret obtained from lark | ||
- `receiveIdType`, the type of receiving Id, the default and recommended use is `open_id`. Optional open_id/user_id/union_id/email/chat_id. | ||
- (Required) `receiveId`, receive Id, needs to correspond to `receiveIdType`. | ||
- `sinkAsync`, whether to asynchronously sink events | ||
- `maxRetryTimes`, the maximum number of retransmissions when the sink event fails. The default is 3 times. | ||
- `retryDelayInMills`, when the sink event fails, the time interval for retransmitting the event. Default is 1s, unit is milliseconds. | ||
|
||
|
||
## Sink CloudEvent To Lark | ||
|
||
When using the eventmesh-connector-lark sinking event, you need to add the corresponding extension filed in CloudEvent: | ||
- When key=`templatetype4lark`, value=`text`/`markdown`, indicating the text type of the event | ||
- When the text type is markdown, you can add extension: key=`markdownmessagetitle4lark`, value indicates the title of the event. | ||
- When key=`atusers4lark`, value=`id-0,name-0;id-1,name-1`, indicating that the event requires `@`certain users | ||
- It is recommended to use **open_id** for id. | ||
- When the text is of text type, the id can be **open_id/union_id/user_id**; when the text is of markdown type, the id can be **open_id/user_id**. In particular, when the application type is [custom robot](https://open.larksuite.com/document/ukTMukTMukTM/ucTM5YjL3ETO24yNxkjN) and the text is of markdown type, only the use of **open_id** to `@` the user is supported. | ||
- When the text is of text type and the id is invalid, name will be used instead for display; when the text is of markdown type and the id is invalid, an exception will be thrown directly (you should try to ensure the correctness of the id, and name can be considered omitted). | ||
- When key=`atall4lark`, value=`true`/`false`, indicating that the event requires `@` everyone. | ||
|
||
|
||
## Lark Open Platform API | ||
|
||
For the Lark open platform API involved in this module, please click the following link: | ||
- **Send Message**, please [view here](https://open.larksuite.com/document/server-docs/im-v1/message/create?appId=cli_a5e1bc31507ed00c) | ||
- **text**, please [view here](https://open.larksuite.com/document/server-docs/im-v1/message-content-description/create_json#c9e08671) | ||
- **markdown**, please [view here](https://open.larksuite.com/document/common-capabilities/message-card/message-cards-content/using-markdown-tags) |
4 changes: 3 additions & 1 deletion
4
...cument/03-connect/07-knative-connector.md → ...cument/03-connect/02-knative-connector.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
49 changes: 49 additions & 0 deletions
49
...usaurus-plugin-content-docs/current/design-document/03-connect/00-connectors.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# 连接器简介 | ||
|
||
## 连接器类型 | ||
|
||
一个连接器就是一座桥梁,代表用户应用程序与特定的外部服务或底层数据源(例如数据库)进行交互。连接器的类型可以是源(Source)或汇(Sink)。 | ||
|
||
## 数据源(Source 端) | ||
|
||
源连接器从底层数据生产者获取数据,并在原始数据被转换为 CloudEvents 后将其传递给目标。源连接器不限制源如何检索数据(例如,源可以从消息队列中获取数据,也可以充当等待接收数据的 HTTP 服务器)。 | ||
|
||
CloudEvents 是一种以通用格式描述事件数据的规范,以提供服务、平台和系统之间的互操作性。 | ||
|
||
## 数据汇(Sink 端) | ||
|
||
汇连接器接收 CloudEvents 并执行特定的业务逻辑(例如,MySQL 的汇连接器从 CloudEvents 中提取有用的数据,并将其写入 MySQL 数据库)。 | ||
|
||
## 实现连接器 | ||
|
||
使用 [eventmesh-openconnect-java](https://github.com/apache/eventmesh/tree/master/eventmesh-openconnect/eventmesh-openconnect-java) 实现 Source/Sink 接口即可添加新的连接器。 | ||
|
||
## 连接器实现状态 | ||
|
||
| 连接器名称 | 源 | 汇 | | ||
|:------------------------------------------:|:------:|:------:| | ||
| [RocketMQ](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-rocketmq) | ✅ | ✅ | | ||
| ChatGPT | ⬜ | ⬜ | | ||
| ClickHouse | ⬜ | ⬜ | | ||
| [钉钉](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-dingtalk) | ⬜ | ✅ | | ||
| 邮件 | ⬜ | ⬜ | | ||
| [飞书/Lark](./lark-connector) | ⬜ | ✅ | | ||
| [文件](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-file) | ✅ | ✅ | | ||
| GitHub | ⬜ | ⬜ | | ||
| [HTTP](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-http) | ✅ | ⬜ | | ||
| [Jdbc](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-jdbc) | ⬜ | ✅ | | ||
| [Kafka](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-kafka) | ✅ | ✅ | | ||
| [Knative](./knative-connector) | ✅ | ✅ | | ||
| [MongoDB](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-mongodb) | ✅ | ✅ | | ||
| [OpenFunction](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-openfunction) | ✅ | ✅ | | ||
| [Pravega](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-pravega) | ✅ | ✅ | | ||
| [Prometheus](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-prometheus) | ✅ | ⬜ | | ||
| [Pulsar](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-pulsar) | ✅ | ✅ | | ||
| [RabbitMQ](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-rabbitmq) | ✅ | ✅ | | ||
| [Redis](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-redis) | ✅ | ✅ | | ||
| [S3 存储](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-s3) | ⬜ | ✅ | | ||
| [Slack](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-slack) | ⬜ | ✅ | | ||
| [Spring](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-spring) | ✅ | ✅ | | ||
| [企业微信](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-wecom) | ⬜ | ✅ | | ||
| [微信](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors/eventmesh-connector-wechat) | ⬜ | ✅ | | ||
| 更多连接器正在计划中... | N/A | N/A | |
35 changes: 35 additions & 0 deletions
35
...rus-plugin-content-docs/current/design-document/03-connect/01-lark-connector.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# 飞书/Lark | ||
|
||
## Lark Sink Server 的配置与启动 | ||
|
||
使用 eventmesh-connector-lark 下沉事件之前,需要进行 server 的配置。 | ||
- 请在`/resource/server-config.yml`中自定义`sinkEnable``=`true`/`false`以开启/关闭 sink 功能。 | ||
- 关于`/resource/sink-config.yml`,在此仅说明`sinkConnectorConfig`下的配置: | ||
- `connectorName`, 指定 connector 名称 | ||
- (必需)`appId`, lark 中获取的 appId | ||
- (必需)`appSecret`, lark 中获取的 appSecret | ||
- `receiveIdType`,接收 Id 的类型,默认且推荐使用`open_id`。可选 open_id/user_id/union_id/email/chat_id。 | ||
- (必需)`receiveId`, 接收 Id,需要和`receiveIdType`对应。 | ||
- `sinkAsync`, 是否异步下沉事件 | ||
- `maxRetryTimes`, sink 事件失败时,最大重传的次数。默认 3 次。 | ||
- `retryDelayInMills`, sink 事件失败时,重传事件的时间间隔。默认 1s,单位为毫秒。 | ||
|
||
|
||
## 可下沉飞书的 CLoudEvent | ||
|
||
使用 eventmesh-connector-lark 下沉事件时,需要在 CloudEvent 中添加对应的 extension filed: | ||
- 当 key=`templatetype4lark`时,value=`text`/`markdown`,表明该事件的文本类型 | ||
- 当文本类型为 markdown 时,可以添加 extension:key=`markdownmessagetitle4lark`,value 表明该事件的标题。 | ||
- 当 key=`atusers4lark`时,value=`id-0,name-0;id-1,name-1`,表明该事件需要`@`某些用户 | ||
- id 推荐使用**open_id**。 | ||
- 当文本属于 text 类型时,id 可以是**open_id/union_id/user_id**;当文本属于 markdown 类型时,id 可以是**open_id/user_id**。特别地,当应用类型为[自定义机器人](https://open.feishu.cn/document/ukTMukTMukTM/ucTM5YjL3ETO24yNxkjN)且文本属于 markdown 类型,则仅支持使用**open_id**来`@`用户。 | ||
- 当文本属于 text 类型且 id 无效时,将利用 name 代替展示;当文本属于 markdown 类型时且 id 无效时,直接抛出异常 (您应该尽量保证 id 的正确性,而 name 则可以考虑省略)。 | ||
- 当 key=`atall4lark`时,value=`true`/`false`,表明该事件需要`@`所有人。 | ||
|
||
|
||
## 飞书开放平台 API | ||
|
||
有关该模块涉及到的飞书开放平台 API,请点击以下链接: | ||
- **发送消息**,请[查看这里](https://open.feishu.cn/document/server-docs/im-v1/message/create?appId=cli_a5e1bc31507ed00c) | ||
- **text**,请[查看这里](https://open.feishu.cn/document/server-docs/im-v1/message-content-description/create_json#c9e08671) | ||
- **markdown**,请[查看这里](https://open.feishu.cn/document/common-capabilities/message-card/message-cards-content/using-markdown-tags) |
4 changes: 3 additions & 1 deletion
4
...cument/03-connect/07-knative-connector.md → ...cument/03-connect/02-knative-connector.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,6 @@ | ||
# Knative Connector 插件 | ||
# Knative | ||
|
||
>随着 Knative Connector 设计的变动,这篇文档目前暂时过时了。 | ||
## 准备 | ||
|
||
|