diff --git a/en_US/data-integration/data-bridge-sqlserver.md b/en_US/data-integration/data-bridge-sqlserver.md index 1819c676d..5704ccb23 100644 --- a/en_US/data-integration/data-bridge-sqlserver.md +++ b/en_US/data-integration/data-bridge-sqlserver.md @@ -42,75 +42,13 @@ The data integration with Microsoft SQL Server offers a range of features and be ## Before You Start -This section describes the preparations you need to complete before you start to create the Microsoft SQL Server data integration, including how to install and connect to the Microsoft SQL Server, create database and data tables, and install and configure the ODBC driver. +This section describes the preparations you need to complete before you start creating the Microsoft SQL Server data integration, including how to install and configure the ODBC driver, install and connect to the Microsoft SQL Server, ands create database and data tables. ### Prerequisites - Knowledge about EMQX data integration [rules](./rules.md) - Knowledge about [data integration](./data-bridges.md) -### Install and Connect to Microsoft SQL Server - -This section describes how to start Microsoft SQL Server 2019 on Linux/MacOS using Docker images and use `sqlcmd` to connect to Microsoft SQL Server. For other installation methods of Microsoft SQL Server, please refer to [Microsoft SQL Server Installation Guide](https://learn.microsoft.com/en-us/sql/database-engine/install-windows/install-sql-server?view=sql-server-ver16). - -1. Install Microsoft SQL Server via Docker, and then start the docker image with the command below. Use `mqtt_public1` as the password. For the password policy of Microsoft SQL Server, see [Password Complexity](https://learn.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver16#password-complexity). - - Note: By starting a Docker container with the environment variable `ACCEPT_EULA=Y` you agree to the terms of Microsoft EULA, see also [End-User Licensing Agreement](https://go.microsoft.com/fwlink/?linkid=857698). - - ```bash - # To start the Microsoft SQL Server docker image and set the password as `mqtt_public1` - $ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2022-CU15-ubuntu-22.04 - ``` - -2. Access the container. - - ```bash - docker exec -it sqlserver bash - ``` - -3. Enter the preset password to connect to the server in the container. The characters are not echoed when entering the password. Click `Enter` directly after entering the password. - - ```bash - $ /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P mqtt_public1 -N -C - 1> - ``` - - ::: tip - - The `mssql-tools18` package have been installed in the Microsoft SQL Server container provided by Microsoft, but the executable file is not in `$PATH`. Therefore, you need to specify the executable file path for `sqlcmd` before proceeding. As for the Docker deployment in this example, the file path should be `/opt`. - - For more information on how to use `mssql-tools18`, see [sqlcmd-utility](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16). - - ::: - -So far, the Microsoft SQL Server 2022 instance has been deployed and can be connected. - -### Create Database and Data Tables - -Use the connection created from the previous section and the following SQL statements to create data tables. - -- Create the following data table for storing the MQTT message, including the message ID, topic, QoS, payload, and publish time of each message. - - ```sql - CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - msgid VARCHAR(64) NULL, - topic VARCHAR(100) NULL, - qos tinyint NOT NULL DEFAULT 0, - payload VARCHAR(100) NULL, - arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - -- Create the following data table for recording the online/offline status of clients. - - ```sql - CREATE TABLE dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - clientid VARCHAR(255) NULL, - event_type VARCHAR(255) NULL, - event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - ### Install and Configure ODBC Driver You need to configure the ODBC driver to be able to access the Microsoft SQL Server database. You can use either FreeTDS or the msodbcsql18 driver provided by Microsoft as the ODBC driver. @@ -173,6 +111,7 @@ Check that the DSN Name in `odbcinst.ini` should be `ms-sql` if you install the This section introduces how to install and configure FreeTDS as an ODBC driver on some of the mainstream distributions. Install and configure FreeTDS ODBC driver on MacOS: + ```bash $ brew install unixodbc freetds $ vim /usr/local/etc/odbcinst.ini @@ -185,6 +124,7 @@ FileUsage = 1 ``` Install and configure FreeTDS ODBC driver on Centos: + ```bash $ yum install unixODBC unixODBC-devel freetds freetds-devel perl-DBD-ODBC perl-local-lib $ vim /etc/odbcinst.ini @@ -199,6 +139,7 @@ FileUsage = 1 ``` Install and configure FreeTDS ODBC driver on Ubuntu (Take Ubuntu20.04 as an example, for other versions, please refer to the official ODBC documentation): + ```bash $ apt-get install unixodbc unixodbc-dev tdsodbc freetds-bin freetds-common freetds-dev libdbd-odbc-perl liblocal-lib-perl $ vim /etc/odbcinst.ini @@ -210,6 +151,74 @@ Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so FileUsage = 1 ``` +### Install and Connect to Microsoft SQL Server + +This section describes how to start Microsoft SQL Server 2019 on Linux/MacOS using Docker images and use `sqlcmd` to connect to Microsoft SQL Server. For other installation methods of Microsoft SQL Server, please refer to [Microsoft SQL Server Installation Guide](https://learn.microsoft.com/en-us/sql/database-engine/install-windows/install-sql-server?view=sql-server-ver16). + +1. Install Microsoft SQL Server via Docker, and then start the docker image with the command below. Use `mqtt_public1` as the password. For the password policy of Microsoft SQL Server, see [Password Complexity](https://learn.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver16#password-complexity). + + Note: By starting a Docker container with the environment variable `ACCEPT_EULA=Y` you agree to the terms of Microsoft EULA, see also [End-User Licensing Agreement](https://go.microsoft.com/fwlink/?linkid=857698). + + ```bash + # To start the Microsoft SQL Server docker image and set the password as `mqtt_public1` + $ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2022-CU15-ubuntu-22.04 + ``` + +2. Access the container. + + ```bash + docker exec -it sqlserver bash + ``` + +3. Enter the preset password to connect to the server in the container. The characters are not echoed when entering the password. Click `Enter` directly after entering the password. + + ```bash + $ /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P mqtt_public1 -N -C + 1> + ``` + + ::: tip + + The `mssql-tools18` package have been installed in the Microsoft SQL Server container provided by Microsoft, but the executable file is not in `$PATH`. Therefore, you need to specify the executable file path for `sqlcmd` before proceeding. As for the Docker deployment in this example, the file path should be `/opt`. + + For more information on how to use `mssql-tools18`, see [sqlcmd-utility](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16). + + ::: + +So far, the Microsoft SQL Server 2022 instance has been deployed and can be connected. + +### Create Database and Data Tables + +Use the connection created from the previous section and the following SQL statements to create data tables. + +::: tip + +Due to ODBC interface limitations, if you need to write Unicode characters, such as CJK characters or Emoji, you need to use a function to convert them into binary format before inserting them. When creating a table, set the column type that stores Unicode characters to `NVARCHAR`. + +::: + +- Create the following data table for storing the MQTT message, including the message ID, topic, QoS, payload, and publish time of each message. + + ```sql + CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, + msgid VARCHAR(64) NULL, + topic VARCHAR(100) NULL, + qos tinyint NOT NULL DEFAULT 0, + payload VARCHAR(100) NULL, + arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); + GO + ``` + +- Create the following data table for recording the online/offline status of clients. + + ```sql + CREATE TABLE dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, + clientid VARCHAR(255) NULL, + event_type VARCHAR(255) NULL, + event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); + GO + ``` + ## Create a Connector This section demonstrates how to create a Connector to connect the Sink to the Microsoft SQL server. @@ -251,6 +260,22 @@ This section demonstrates how to create a rule in the Dashboard for processing m ::: tip + Due to ODBC interface limitations, if you need to write Unicode characters, such as CJK characters or Emoji, you need to use a function to convert them into binary format before inserting them. + + You can use built-in functions to convert strings to UTF-16-little-endian encoded binary strings when creating rules. For example: + + ```sql + SELECT + sqlserver_bin2hexstr(str_utf16_le(payload)) as payload + * + FROM + "t/#" + ``` + + ::: + + ::: tip + If you are a beginner user, click **SQL Examples** and **Enable Test** to learn and test the SQL rule. ::: @@ -271,6 +296,18 @@ This section demonstrates how to create a rule in the Dashboard for processing m insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, ${payload} ) ``` + ::: tip + + Due to ODBC interface limitations, if you need to write Unicode characters, such as CJK characters or Emoji, you need to use a function to convert them into binary format before inserting them. + + You can use the `CONVERT` function in the SQL template to convert the corresponding binary data into a string by Microsoft SQL Server. + + ```sql + insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, CONVERT(NVARCHAR(100), ${payload}) ) + ``` + + ::: + If a placeholder variable is undefined in the SQL template, you can toggle the **Undefined Vars as Null** switch above the **SQL template** to define the rule engine behavior: - **Disabled** (default): The rule engine can insert the string `undefined` into the database. @@ -295,36 +332,6 @@ You have now successfully created the rule for the Microsoft SQL Server Sink. Yo You can also click **Integration** -> **Flow Designer** to view the topology and you can see that the messages under topic `t/#` are sent and saved to Microsoft SQL Server after parsing by rule `my_rule`. -Due to ODBC interface limitations, if you need to write Unicode characters, such as CJK characters or Emoji, you need to use a function to convert them into binary format before inserting them. - -- When creating a table, set the column type that needs to store Unicode characters to `NVARCHAR`. - - ```sql{5} - CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - msgid VARCHAR(64) NULL, - topic VARCHAR(100) NULL, - qos tinyint NOT NULL DEFAULT 0, - payload NVARCHAR(100) NULL, - arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - -- Use built-in functions to convert strings to UTF-16-little-endian encoded binary strings when creating rules. - - ```sql{2} - SELECT - sqlserver_bin2hexstr(str_utf16_le(payload)) as payload - * - FROM - "t/#" - ``` - -- Use the `CONVERT` function in the SQL template to convert the corresponding binary data into a string by Microsoft SQL Server. - - ```sql - insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, CONVERT(NVARCHAR(100), ${payload}) ) - ``` - ## Create a Rule with Microsoft SQL Server for Events Recording This section demonstrates how to create a rule for recording the clients' online/offline status and storing the events data to the Microsoft SQL Server table `dbo.t_mqtt_events` via a configured Sink. diff --git a/zh_CN/data-integration/data-bridge-clickhouse.md b/zh_CN/data-integration/data-bridge-clickhouse.md index 5b5289407..aa02dfe11 100644 --- a/zh_CN/data-integration/data-bridge-clickhouse.md +++ b/zh_CN/data-integration/data-bridge-clickhouse.md @@ -150,7 +150,7 @@ ClickHouse 数据集成是 EMQX 中的开箱即用功能，旨在结合 MQTT 的 ::: tip - 如果您初次使用 SQL，可以点击 **SQL 示例** 和**启用调试**来学习和测试规则 SQL 的结果。 + 如果可能，应该始终启用此选项；禁用该选项仅用于确保向后兼容性。 ::: diff --git a/zh_CN/data-integration/data-bridge-dynamo.md b/zh_CN/data-integration/data-bridge-dynamo.md index 5e9fb9410..cf436a790 100644 --- a/zh_CN/data-integration/data-bridge-dynamo.md +++ b/zh_CN/data-integration/data-bridge-dynamo.md @@ -196,7 +196,7 @@ DynamoDB 数据集成是 EMQX 中的开箱即用功能，它结合了 EMQX 的 ::: tip - 如果您初次使用 SQL，可以点击 **SQL 示例** 和**启用调试**来学习和测试规则 SQL 的结果。 + 如果可能，应该始终启用此选项；禁用该选项仅用于确保向后兼容性。 ::: diff --git a/zh_CN/data-integration/data-bridge-mysql.md b/zh_CN/data-integration/data-bridge-mysql.md index da5bac4ad..0cab9a839 100644 --- a/zh_CN/data-integration/data-bridge-mysql.md +++ b/zh_CN/data-integration/data-bridge-mysql.md @@ -163,7 +163,7 @@ CREATE TABLE emqx_client_events ( ::: tip - 如果您初次使用 SQL，可以点击 **SQL 示例**和**启用调试**来学习和测试规则 SQL 的结果。 + 如果可能，应该始终启用此选项；禁用该选项仅用于确保向后兼容性。 ::: diff --git a/zh_CN/data-integration/data-bridge-sqlserver.md b/zh_CN/data-integration/data-bridge-sqlserver.md index 363c2cb94..a07b34ad7 100644 --- a/zh_CN/data-integration/data-bridge-sqlserver.md +++ b/zh_CN/data-integration/data-bridge-sqlserver.md @@ -42,78 +42,13 @@ Microsoft SQL Server 数据集成是 EMQX 的开箱即用功能，结合了 EMQX ## 准备工作 -本节介绍了在 EMQX 中创建 Microsoft SQL Server 数据集成之前需要做的准备工作，包括如何设置 Microsoft SQL Server 服务器并创建数据库和数据表、安装并配置 ODBC 驱动程序。 +本节介绍了在 EMQX 中创建 Microsoft SQL Server 数据集成之前需要做的准备工作，包括如何安装并配置 ODBC 驱动程序、设置 Microsoft SQL Server 服务器并创建数据库和数据表。 ### 前置准备 - 了解[规则](./rules.md)。 - 了解[数据集成](./data-bridges.md)。 -### 安装并连接到 Microsoft SQL Server - -本节描述如何使用 Docker 镜像在 Linux/MacOS 安装启动 Microsoft SQL Server 2019 以及如何使用 `sqlcmd` 连接到 Microsoft SQL Server。关于其他 Microsoft SQL Server 的安装方式，请参阅微软提供的 [Microsoft SQL Server 安装指南](https://learn.microsoft.com/zh-cn/sql/database-engine/install-windows/install-sql-server?view=sql-server-ver16)。 - -1. 通过 Docker 安装并启动 Microsoft SQL Server。 - - Microsoft SQL Server 要求使用复杂密码，请参阅[使用复杂密码](https://learn.microsoft.com/zh-cn/sql/relational-databases/security/password-policy?view=sql-server-ver16#password-complexity)。 - 使用环境变量 `ACCEPT_EULA=Y` 启动 Docker 容器代表您同意 Microsoft 的 EULA 条款，另请参阅 [End-User Licensing Agreement](https://go.microsoft.com/fwlink/?linkid=857698)。 - - ```bash - # 启动一个 Microsoft SQL Server 容器并设置密码为 `mqtt_public1` - $ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2022-CU15-ubuntu-22.04 - ``` - -2. 进入 Docker 容器。 - - ```bash - $ docker exec -it sqlserver bash - ``` - -3. 在容器中连接到 Microsoft SQL Server 服务器，需要输入预设的密码。输入密码时字符不会回显。请输入密码后直接键入 `Enter`。 - - ```bash - $ /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P mqtt_public1 -N -C - 1> - ``` - - -::: tip - -Microsoft 提供的 Microsoft SQL Server 容器内已安装 `mssql-tools18`，但可执行文件并不在 `$PATH` 中。因此，在继续操作之前，您需要为 `sqlcmd` 指定可执行文件的路径。对于本例中的 Docker 部署，文件路径应为 `/opt`。 - -关于更多 `mssql-tools18` 的使用，请阅读 [sqlcmd 实用工具](https://learn.microsoft.com/zh-cn/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16)。 - -::: - -至此 Microsoft SQL Server 2022 实例已经完成部署并可以连接。 - -### 创建数据表 - -使用已创建的连接和下面的 SQL 语句在 Microsoft SQL Server 中创建数据表。 - -- 如需用于 MQTT 消息存储，创建数据表 `dbo.t_mqtt_msg`。该表存储每条消息的 MsgID、主题、QoS、Payload 以及发布时间。 - - ```sql - CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - msgid VARCHAR(64) NULL, - topic VARCHAR(100) NULL, - qos tinyint NOT NULL DEFAULT 0, - payload VARCHAR(100) NULL, - arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - -- 如需用于设备上下线状态记录，创建数据表 `dbo.t_mqtt_events`。 - - ```sql - CREATE TABLE dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - clientid VARCHAR(255) NULL, - event_type VARCHAR(255) NULL, - event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - - ### 安装并配置 ODBC 驱动程序 为了能够访问 Microsoft SQL Server 数据库，您需要安装并配置 ODBC 驱动程序。您可以使用 Microsoft 发布的 msodbcsql18 或者 FreeTDS 作为 ODBC 驱动程序。 @@ -145,9 +80,9 @@ EMQX 使用 `odbcinst.ini` 配置中的 DSN Name 来确定驱动动态库的路 ```dockerfile FROM emqx/emqx-enterprise:5.8.1 - + USER root - + RUN apt-get -qq update && apt-get install -yqq curl gpg && \ . /etc/os-release && \ curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg && \ @@ -157,11 +92,12 @@ EMQX 使用 `odbcinst.ini` 配置中的 DSN Name 来确定驱动动态库的路 sed -i 's/ODBC Driver 18 for SQL Server/ms-sql/g' /etc/odbcinst.ini && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* - + USER emqx ``` 2. 使用命令 `docker build -t emqx/emqx-enterprise:5.8.1-msodbc ` 构建镜像。 + 3. 构建完成后可以使用 `docker image ls` 来获取本地的 image 列表，您也可以将镜像上传或保存备用。 ::: tip 注意 @@ -175,6 +111,7 @@ EMQX 使用 `odbcinst.ini` 配置中的 DSN Name 来确定驱动动态库的路 本节介绍了在几种主流发行版上安装配置 FreeTDS 作为 ODBC 驱动程序的方式。在此处给出的示例中，DSN Name 均为 `ms-sql`。 在 MacOS 上安装配置 FreeTDS 作为 ODBC 驱动程序: + ```bash $ brew install unixodbc freetds $ vim /usr/local/etc/odbcinst.ini @@ -187,6 +124,7 @@ FileUsage = 1 ``` 在 Centos 上安装配置 FreeTDS 作为 ODBC 驱动程序: + ```bash $ yum install unixODBC unixODBC-devel freetds freetds-devel perl-DBD-ODBC perl-local-lib $ vim /etc/odbcinst.ini @@ -201,6 +139,7 @@ FileUsage = 1 ``` 在 Ubuntu 上安装配置 FreeTDS 作为 ODBC 驱动程序（以 Ubuntu20.04 为例，其他版本请参考 ODBC 官方文档）: + ```bash $ apt-get install unixodbc unixodbc-dev tdsodbc freetds-bin freetds-common freetds-dev libdbd-odbc-perl liblocal-lib-perl $ vim /etc/odbcinst.ini @@ -212,6 +151,76 @@ Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so FileUsage = 1 ``` +### 安装并连接到 Microsoft SQL Server + +本节描述如何使用 Docker 镜像在 Linux/MacOS 安装启动 Microsoft SQL Server 2019 以及如何使用 `sqlcmd` 连接到 Microsoft SQL Server。关于其他 Microsoft SQL Server 的安装方式，请参阅微软提供的 [Microsoft SQL Server 安装指南](https://learn.microsoft.com/zh-cn/sql/database-engine/install-windows/install-sql-server?view=sql-server-ver16)。 + +1. 通过 Docker 安装并启动 Microsoft SQL Server。 + + Microsoft SQL Server 要求使用复杂密码，请参阅[使用复杂密码](https://learn.microsoft.com/zh-cn/sql/relational-databases/security/password-policy?view=sql-server-ver16#password-complexity)。 + 使用环境变量 `ACCEPT_EULA=Y` 启动 Docker 容器代表您同意 Microsoft 的 EULA 条款，另请参阅 [End-User Licensing Agreement](https://go.microsoft.com/fwlink/?linkid=857698)。 + + ```bash + # 启动一个 Microsoft SQL Server 容器并设置密码为 `mqtt_public1` + $ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2022-CU15-ubuntu-22.04 + ``` + +2. 进入 Docker 容器。 + + ```bash + $ docker exec -it sqlserver bash + ``` + +3. 在容器中连接到 Microsoft SQL Server 服务器，需要输入预设的密码。输入密码时字符不会回显。请输入密码后直接键入 `Enter`。 + + ```bash + $ /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P mqtt_public1 -N -C + 1> + ``` + + +::: tip + +Microsoft 提供的 Microsoft SQL Server 容器内已安装 `mssql-tools18`，但可执行文件并不在 `$PATH` 中。因此，在继续操作之前，您需要为 `sqlcmd` 指定可执行文件的路径。对于本例中的 Docker 部署，文件路径应为 `/opt`。 + +关于更多 `mssql-tools18` 的使用，请阅读 [sqlcmd 实用工具](https://learn.microsoft.com/zh-cn/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16)。 + +::: + +至此 Microsoft SQL Server 2022 实例已经完成部署并可以连接。 + +### 创建数据表 + +使用已创建的连接和下面的 SQL 语句在 Microsoft SQL Server 中创建数据表。 + +::: tip + +由于 ODBC 接口限制，如需要写入 Unicode 字符，如 CJK 字符或 Emoji 等，则需要使用函数转换为二进制格式后插入。在创建表时将需要存储 Unicode 字符的列类型设置为 `NVARCHAR`。 + +::: + +- 如需用于 MQTT 消息存储，创建数据表 `dbo.t_mqtt_msg`。该表存储每条消息的 MsgID、主题、QoS、Payload 以及发布时间。 + + ```sql + CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, + msgid VARCHAR(64) NULL, + topic VARCHAR(100) NULL, + qos tinyint NOT NULL DEFAULT 0, + payload VARCHAR(100) NULL, + arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); + GO + ``` + +- 如需用于设备上下线状态记录，创建数据表 `dbo.t_mqtt_events`。 + + ```sql + CREATE TABLE dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, + clientid VARCHAR(255) NULL, + event_type VARCHAR(255) NULL, + event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); + GO + ``` + ## 创建连接器 在创建 Microsoft SQL Server Sink 之前，您需要创建一个连接器，以便 EMQX 与 Microsoft SQL Server 服务建立连接。以下示例假定您在本地机器上同时运行 EMQX 和 Microsoft SQL Server。如果您在远程运行 Microsoft SQL Server 和 EMQX，请相应地调整设置。 @@ -251,10 +260,24 @@ FileUsage = 1 ::: tip - 如果您初次使用 SQL，可以点击 **SQL 示例**和**启用调试**来学习和测试规则 SQL 的结果。 + 由于 ODBC 接口限制，如需要写入 Unicode 字符，如 CJK 字符或 Emoji 等，则需要使用函数转换为二进制格式后插入。在创建规则时使用内置函数将字符串转换为 UTF-16-little-endian 编码的二进制串。 + ```sql{2} + SELECT + sqlserver_bin2hexstr(str_utf16_le(payload)) as payload + * + FROM + "t/#" + ``` + ::: - + + ::: tip + + 如果您初次使用 SQL，可以点击 **SQL 示例**和**启用调试**来学习和测试规则 SQL 的结果。 + + ::: + 4. 点击右侧的**添加动作**按钮，为规则在被触发的情况下指定一个动作。通过这个动作，EMQX 会将经规则处理的数据发送到 Microsoft SQL Server。 5. 在**动作类型**下拉框中选择 `Microsoft SQL Server`，保持**动作**下拉框为默认的`创建动作`选项，您也可以选择一个之前已经创建好的 Microsoft SQL Server Sink。此示例将创建一个全新的 Sink 并添加到规则中。 @@ -263,22 +286,32 @@ FileUsage = 1 7. 从**连接器**下拉框中选择刚刚创建的 `my_sqlserver`。您也可以通过点击下拉框旁边的按钮创建一个新的连接器。有关配置参数，请参见[创建连接器](#创建连接器)。 -9. 配置 **SQL 模板**。如需实现对指定主题消息的转发，使用如下 SQL 语句完成数据插入。此处为[预处理 SQL](./data-bridges.md#sql-预处理)，字段不应当包含引号，SQL 末尾不要带分号 `;`。 +8. 配置 **SQL 模板**。如需实现对指定主题消息的转发，使用如下 SQL 语句完成数据插入。此处为[预处理 SQL](./data-bridges.md#sql-预处理)，字段不应当包含引号，SQL 末尾不要带分号 `;`。 ```sql insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, ${payload} ) ``` - 如果在模板中使用未定义的占位符变量，您可以切换**未定义变量作为 NULL** 开关（位于 **SQL 模板** 上方）来定义规则引擎的行为： + ::: tip - - **关闭**（默认）：规则引擎可以将字符串 `undefined` 插入数据库。 + 由于 ODBC 接口限制，如需要写入 Unicode 字符，如 CJK 字符或 Emoji 等，则需要使用函数转换为二进制格式后插入。在 SQL 模板中使用 `CONVERT` 函数，由 Microsoft SQL Server 将对应的二进制数据转为字符串。 + ```sql + insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, CONVERT(NVARCHAR(100), ${payload}) ) + ``` + + ::: + + 如果在模板中使用未定义的占位符变量，您可以切换**未定义变量作为 NULL** 开关（位于 **SQL 模板** 上方）来定义规则引擎的行为： + + - **关闭**（默认）：规则引擎可以将字符串 `undefined` 插入数据库。 + - **启用**：允许规则引擎在变量未定义时将 `NULL` 插入数据库。 - + ::: tip - - 如果您初次使用 SQL，可以点击 **SQL 示例** 和**启用调试**来学习和测试规则 SQL 的结果。 - + + 如果可能，应该始终启用此选项；禁用该选项仅用于确保向后兼容性。 + ::: 10. 高级配置（可选），根据情况配置同步/异步模式，队列与批量等参数，详细请参考 [Sink 的特性](./data-bridges.md#sink-的特性)。 @@ -293,36 +326,6 @@ FileUsage = 1 您还可以点击 **集成** -> **Flow 设计器**可以查看拓扑，通过拓扑可以直观的看到，主题 `t/#` 下的消息在经过规则 `my_rule` 解析后被发送到 Microsoft SQL Server 中。 -由于 ODBC 接口限制，需要写入 Unicode 字符，如 CJK 字符或 Emoji 等，需要使用函数转换为二进制格式后插入。 - -- 在创建表时将需要存储 Unicode 字符的列类型设置为 `NVARCHAR`。 - - ```sql{5} - CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL, - msgid VARCHAR(64) NULL, - topic VARCHAR(100) NULL, - qos tinyint NOT NULL DEFAULT 0, - payload NVARCHAR(100) NULL, - arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP); - GO - ``` - -- 在创建规则时使用内置函数将字符串转换为 UTF-16-little-endian 编码的二进制串。 - - ```sql{2} - SELECT - sqlserver_bin2hexstr(str_utf16_le(payload)) as payload - * - FROM - "t/#" - ``` - -- 在 SQL 模板中使用 `CONVERT` 函数，由 Microsoft SQL Server 将对应的二进制数据转为字符串。 - - ```sql - insert into dbo.t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, CONVERT(NVARCHAR(100), ${payload}) ) - ``` - ## 创建事件记录 Sink 规则 本节展示如何创建用于记录客户端上/下线状态的规则，并通过配置的 Sink 将记录写入到 Microsoft SQL Server 的数据表 `dbo.t_mqtt_events` 中。 diff --git a/zh_CN/data-integration/data-bridge-tdengine.md b/zh_CN/data-integration/data-bridge-tdengine.md index 2ff908ddd..0fc005551 100644 --- a/zh_CN/data-integration/data-bridge-tdengine.md +++ b/zh_CN/data-integration/data-bridge-tdengine.md @@ -165,7 +165,7 @@ CREATE TABLE emqx_client_events ( ::: tip - 如果您初次使用 SQL，可以点击 **SQL 示例** 和**启用调试**来学习和测试规则 SQL 的结果。 + 如果可能，应该始终启用此选项；禁用该选项仅用于确保向后兼容性。 :::