Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add upgrade note for using InfluxDB protocol #963

Merged
merged 4 commits into from
May 20, 2024
Merged

docs: add upgrade note for using InfluxDB protocol #963

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
59c7d27
docs: update upgrade doc
WenyXu May 17, 2024
17adbb4
docs: add upgrade note for using InfluxDB protocol
WenyXu May 17, 2024
1bcca74
chore: apply suggestions from CR
WenyXu May 20, 2024
e610527
chore: add `Known Issues`
WenyXu May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 46 additions & 8 deletions docs/nightly/en/user-guide/upgrade.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@ For a complete upgrade, you will need to execute this tools twice with each targ

Here is a complete example for upgrading from `v0.7.x` to `v0.8.0`.


### Export `CREATE TABLE`

Assuming the HTTP service port of the old database is `4000`.

```shell
greptime cli export --addr '127.0.0.1:4000' --output-dir /tmp/greptimedb-export --target create-table
```
Expand Down Expand Up @@ -82,7 +83,7 @@ CREATE TABLE foo (
TIME INDEX (ts),
PRIMARY KEY(host)
) ENGINE=mito
WITH(
WITH( # Delete
regions=1
);
```
Expand Down Expand Up @@ -130,7 +131,7 @@ CREATE TABLE IF NOT EXISTS "phy" (
"__tsid" BIGINT UNSIGNED NOT NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("__table_id", "__tsid", "host", "job")
PRIMARY KEY ("__table_id", "__tsid", "host", "job") # Modify this line
)
ENGINE=metric
WITH(
Expand Down Expand Up @@ -179,14 +180,43 @@ CREATE TABLE IF NOT EXISTS "phy" (
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job")
TIME INDEX ("ts")
TIME INDEX ("ts") # Add this line
)
ENGINE=metric
WITH(
physical_metric_table = ''
);
```

#### Update the create table statement for tables written using the InfluxDB protocol

Related [issue](https://github.com/GreptimeTeam/greptimedb/pull/3794)

Before:
```sql
CREATE TABLE IF NOT EXISTS "phy" (
"ts" TIMESTAMP(6) NOT NULL, # Modify to TIMESTAMP(9)
"val" DOUBLE NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job"),
TIME INDEX ("ts")
)
ENGINE=mito;
```

After:
```sql
CREATE TABLE IF NOT EXISTS "phy" (
"ts" TIMESTAMP(9) NOT NULL,
"val" DOUBLE NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job"),
TIME INDEX ("ts")
)
ENGINE=mito;
```

### Export table data

Expand Down Expand Up @@ -231,6 +261,14 @@ And then import the data
psql -h 127.0.0.1 -p 4003 -d public -f /tmp/greptime-public_copy_from.sql
```

### Known Issues

#### The upgrade tool will still export physical table data from v0.7.0
When importing v0.7.0 data into v0.8.0, the database may encounter the following error. This error can be safely ignored as it does not affect data integrity.
```
psql:/tmp/greptimedb-export/greptime-public_copy_from.sql:2: ERROR: Alter request to physical region is forbidden
```

### Clean up

At this step all the data is migrated. You can check the data in the new cluster.
Expand All @@ -241,10 +279,10 @@ After confirming that the data is correct, you can clean up the old cluster and

This section gives a recommended overall process for upgrading GreptimeDB smoothly. You can skip this section if your environment can go offline on the upgrade progress.

1. Create a brand new v0.4 cluster
2. Export and import `create-table`
3. Switch workload to the new cluster
4. Export and import table data
1. Create a brand new v0.8.0 cluster.
2. Use the v0.8.0 CLI tool to export and import `create-table`.
3. Switch the workload to the new cluster.
4. Use the v0.8.0 CLI tool to export and import `table-data`.

Caveats
- Changes to table structure between step 2 and 3 will be lost
Expand Down
91 changes: 64 additions & 27 deletions docs/nightly/zh/user-guide/upgrade.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,11 @@ OPTIONS:

对于完整的升级，您需要使用每个目标选项两次执行此工具。

## 示例
## 从 0.7.x 升级

这一节将演示如何从 `v0.7.x` 升级到 `v0.8.0`。

在下面的文本中，我们假设您的 Frontend 的 HTTP 端口为 `127.0.0.1:4000`。输出目录是 `/tmp/greptimedb-export`。
在下面的文本中，我们假设您的数据库的 HTTP 端口为 `127.0.0.1:4000`。

### 导出 `CREATE TABLE`

Expand All @@ -67,25 +67,6 @@ greptime cli export --addr '127.0.0.1:4000' --output-dir /tmp/greptimedb-export
└── greptime-public.sql
```

### 导出表数据

```shell
greptime cli export --addr '127.0.0.1:4000' --database greptime-public --output-dir /tmp/greptimedb-export --target table-data
```

日志输出与上面类似。输出目录的结构如下

```plaintext
/tmp/greptimedb-export
├── greptime-public
│   ├── up.parquet
│   └── other-tables.parquet
├── greptime-public_copy_from.sql
└── greptime-public.sql
```

新的内容是 `greptime-public_copy_from.sql` 和 `greptime-public`。前者包含每个表的 `COPY FROM` 语句。后者包含每个表的数据。

### 处理 Breaking Changes
:::warning 注意
从版本 0.7.x 升级时存在已知的 Breaking Changes。您需要手动编辑导出的 SQL 文件（即 /tmp/greptimedb-export/greptime-public.sql）
Expand All @@ -101,7 +82,7 @@ CREATE TABLE foo (
TIME INDEX (ts),
PRIMARY KEY(host)
) ENGINE=mito
WITH(
WITH( # 删除
regions=1
);
```
Expand Down Expand Up @@ -149,7 +130,7 @@ CREATE TABLE IF NOT EXISTS "phy" (
"__tsid" BIGINT UNSIGNED NOT NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("__table_id", "__tsid", "host", "job")
PRIMARY KEY ("__table_id", "__tsid", "host", "job") # 修改此处
)
ENGINE=metric
WITH(
Expand Down Expand Up @@ -199,14 +180,62 @@ CREATE TABLE IF NOT EXISTS "phy" (
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job")
TIME INDEX ("ts")
TIME INDEX ("ts") # 添加在此处
)
ENGINE=metric
WITH(
physical_metric_table = ''
);
```

#### 为 InfluxDB 协议的表更新建表语句

相关 [issue](https://github.com/GreptimeTeam/greptimedb/pull/3794)

修改前:
```sql
CREATE TABLE IF NOT EXISTS "phy" (
"ts" TIMESTAMP(6) NOT NULL, # 修改此处
"val" DOUBLE NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job"),
TIME INDEX ("ts")
)
ENGINE=mito;
```

修改后:
```sql
CREATE TABLE IF NOT EXISTS "phy" (
"ts" TIMESTAMP(9) NOT NULL,
"val" DOUBLE NULL,
"host" STRING NULL,
"job" STRING NULL,
PRIMARY KEY ("host", "job"),
TIME INDEX ("ts")
)
ENGINE=mito;
```

### 导出表数据

```shell
greptime cli export --addr '127.0.0.1:4000' --database greptime-public --output-dir /tmp/greptimedb-export --target table-data
```

日志输出与上面类似。输出目录的结构如下

```plaintext
/tmp/greptimedb-export
├── greptime-public
│   ├── up.parquet
│   └── other-tables.parquet
├── greptime-public_copy_from.sql
└── greptime-public.sql
```

新的内容是 `greptime-public_copy_from.sql` 和 `greptime-public`。前者包含每个表的 `COPY FROM` 语句。后者包含每个表的数据。

### 导入表结构和数据

Expand All @@ -232,6 +261,14 @@ psql -h 127.0.0.1 -p 4003 -d public -f /tmp/greptime-public.sql
psql -h 127.0.0.1 -p 4003 -d public -f /tmp/greptime-public_copy_from.sql
```

### 已知问题

#### 升级工具依然会导出 0.7.0 中的物理表数据
在将 v0.7.0 的数据导入至 v0.8.0 时，数据库可能会出现以下错误。可直接忽略，该错误并不会影响数据的完整性。
```
psql:/tmp/greptimedb-export/greptime-public_copy_from.sql:2: ERROR: Alter request to physical region is forbidden
```

### 清理

到这一步，所有的数据都已经迁移完毕。您可以在新集群中检查数据。
Expand All @@ -242,10 +279,10 @@ psql -h 127.0.0.1 -p 4003 -d public -f /tmp/greptime-public_copy_from.sql

该部分给出了一个推荐的整体流程，以便平滑升级 GreptimeDB。如果您的环境可以在升级过程中离线，可以跳过此部分。

1. 创建一个全新的 v0.4 集群
2. 导出并导入 `create-table`
1. 创建一个全新的 v0.8.0 集群
2. 使用 v0.8.0 版本的 cli 工具导出并导入 `create-table`
3. 将工作负载切换到新集群
4. 导出并导入 `table-data`
4. 使用 v0.8.0 版本的 cli 工具导出并导入 `table-data`

注意

Expand Down