Skip to content

Commit

Permalink
lightning: delete content about tpcc (#14531)
Browse files Browse the repository at this point in the history
  • Loading branch information
Frank945946 authored Aug 18, 2023
1 parent 7004e72 commit 1fc6274
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 23 deletions.
25 changes: 2 additions & 23 deletions tidb-lightning/tidb-lightning-physical-import-mode-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ summary: Learn how to use the physical import mode in TiDB Lightning.

This document introduces how to use the [physical import mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) in TiDB Lightning, including writing the configuration file, tuning performance, and configuring disk quota.

There are limitations on the physical import mode. Before using the physical import mode, make sure to read [Limitations](/tidb-lightning/tidb-lightning-physical-import-mode.md#limitations).

## Configure and use the physical import mode

You can use the following configuration file to execute data import using the physical import mode:
Expand Down Expand Up @@ -200,31 +202,8 @@ store-write-bwlimit = "128MiB"
[tidb]
# Use smaller concurrency to reduce the impact of Checksum and Analyze on the transaction latency.
distsql-scan-concurrency = 3

[cron]
# Prevent TiKV from switching to import mode.
switch-mode = '0'
```

You can measure the impact of data import on TPCC results by simulating the online application using TPCC and importing data into a TiDB cluster using TiDB Lightning. The test result is as follows:

| Concurrency | TPM | P99 | P90 | AVG |
| ----- | --- | --- | --- | --- |
| 1 | 20%~30% | 60%~80% | 30%~50% | 30%~40% |
| 8 | 15%~25% | 70%~80% | 35%~45% | 20%~35% |
| 16 | 20%~25% | 55%~85% | 35%~40% | 20%~30% |
| 64 | No significant impact |
| 256 | No significant impact |

The percentage in the preceding table indicates the impact of data import on TPCC results.

* For the TPM column, the number indicates the percentage of TPM decrease.
* For the P99, P90, and AVG columns, the number indicates the percentage of latency increase.

The test results show that the smaller the concurrency, the larger the impact of data import on TPCC results. When the concurrency is 64 or more, the impact of data import on TPCC results is negligible.

Therefore, if your TiDB cluster has a latency-sensitive application and a low concurrency, it is strongly recommended **not** to use TiDB Lightning to import data into the cluster. This will cause a significant impact on the online application.

## Performance tuning

**The most direct and effective ways to improve import performance of the physical import mode are as follows:**
Expand Down
1 change: 1 addition & 0 deletions tidb-lightning/tidb-lightning-physical-import-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ It is recommended that you allocate CPU more than 32 cores and memory greater th
### Limitations

- Do not use the physical import mode to directly import data to TiDB clusters in production. It has severe performance implications. If you need to do so, refer to [Pause scheduling on the table level](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#scope-of-pausing-scheduling-during-import).
- If your TiDB cluster has a latency-sensitive application and a low concurrency, it is strongly recommended that you **do not** use the physical import mode to import data into the cluster. This mode might have significant impact on the online application.
- Do not use multiple TiDB Lightning instances to import data to the same TiDB cluster by default. Use [Parallel Import](/tidb-lightning/tidb-lightning-distributed-import.md) instead.
- When you use multiple TiDB Lightning to import data to the same target cluster, do not mix the import modes. That is, do not use the physical import mode and the logical import mode at the same time.
- During the process of importing data, do not perform DDL and DML operations in the target table. Otherwise the import will fail or the data will be inconsistent. At the same time, it is not recommended to perform read operations, because the data you read might be inconsistent. You can perform read and write operations after the import operation is completed.
Expand Down

0 comments on commit 1fc6274

Please sign in to comment.