From 1fc627472a5e8c6b068179d9679034b3b44dee6d Mon Sep 17 00:00:00 2001 From: Frank945946 <108602632+Frank945946@users.noreply.github.com> Date: Fri, 18 Aug 2023 17:35:02 +0800 Subject: [PATCH] lightning: delete content about tpcc (#14531) --- ...db-lightning-physical-import-mode-usage.md | 25 ++----------------- .../tidb-lightning-physical-import-mode.md | 1 + 2 files changed, 3 insertions(+), 23 deletions(-) diff --git a/tidb-lightning/tidb-lightning-physical-import-mode-usage.md b/tidb-lightning/tidb-lightning-physical-import-mode-usage.md index 5c86465dcc7e0..8f57dd97aa0e0 100644 --- a/tidb-lightning/tidb-lightning-physical-import-mode-usage.md +++ b/tidb-lightning/tidb-lightning-physical-import-mode-usage.md @@ -7,6 +7,8 @@ summary: Learn how to use the physical import mode in TiDB Lightning. This document introduces how to use the [physical import mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) in TiDB Lightning, including writing the configuration file, tuning performance, and configuring disk quota. +There are limitations on the physical import mode. Before using the physical import mode, make sure to read [Limitations](/tidb-lightning/tidb-lightning-physical-import-mode.md#limitations). + ## Configure and use the physical import mode You can use the following configuration file to execute data import using the physical import mode: @@ -200,31 +202,8 @@ store-write-bwlimit = "128MiB" [tidb] # Use smaller concurrency to reduce the impact of Checksum and Analyze on the transaction latency. distsql-scan-concurrency = 3 - -[cron] -# Prevent TiKV from switching to import mode. -switch-mode = '0' ``` -You can measure the impact of data import on TPCC results by simulating the online application using TPCC and importing data into a TiDB cluster using TiDB Lightning. The test result is as follows: - -| Concurrency | TPM | P99 | P90 | AVG | -| ----- | --- | --- | --- | --- | -| 1 | 20%~30% | 60%~80% | 30%~50% | 30%~40% | -| 8 | 15%~25% | 70%~80% | 35%~45% | 20%~35% | -| 16 | 20%~25% | 55%~85% | 35%~40% | 20%~30% | -| 64 | No significant impact | -| 256 | No significant impact | - -The percentage in the preceding table indicates the impact of data import on TPCC results. - -* For the TPM column, the number indicates the percentage of TPM decrease. -* For the P99, P90, and AVG columns, the number indicates the percentage of latency increase. - -The test results show that the smaller the concurrency, the larger the impact of data import on TPCC results. When the concurrency is 64 or more, the impact of data import on TPCC results is negligible. - -Therefore, if your TiDB cluster has a latency-sensitive application and a low concurrency, it is strongly recommended **not** to use TiDB Lightning to import data into the cluster. This will cause a significant impact on the online application. - ## Performance tuning **The most direct and effective ways to improve import performance of the physical import mode are as follows:** diff --git a/tidb-lightning/tidb-lightning-physical-import-mode.md b/tidb-lightning/tidb-lightning-physical-import-mode.md index 30ad504d6a9a3..039a055407338 100644 --- a/tidb-lightning/tidb-lightning-physical-import-mode.md +++ b/tidb-lightning/tidb-lightning-physical-import-mode.md @@ -74,6 +74,7 @@ It is recommended that you allocate CPU more than 32 cores and memory greater th ### Limitations - Do not use the physical import mode to directly import data to TiDB clusters in production. It has severe performance implications. If you need to do so, refer to [Pause scheduling on the table level](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#scope-of-pausing-scheduling-during-import). +- If your TiDB cluster has a latency-sensitive application and a low concurrency, it is strongly recommended that you **do not** use the physical import mode to import data into the cluster. This mode might have significant impact on the online application. - Do not use multiple TiDB Lightning instances to import data to the same TiDB cluster by default. Use [Parallel Import](/tidb-lightning/tidb-lightning-distributed-import.md) instead. - When you use multiple TiDB Lightning to import data to the same target cluster, do not mix the import modes. That is, do not use the physical import mode and the logical import mode at the same time. - During the process of importing data, do not perform DDL and DML operations in the target table. Otherwise the import will fail or the data will be inconsistent. At the same time, it is not recommended to perform read operations, because the data you read might be inconsistent. You can perform read and write operations after the import operation is completed.