From 45c0a2f7c17f2813ee0ba36a6cd0a6fa7cf89e59 Mon Sep 17 00:00:00 2001 From: Maksim Davydov Date: Sat, 26 Oct 2024 14:57:38 +0300 Subject: [PATCH] IGNITE-19528 * removed redundant options * updated doc --- .../change-data-capture-extensions.adoc | 52 ++++++++----------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc index 34b1573c201ff..586b3ab6680e1 100644 --- a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc +++ b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc @@ -491,7 +491,7 @@ The former strategy implies that only one cluster would be activly used for data The latter approach allows users to use both cluster simultaneously, meanwhile the CDC clients transfer the updates between them. For that strategy to work, the user should configure conflict resolver. -* To make strategies work you have two independent tools to configure: +* To make things work you have two independent tools to configure: ** Thin/Thick clients for connection to destination clusters - responsible for 'put' opertaions on the destination cluster. @@ -509,7 +509,7 @@ The script will use predefined configuration '.xml' files from '../examples/conf NOTE: Use `--help` to explore the manager capabilities -NOTE: Please, enable `ignite-rest-http` and `ignite-json` to be able to run `cdc-start-up.sh` with `--check`. +NOTE: Please, enable `ignite-rest-http` and `ignite-json` to be able to run `cdc-start-up.sh` with `--check-cdc`. Examples for reference: @@ -538,9 +538,9 @@ NOTE: All properties files are preconfigured to work out of the box. ** There are 5 options you can specify CDC client mode from. Take a look at `--help` command output to learn about them. -** You can optionaly activate both clusters at CDC client start-up with `--activate-cluster`. You need this for steady data replication. You can also use `control.sh` capabilities to activate clusters manually. +** You can optionaly activate both clusters at CDC client start-up with `--activate-cluster`. You need this for data persistance at source and destination clusters. You can also use `control.sh` capabilities to activate clusters manually. -NOTE: Start both clusters (as in previous example with Ignite nodes) before starting CDC client +NOTE: Start both clusters (as in previous example with Ignite nodes) before starting CDC client. Here is an example on how to start Active-Passive inter-cluster communication with 2 separate nodes and one thin CDC client for Ignite-to-Ignite replication from cluster 1 to cluster 2 (Run the commands independently): ``` @@ -549,6 +549,16 @@ Here is an example on how to start Active-Passive inter-cluster communication wi ./cdc-start-up.sh --cdc-client --ignite-to-ignite-thin --activate-cluster ../examples/config/cdc-start-up/cluster-1 ``` +NOTE: Make sure clusters fully started up before starting CDC client. + +Here is an example on how to start Active-Active inter-cluster communication with 2 separate nodes and 2 CDC clients (thick) for Ignite-to-Ignite replication (Run the commands independently): +``` +./cdc-start-up.sh --ignite ../examples/config/cdc-start-up/cluster-1 +./cdc-start-up.sh --ignite ../examples/config/cdc-start-up/cluster-2 +./cdc-start-up.sh --cdc-client --ignite-to-ignite --activate-cluster ../examples/config/cdc-start-up/cluster-1 +./cdc-start-up.sh --cdc-client --ignite-to-ignite ../examples/config/cdc-start-up/cluster-2 +``` + NOTE: To start CDC with Kafka you need to start topics beforehand. We use the following topics naming for our examples: @@ -567,26 +577,12 @@ We use the following topics naming for our examples: ./kafka-topics.sh --create --partitions 1 --replication-factor 1 --topic metadata_from_dc2 --bootstrap-server localhost:9092 ``` -* Start a whole replication system with specified strategy and properties for both clusters: - -** Use `--active-active` for Active-Active replication and `--active-passive` for Active-Passive replication. - -** You should provide properties files paths for both cluster nodes in the end of the command - -** By default, this script uses Ignite clients (thick) for replication. You can optionally use `--with-kafka` to transfer data through Kafka, and `--thin` to use thin clients for connections to the destination clusters. - -NOTE: There is no need to preactivate clusters, as the script will do it for you. - -To repeat the replication example from the previous point, you can use the following: - -``` -./cdc-start-up.sh --active-passive --thin ../examples/config/cdc-start-up/cluster-1 ../examples/config/cdc-start-up/cluster-2 -``` - -Active-Active with thin clients and Kafka as data transport: - +Here is an example on how to start Active-Passive inter-cluster communication with 2 separate nodes and 2 CDC clients for replication with Kafka from cluster 1 to cluster 2 (Run the commands independently): ``` -./cdc-start-up.sh --active-active --with-kafka --thin ../examples/config/cdc-start-up/cluster-1 ../examples/config/cdc-start-up/cluster-2 +./cdc-start-up.sh --ignite ../examples/config/cdc-start-up/cluster-1 +./cdc-start-up.sh --ignite ../examples/config/cdc-start-up/cluster-2 +./cdc-start-up.sh --cdc-client --ignite-to-kafka --activate-cluster ../examples/config/cdc-start-up/cluster-1 +./cdc-start-up.sh --cdc-client --kafka-to-ignite-thin ../examples/config/cdc-start-up/cluster-2 ``` * You can check CDC replication with `--check-cdc`. Use it in parallel with Active-Passive/Active-Active replication. To start CDC check for proposed entry: @@ -596,13 +592,7 @@ Active-Active with thin clients and Kafka as data transport: The command basically puts the entry to the chosen cluster and shows the difference between the two clusters until the data fully transfered. -NOTE: To use it with Active-Passive, push entries only in the Active part of the CDC inter-cluster system. See the example below - -``` -./cdc-start-up.sh --active-passive ../examples/config/cdc-start-up/cluster-1 ../examples/config/cdc-start-up/cluster-2 -``` - -`cluster-1` is an active part since its properties' directory goes first in the command. In this case there is no reason to push data to the second cluster during the check step, as the data is not replicated from the second cluster. +NOTE: To use it with Active-Passive, push entries only in the Active part of the CDC inter-cluster system. NOTE: Try to play with version value to see how the conflict resolver works. We propose the following sequence of operations with Active-Active: @@ -612,4 +602,4 @@ NOTE: Try to play with version value to see how the conflict resolver works. We ./cdc-start-up.sh --check-cdc --key 11006 --value 3 --version 3 --cluster 1 ./cdc-start-up.sh --check-cdc --key 11006 --value 2 --version 2 --cluster 2 ``` -This sequence simulates the case when the first cluster receives outdated value from the second. In our case the data will not be replicated in the last command and the check would fail after 20 tries. \ No newline at end of file +This sequence simulates the case when the first cluster receives outdated value from the second. In our case the data will not be replicated in the last command and the check would fail after 20 tries.