Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #202] refine the supporting document for cluster deploy #230

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Distributed Consistency Protocol for EventMesh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Distributed Consistency Protocol is applicable only to protocols akin to KRaft/JRaft and the like. Reverse Proxy does not fall under the purview of the Distributed Consistency Protocol.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix the title into 'Distributed Consistency Protocol or Reverse proxy'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can split them into two documents, or use a more general title, such as "Cluster Design."

## **Background Knowledge and Current State:**
* Kafka utilizes KRaft for insync replication of message from main broker to following brokers.
Comment on lines +2 to +3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a gentle correction, it should be "In-Sync" replication, not "insync" replication. The meaning of these two words in English is completely opposite.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix the typo

* In order to prevent the loss of messages, Kafka has a leadership election mechanism
* EventMesh does not fully implement JRaft protocol.
* A partially functional JRaft PR has been merged, but it requires a fixed IP cluster list defined in the `eventmesh.properties` file (immutable after initiation). Once a node fails the whole cluster fails. This limits elastic scaling of EventMesh cluster nodes during runtime.

## **Using a Distributed Consistency Protocol:**
### **Solutions: Implement fully functional JRaft Protocol Interface**
**Result**
* Enables dynamic scaling and dynamic cluster discovery.
* If a main node registers in the cluster it will be the main node and start to sync the message among followers.
* Leader election occurs if the leading node fails.
Recovered leading node will become a follower.
* Node failure will no longer results in service failure.
* Once implement the Jraft interface no need to deploy Nacos.

## **Using the Reverse Proxy:**
**Current State:** Nginx can be the reverse proxy of EventMesh. In the SDK we set the Nginx IP address, Nginx will transfer the subscription address to a EventMesh runtime.
Comment on lines +18 to +19
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section requires additional information on nacos.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need your advisory in Nacos deployment.

### **Solutions: Add Gateway Proxy Layer:** A next generation of functional solution for the new architecture. Like Nginx.
**Current Flaws**
Comment on lines +20 to +21
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The title is quite lengthy and also, Nginx is not exactly the next-gen solution for the new architecture, but rather a compromise implementation for load balancing as of now.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix the title into 'A compromise implementation for load balancing for now'

* Nginx is not in EventMesh eco-system, It can't percept the EventMesh runtime state like online or offline.
* Once the runtime node is down the client have nothing to do but connect to another runtime node, at that time the runtime just sync the subscription infomation from the Nacos.(Currently we don't have the toggle stream mechanism and we don't have further plan of solution).
* Currently failover mechanism is not functional.
5 changes: 5 additions & 0 deletions docs/distributed-cluster-deployment-guide/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"position": 11,
"label": "Distributed Cluster Deployment",
"collapsed": false
}
4 changes: 4 additions & 0 deletions i18n/zh/docusaurus-plugin-content-docs/current.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,9 @@
"sidebar.tutorialSidebar.category.Connect": {
"message": "连接器",
"description": "The label for category Connect in sidebar tutorialSidebar"
},
"sidebar.tutorialSidebar.category.Distributed Cluster Deployment": {
"message": "分布式集群部署",
"description": "The label for category Distributed Cluster Deployment in sidebar tutorialSidebar"
}
}
12 changes: 12 additions & 0 deletions sidebars/docs.js
Original file line number Diff line number Diff line change
Expand Up @@ -103,5 +103,17 @@ module.exports = {
},
],
},
{
type: 'category',
label: 'Distributed Cluster Deployment',
collapsible: true,
collapsed: false,
items: [
{
type: 'autogenerated',
dirName: 'distributed-cluster-deployment-guide',
},
],
}
Comment on lines +106 to +117
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to place this chapter under the Design Document category.

Once you have written the quick start guide for cluster deployment, this document will need to be placed under the Installation and Deployment category.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix it in the 'Design Document' category

],
};
Loading