-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ISSUE #202] refine the supporting document for cluster deploy #230
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Distributed Consistency Protocol for EventMesh | ||
## **Background Knowledge and Current State:** | ||
* Kafka utilizes KRaft for insync replication of message from main broker to following brokers. | ||
Comment on lines
+2
to
+3
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just a gentle correction, it should be "In-Sync" replication, not "insync" replication. The meaning of these two words in English is completely opposite. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will fix the typo |
||
* In order to prevent the loss of messages, Kafka has a leadership election mechanism | ||
* EventMesh does not fully implement JRaft protocol. | ||
* A partially functional JRaft PR has been merged, but it requires a fixed IP cluster list defined in the `eventmesh.properties` file (immutable after initiation). Once a node fails the whole cluster fails. This limits elastic scaling of EventMesh cluster nodes during runtime. | ||
|
||
## **Using a Distributed Consistency Protocol:** | ||
### **Solutions: Implement fully functional JRaft Protocol Interface** | ||
**Result** | ||
* Enables dynamic scaling and dynamic cluster discovery. | ||
* If a main node registers in the cluster it will be the main node and start to sync the message among followers. | ||
* Leader election occurs if the leading node fails. | ||
Recovered leading node will become a follower. | ||
* Node failure will no longer results in service failure. | ||
* Once implement the Jraft interface no need to deploy Nacos. | ||
|
||
## **Using the Reverse Proxy:** | ||
**Current State:** Nginx can be the reverse proxy of EventMesh. In the SDK we set the Nginx IP address, Nginx will transfer the subscription address to a EventMesh runtime. | ||
Comment on lines
+18
to
+19
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This section requires additional information on There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I need your advisory in Nacos deployment. |
||
### **Solutions: Add Gateway Proxy Layer:** A next generation of functional solution for the new architecture. Like Nginx. | ||
**Current Flaws** | ||
Comment on lines
+20
to
+21
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The title is quite lengthy and also, Nginx is not exactly the next-gen solution for the new architecture, but rather a compromise implementation for load balancing as of now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will fix the title into 'A compromise implementation for load balancing for now' |
||
* Nginx is not in EventMesh eco-system, It can't percept the EventMesh runtime state like online or offline. | ||
* Once the runtime node is down the client have nothing to do but connect to another runtime node, at that time the runtime just sync the subscription infomation from the Nacos.(Currently we don't have the toggle stream mechanism and we don't have further plan of solution). | ||
* Currently failover mechanism is not functional. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"position": 11, | ||
"label": "Distributed Cluster Deployment", | ||
"collapsed": false | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -103,5 +103,17 @@ module.exports = { | |
}, | ||
], | ||
}, | ||
{ | ||
type: 'category', | ||
label: 'Distributed Cluster Deployment', | ||
collapsible: true, | ||
collapsed: false, | ||
items: [ | ||
{ | ||
type: 'autogenerated', | ||
dirName: 'distributed-cluster-deployment-guide', | ||
}, | ||
], | ||
} | ||
Comment on lines
+106
to
+117
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be better to place this chapter under the Design Document category. Once you have written the quick start guide for cluster deployment, this document will need to be placed under the Installation and Deployment category. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will fix it in the 'Design Document' category |
||
], | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Distributed Consistency Protocol is applicable only to protocols akin to KRaft/JRaft and the like. Reverse Proxy does not fall under the purview of the Distributed Consistency Protocol.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix the title into 'Distributed Consistency Protocol or Reverse proxy'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can split them into two documents, or use a more general title, such as "Cluster Design."