-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiple webhook brokers #56
Comments
At this moment, we only support one pulsar-beam broker. So the broker just reads webhook configuration and pushes messages. Multiple broker is required for capacity and large volume of messages. I will have some algorithm to balance the load among brokers. Most likely I will hash the topic names and spread them across multiple brokers. So one broker only reads a range of topics and a topic would not be read by multiple brokers. Actually I do have an issue open to track this #42. |
This is more of a question. If I have multiple webhook configurations and multiple instances of
pulsar-beam
brokers are running, how does the assignment of topics happens? Or, do all brokers try to read from all topics?The text was updated successfully, but these errors were encountered: