Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement periodic background tasks to check room consistency #208

Open
tcpipuk opened this issue Mar 9, 2024 · 0 comments
Open

Implement periodic background tasks to check room consistency #208

tcpipuk opened this issue Mar 9, 2024 · 0 comments

Comments

@tcpipuk
Copy link
Contributor

tcpipuk commented Mar 9, 2024

Apologies if some of this already exists, but I've spent some time investigating the source and can't see it.

Synapse has a number of background tasks that do things like walk back through the room history checking for relations with missing events to try collecting them from other servers in the room.

Conduwuit shouldn't just copy exactly what Synapse does, but it would help for room consistency that we don't just wait for new relations to come in to check for old missing events, and instead proactively walk back through the room on a periodic basis and backfill missing events.

My personal recommendation is that, while running through the server list trying to find the missing event, it would store a list of the servers that didn't have the event, then after finding the event, it uses the /send endpoint on those servers to send that event to the servers that didn't have it.

This would not only help improve the stability of the room, but also help fix the history on the oldest servers so new people joining the room won't immediately inherit an incomplete room.

I've checked in #matrix-spec and (based on advice from Patrick and Tulir) not only is this behaviour allowed in the spec - it's likely to be encouraged in the future to help backfill events into P2P servers.

Given the serious multi-threaded potential of Rust, this seems like something it could potentially do really effectively, but I'm not sure how it'd be best to implement... maybe tasks that simply run tokio::task::yield_now().await frequently to yield Tokio more often, or a second Tokio runtime (e.g. have a configurable number of threads for the "background" workers, defaulting to 25% of the cores) to stop them hogging the live resources?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants