You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Apologies if some of this already exists, but I've spent some time investigating the source and can't see it.
Synapse has a number of background tasks that do things like walk back through the room history checking for relations with missing events to try collecting them from other servers in the room.
Conduwuit shouldn't just copy exactly what Synapse does, but it would help for room consistency that we don't just wait for new relations to come in to check for old missing events, and instead proactively walk back through the room on a periodic basis and backfill missing events.
My personal recommendation is that, while running through the server list trying to find the missing event, it would store a list of the servers that didn't have the event, then after finding the event, it uses the /send endpoint on those servers to send that event to the servers that didn't have it.
This would not only help improve the stability of the room, but also help fix the history on the oldest servers so new people joining the room won't immediately inherit an incomplete room.
I've checked in #matrix-spec and (based on advice from Patrick and Tulir) not only is this behaviour allowed in the spec - it's likely to be encouraged in the future to help backfill events into P2P servers.
Given the serious multi-threaded potential of Rust, this seems like something it could potentially do really effectively, but I'm not sure how it'd be best to implement... maybe tasks that simply run tokio::task::yield_now().await frequently to yield Tokio more often, or a second Tokio runtime (e.g. have a configurable number of threads for the "background" workers, defaulting to 25% of the cores) to stop them hogging the live resources?
The text was updated successfully, but these errors were encountered:
Apologies if some of this already exists, but I've spent some time investigating the source and can't see it.
Synapse has a number of background tasks that do things like walk back through the room history checking for relations with missing events to try collecting them from other servers in the room.
Conduwuit shouldn't just copy exactly what Synapse does, but it would help for room consistency that we don't just wait for new relations to come in to check for old missing events, and instead proactively walk back through the room on a periodic basis and backfill missing events.
My personal recommendation is that, while running through the server list trying to find the missing event, it would store a list of the servers that didn't have the event, then after finding the event, it uses the
/send
endpoint on those servers to send that event to the servers that didn't have it.This would not only help improve the stability of the room, but also help fix the history on the oldest servers so new people joining the room won't immediately inherit an incomplete room.
I've checked in #matrix-spec and (based on advice from Patrick and Tulir) not only is this behaviour allowed in the spec - it's likely to be encouraged in the future to help backfill events into P2P servers.
Given the serious multi-threaded potential of Rust, this seems like something it could potentially do really effectively, but I'm not sure how it'd be best to implement... maybe tasks that simply run
tokio::task::yield_now().await
frequently to yield Tokio more often, or a second Tokio runtime (e.g. have a configurable number of threads for the "background" workers, defaulting to 25% of the cores) to stop them hogging the live resources?The text was updated successfully, but these errors were encountered: