Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send IDONTWANT on publish to avoid downloading data we already have #6513

Open
wants to merge 3 commits into
base: unstable
Choose a base branch
from

Conversation

jimmygchen
Copy link
Member

Issue Addressed

With the introduction of getBlobsV1 optimisation, nodes may be publishing blobs if they retrieved it from the EL but haven't received it from gossip. In these scenarios, the node already have the blobs and can save download bandwidth by sending out IDONTWANT to peers on the blobs we publish, so that it avoids downloading them from gossip.

This would apply to all messages > 1000 bytes (idontwant_message_size_threshold) though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding a send_idontwant bool flag to the behavour.publish function, and only set it to true when publishing blobs?

@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Oct 18, 2024
@michaelsproul michaelsproul added the v6.0.0 New major release for hierarchical state diffs label Oct 18, 2024
Copy link
Member

@jxs jxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for starting this JImmy, left a comment.
Meanwhile the spec is clear in stating that IDONTWANT's should be sent for received messages:

In more specific terms, a new control message is introduced: IDONTWANT. It's primarily intended to notify mesh peers that the node already received a message and there is no need to send its duplicate.

and:

When the peer receives the first message instance it immediately broadcasts (not queue for later piggybacking) IDONTWANT with the messageId to all its mesh peers. This could be performed prior to the message validation to further increase the effectiveness of the approach.

can we forward this conversation to the spec and open an issue there proposing to also update the spec?

beacon_node/lighthouse_network/gossipsub/src/behaviour.rs Outdated Show resolved Hide resolved
@jimmygchen jimmygchen added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-merge This PR is ready to merge. labels Oct 18, 2024
Copy link
Member

@jxs jxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reading on your comment Jimmy,

though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding a send_idontwant bool flag to the behavour.publish function, and only set it to true when publishing blobs

I think this makes sense, we probably should offer this option to users

beacon_node/lighthouse_network/gossipsub/src/behaviour.rs Outdated Show resolved Hide resolved
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Oct 31, 2024
@jimmygchen
Copy link
Member Author

I think this makes sense, we probably should offer this option to users

I had a chat with Age regarding this and we decided not to add this flag.

The idontwant_message_size_threshold should limit the amount of IDONTWANT being sent to peers - if a message exceeds idontwant_message_size_threshold, it's probably worth sending an IDONTWANT.

Let me know what you think!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Networking ready-for-review The code is ready for review v6.0.0 New major release for hierarchical state diffs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants