Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement data columns by network boilerplate #6224

Merged
merged 7 commits into from
Aug 13, 2024

Conversation

dapplion
Copy link
Collaborator

@dapplion dapplion commented Aug 5, 2024

Issue Addressed

Start porting network changes of das branch. This PR introduces mindless boilerplate around the new RPC routes data columns by range and by root. Both will be used by PeerDAS custody sync on both range sync and lookup sync.

Part of

Proposed Changes

Upstream PRs:

beacon_node/beacon_processor/src/lib.rs Outdated Show resolved Hide resolved
beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs Outdated Show resolved Hide resolved
beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs Outdated Show resolved Hide resolved
beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs Outdated Show resolved Hide resolved
beacon_node/lighthouse_network/src/rpc/config.rs Outdated Show resolved Hide resolved
@jimmygchen jimmygchen added waiting-on-author The reviewer has suggested changes and awaits thier implementation. das Data Availability Sampling das-unstable-merge labels Aug 6, 2024
@dapplion dapplion added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Aug 6, 2024
@@ -1142,6 +1131,10 @@ impl<E: EthSpec> BeaconProcessor<E> {
self.spawn_worker(item, idle_tx);
} else if let Some(item) = blbroots_queue.pop() {
self.spawn_worker(item, idle_tx);
} else if let Some(item) = dcbroots_queue.pop() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the intent before was to prioritize all by roots requests ahead of all by range requests?

// Prioritize by_root requests over by_range as the former are time sensitive for recovery

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no evidence this is useful, so let's just extend the deneb priorization

"Invalid fork name for data columns by root".to_string(),
)),
Some(fork_name) => {
if fork_name.deneb_enabled() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we want electra_enabled here right? otherwise we'll be incorrectly advertising this protocol as supported immediately on mainnet when we release

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! we can perhaps add a peer_das_enabled function on chain_spec so we can test this on deneb-based devnets.

if self.fork_context.spec.peer_das_enabled()

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PeerDAS is currently supported for both deneb and electra. This check does not advertise the topic on deneb, simply allows it to decode it. Advertise logic is in SupportedTopic::currently_supported.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check does not advertise the topic on deneb, simply allows it to decode it. Advertise logic is in SupportedTopic::currently_supported.

Ah right, thanks

@jimmygchen jimmygchen added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Aug 7, 2024
Copy link
Member

@AgeManning AgeManning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looks like standard boilerplate to me. Apart from the nice catches by Sean, I dont have anything further to add.

Comment on lines 404 to 409
if fork_context.spec.peerdas_scheduled() {
supported.extend_from_slice(&[
ProtocolId::new(SupportedProtocol::DataColumnsByRootV1, Encoding::SSZSnappy),
ProtocolId::new(SupportedProtocol::DataColumnsByRangeV1, Encoding::SSZSnappy),
]);
}
Copy link
Collaborator Author

@dapplion dapplion Aug 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is the one modifying the advertising rules. Because PeerDAS is still a "soft fork" not tied to Electra this check will advertise the topics in this cases:

EIP7594_FORK_EPOCH topics advertised
no value no
18446744073709551615 no
0 yes
128 yes

So, not advertised in mainnet, and advertised immediately on devnets.

@dapplion dapplion added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Aug 10, 2024
Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 🎉
Please resolve conflicts and see some comments above.

@jimmygchen jimmygchen added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Aug 12, 2024
@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Aug 12, 2024
Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 👍

@jimmygchen
Copy link
Member

@mergify queue

Copy link

mergify bot commented Aug 12, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at ff15c78

mergify bot added a commit that referenced this pull request Aug 12, 2024
@mergify mergify bot merged commit ff15c78 into sigp:unstable Aug 13, 2024
28 checks passed
@dapplion dapplion deleted the peerdas-network-boilerplate branch August 13, 2024 21:36
AgeManning pushed a commit to AgeManning/lighthouse that referenced this pull request Sep 3, 2024
* Implement data columns by network boilerplate

* Use correct quota values

* Address PR review

* Update currently_supported

* Merge remote-tracking branch 'sigp/unstable' into peerdas-network-boilerplate

* PR reviews

* Fix data column rpc request not being sent due to incorrect limits set. (sigp#6000)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
das Data Availability Sampling das-unstable-merge ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants