Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Builder updates for Blobs (EIP-4844) #3808

Closed
wants to merge 21 commits into from
Closed

Builder updates for Blobs (EIP-4844) #3808

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d7a9a35
add new blob_kzg_commitments field to builderBid
jimmygchen Dec 14, 2022
80bd9ff
add superstruct to BuilderBid
jimmygchen Dec 14, 2022
2df4c34
builder bid compiling with variants \o/
jimmygchen Dec 15, 2022
16fc69f
add feature config to tests
jimmygchen Dec 15, 2022
991ab98
Handle kzg commitments from getPayload response
jimmygchen Dec 15, 2022
fa4f2d6
Merge remote-tracking branch 'origin/eip4844' into builder-bid-4844
jimmygchen Dec 15, 2022
6c85b16
Fix comment
jimmygchen Dec 15, 2022
e90db2d
Fix formatting only
jimmygchen Dec 15, 2022
83154f0
Merge remote-tracking branch 'origin/eip4844' into builder-bid-4844
jimmygchen Dec 16, 2022
9f64803
Merge branch 'eip4844' into builder-bid-4844
jimmygchen Jan 3, 2023
a26f1a7
Merge branch 'eip4844' into builder-bid-4844
jimmygchen Jan 9, 2023
46829ad
Builder bid serde working
jimmygchen Jan 10, 2023
75bf1d5
Remove unnecessary Merge variant from BuilderBid
jimmygchen Jan 10, 2023
0b8f154
Add handling of ExecutionPayloadAndBlobsSidecar response from builder
jimmygchen Jan 10, 2023
78aa8a8
Merge branch 'eip4844' into builder-bid-4844
jimmygchen Jan 10, 2023
9222050
Publish blobs received from builder
jimmygchen Jan 11, 2023
6d232f9
Fix code warnings
jimmygchen Jan 11, 2023
a64eaad
Add mising kzg trusted setup in tests. Fix compilation errors.
jimmygchen Jan 11, 2023
1cac42c
Remove unnecessary clones and FIXMEs
jimmygchen Jan 11, 2023
9821acd
Merge branch 'eip4844' into builder-bid-4844
jimmygchen Jan 30, 2023
427581e
Merge branch 'eip4844' into builder-bid-4844
jimmygchen Feb 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions beacon_node/beacon_chain/src/blob_verification.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ use crate::{kzg_utils, BeaconChainError};
use state_processing::per_block_processing::eip4844::eip4844::verify_kzg_commitments_against_transactions;
use types::signed_beacon_block::BlobReconstructionError;
use types::{
BeaconBlockRef, BeaconStateError, BlobsSidecar, EthSpec, Hash256, KzgCommitment,
SignedBeaconBlock, SignedBeaconBlockAndBlobsSidecar, SignedBeaconBlockHeader, Slot,
Transactions,
BeaconBlock, BeaconBlockRef, BeaconStateError, BlobsSidecar, EthSpec, ExecutionPayload,
ExecutionPayloadHeader, Hash256, KzgCommitment, SignedBeaconBlock,
SignedBeaconBlockAndBlobsSidecar, SignedBeaconBlockHeader, Slot, Transactions,
};
use types::{Epoch, ExecPayload};

Expand Down
9 changes: 4 additions & 5 deletions beacon_node/builder_client/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
use eth2::types::builder_bid::SignedBuilderBid;
use eth2::types::{
AbstractExecPayload, BlindedPayload, EthSpec, ExecutionBlockHash, ExecutionPayload,
ForkVersionedResponse, PublicKeyBytes, SignedBeaconBlock, SignedValidatorRegistrationData,
Slot,
AbstractExecPayload, BlindedPayload, EthSpec, ExecutionBlockHash, ForkVersionedResponse,
PublicKeyBytes, SignedBeaconBlock, SignedValidatorRegistrationData, Slot,
};
pub use eth2::Error;
use eth2::{ok_or_error, StatusCode};
Expand Down Expand Up @@ -135,10 +134,10 @@ impl BuilderHttpClient {
}

/// `POST /eth/v1/builder/blinded_blocks`
pub async fn post_builder_blinded_blocks<E: EthSpec>(
pub async fn post_builder_blinded_blocks<E: EthSpec, Payload: DeserializeOwned>(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking to create a trait for ExecutionPayload and ExecutionPayloadAndSidecar to restrict the bound, but having some issues with getting this to compile

pub trait BuilderExecutionPayloadResponse: DeserializeOwned {}
impl<T: EthSpec> BuilderExecutionPayloadResponse for ExecutionPayloadAndBlobsSidecar<T> {}
impl<T: EthSpec> BuilderExecutionPayloadResponse for ExecutionPayload<T> {}

&self,
blinded_block: &SignedBeaconBlock<E, BlindedPayload<E>>,
) -> Result<ForkVersionedResponse<ExecutionPayload<E>>, Error> {
) -> Result<ForkVersionedResponse<Payload>, Error> {
let mut path = self.server.full.clone();

path.path_segments_mut()
Expand Down
177 changes: 129 additions & 48 deletions beacon_node/execution_layer/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,17 @@ use tokio::{
use tokio_stream::wrappers::WatchStream;
use tree_hash::TreeHash;
use types::consts::eip4844::BLOB_TX_TYPE;
use types::execution_payload::{ExecutionPayloadAndBlobsSidecar, PayloadWrapper};
use types::transaction::{AccessTuple, BlobTransaction, EcdsaSignature, SignedBlobTransaction};
use types::Withdrawals;
use types::{
blobs_sidecar::{Blobs, KzgCommitments},
BlindedPayload, BlockType, ChainSpec, Epoch, ExecutionBlockHash, ExecutionPayload,
ExecutionPayloadCapella, ExecutionPayloadEip4844, ExecutionPayloadMerge, ForkName,
};
use types::{AbstractExecPayload, BeaconStateError, ExecPayload, VersionedHash};
use types::{
AbstractExecPayload, BeaconStateError, Blob, ExecPayload, KzgCommitment, VersionedHash,
};
use types::{
ProposerPreparationData, PublicKeyBytes, Signature, SignedBeaconBlock, Slot, Transaction,
Uint256,
Expand Down Expand Up @@ -797,7 +800,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
self.log(),
"Requested blinded execution payload";
"relay_fee_recipient" => match &relay_result {
Ok(Some(r)) => format!("{:?}", r.data.message.header.fee_recipient()),
Ok(Some(r)) => format!("{:?}", r.data.message.header().fee_recipient()),
Ok(None) => "empty response".to_string(),
Err(_) => "request failed".to_string(),
},
Expand Down Expand Up @@ -833,7 +836,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
Ok(ProvenancedPayload::Local(local))
}
(Ok(Some(relay)), Ok(local)) => {
let header = &relay.data.message.header;
let header = relay.data.message.header().clone();

info!(
self.log(),
Expand Down Expand Up @@ -864,12 +867,30 @@ impl<T: EthSpec> ExecutionLayer<T> {
current_fork,
spec,
) {
Ok(()) => Ok(ProvenancedPayload::Builder(
BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
)),
Ok(()) => {
Ok(ProvenancedPayload::Builder(match relay.version.unwrap() {
ForkName::Base
| ForkName::Altair
| ForkName::Merge
| ForkName::Capella => BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
ForkName::Eip4844 => {
BlockProposalContents::PayloadAndBlobs {
payload: relay.data.message.header,
block_value: relay.data.message.value,
blobs: VariableList::default(),
kzg_commitments: relay
.data
.message
.blob_kzg_commitments()
.unwrap()
.clone(),
}
}
}))
}
Err(reason) if !reason.payload_invalid() => {
info!(
self.log(),
Expand Down Expand Up @@ -899,7 +920,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
}
}
(Ok(Some(relay)), Err(local_error)) => {
let header = &relay.data.message.header;
let header = relay.data.message.header().clone();

info!(
self.log(),
Expand All @@ -918,20 +939,56 @@ impl<T: EthSpec> ExecutionLayer<T> {
current_fork,
spec,
) {
Ok(()) => Ok(ProvenancedPayload::Builder(
BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
)),
Ok(()) => {
Ok(ProvenancedPayload::Builder(match relay.version.unwrap() {
ForkName::Base
| ForkName::Altair
| ForkName::Merge
| ForkName::Capella => BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
ForkName::Eip4844 => {
BlockProposalContents::PayloadAndBlobs {
payload: relay.data.message.header,
block_value: relay.data.message.value,
blobs: VariableList::default(),
kzg_commitments: relay
.data
.message
.blob_kzg_commitments()
.unwrap()
.clone(),
}
}
}))
}
// If the payload is valid then use it. The local EE failed
// to produce a payload so we have no alternative.
Err(e) if !e.payload_invalid() => Ok(ProvenancedPayload::Builder(
BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
)),
Err(e) if !e.payload_invalid() => {
Ok(ProvenancedPayload::Builder(match relay.version.unwrap() {
ForkName::Base
| ForkName::Altair
| ForkName::Merge
| ForkName::Capella => BlockProposalContents::Payload {
payload: relay.data.message.header,
block_value: relay.data.message.value,
},
ForkName::Eip4844 => {
BlockProposalContents::PayloadAndBlobs {
payload: relay.data.message.header,
block_value: relay.data.message.value,
blobs: VariableList::default(),
kzg_commitments: relay
.data
.message
.blob_kzg_commitments()
.unwrap()
.clone(),
}
}
}))
}
Err(reason) => {
metrics::inc_counter_vec(
&metrics::EXECUTION_LAYER_GET_PAYLOAD_BUILDER_REJECTIONS,
Expand Down Expand Up @@ -1779,7 +1836,8 @@ impl<T: EthSpec> ExecutionLayer<T> {
&self,
block_root: Hash256,
block: &SignedBeaconBlock<T, BlindedPayload<T>>,
) -> Result<ExecutionPayload<T>, Error> {
fork_name: ForkName,
) -> Result<PayloadWrapper<T>, Error> {
debug!(
self.log(),
"Sending block to builder";
Expand All @@ -1789,29 +1847,31 @@ impl<T: EthSpec> ExecutionLayer<T> {
if let Some(builder) = self.builder() {
let (payload_result, duration) =
timed_future(metrics::POST_BLINDED_PAYLOAD_BUILDER, async {
builder
.post_builder_blinded_blocks(block)
.await
.map_err(Error::Builder)
.map(|d| d.data)
match fork_name {
ForkName::Base | ForkName::Altair | ForkName::Merge | ForkName::Capella => {
let payload = builder
.post_builder_blinded_blocks::<T, ExecutionPayload<T>>(block)
.await
.map_err(Error::Builder)
.map(|d| d.data)?;
Ok(PayloadWrapper::Payload(payload))
}
ForkName::Eip4844 => {
let payload_and_blob = builder
.post_builder_blinded_blocks::<T, ExecutionPayloadAndBlobsSidecar<T>>(block)
.await
.map_err(Error::Builder)
.map(|d| d.data)?;
Ok(PayloadWrapper::PayloadAndBlob(payload_and_blob))
},
}
})
.await;

match &payload_result {
Ok(payload) => {
metrics::inc_counter_vec(
&metrics::EXECUTION_LAYER_BUILDER_REVEAL_PAYLOAD_OUTCOME,
&[metrics::SUCCESS],
);
info!(
self.log(),
"Builder successfully revealed payload";
"relay_response_ms" => duration.as_millis(),
"block_root" => ?block_root,
"fee_recipient" => ?payload.fee_recipient(),
"block_hash" => ?payload.block_hash(),
"parent_hash" => ?payload.parent_hash()
)
let payload_maybe = match &payload_result {
Ok(PayloadWrapper::Payload(payload)) => Some(payload),
Ok(PayloadWrapper::PayloadAndBlob(payload_and_blob)) => {
Some(&payload_and_blob.execution_payload)
}
Err(e) => {
metrics::inc_counter_vec(
Expand All @@ -1830,8 +1890,29 @@ impl<T: EthSpec> ExecutionLayer<T> {
.execution_payload()
.map(|payload| format!("{}", payload.parent_hash()))
.unwrap_or_else(|_| "unknown".to_string())
)
);
None
}
};

match payload_maybe {
Some(payload) => {
metrics::inc_counter_vec(
&metrics::EXECUTION_LAYER_BUILDER_REVEAL_PAYLOAD_OUTCOME,
&[metrics::SUCCESS],
);

info!(
self.log(),
"Builder successfully revealed payload";
"relay_response_ms" => duration.as_millis(),
"block_root" => ?block_root,
"fee_recipient" => ?payload.fee_recipient(),
"block_hash" => ?payload.block_hash(),
"parent_hash" => ?payload.parent_hash()
);
}
None => {}
}

payload_result
Expand Down Expand Up @@ -1954,11 +2035,11 @@ fn verify_builder_bid<T: EthSpec, Payload: AbstractExecPayload<T>>(
spec: &ChainSpec,
) -> Result<(), Box<InvalidBuilderPayload>> {
let is_signature_valid = bid.data.verify_signature(spec);
let header = &bid.data.message.header;
let payload_value = bid.data.message.value;
let header = &bid.data.message.header();
let payload_value = bid.data.message.value().clone();

// Avoid logging values that we can't represent with our Prometheus library.
let payload_value_gwei = bid.data.message.value / 1_000_000_000;
let payload_value_gwei = bid.data.message.value() / 1_000_000_000;
if payload_value_gwei <= Uint256::from(i64::max_value()) {
metrics::set_gauge_vec(
&metrics::EXECUTION_LAYER_PAYLOAD_BIDS,
Expand Down Expand Up @@ -2007,7 +2088,7 @@ fn verify_builder_bid<T: EthSpec, Payload: AbstractExecPayload<T>>(
} else if !is_signature_valid {
Err(Box::new(InvalidBuilderPayload::Signature {
signature: bid.data.signature.clone(),
pubkey: bid.data.message.pubkey,
pubkey: *bid.data.message.pubkey(),
}))
} else if payload_withdrawals_root != expected_withdrawals_root {
Err(Box::new(InvalidBuilderPayload::WithdrawalsRoot {
Expand Down
4 changes: 3 additions & 1 deletion beacon_node/http_api/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ use sysinfo::{System, SystemExt};
use system_health::observe_system_health_bn;
use tokio::sync::mpsc::{Sender, UnboundedSender};
use tokio_stream::{wrappers::BroadcastStream, StreamExt};
use types::signed_block_and_blobs::BlockWrapper;
use types::{
Attestation, AttestationData, AttesterSlashing, BeaconStateError, BlindedPayload,
CommitteeCache, ConfigAndPreset, Epoch, EthSpec, ForkName, FullPayload,
Expand Down Expand Up @@ -1123,7 +1124,8 @@ pub fn serve<T: BeaconChainTypes>(
chain: Arc<BeaconChain<T>>,
network_tx: UnboundedSender<NetworkMessage<T::EthSpec>>,
log: Logger| async move {
publish_blocks::publish_block(None, block, chain, &network_tx, log)
let block_wrapper = BlockWrapper::new(block);
publish_blocks::publish_block(None, block_wrapper, chain, &network_tx, log)
.await
.map(|()| warp::reply())
},
Expand Down
Loading