Skip to content

Commit

Permalink
Implement Electra devnet-1 changes (sigp#5937)
Browse files Browse the repository at this point in the history
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix genesis from eth1 for electra

* Fix Electra Fork Choice Tests (sigp#5764)

* add new operation tests

* Fix Electra Fork Choice Tests (sigp#5764)

* Fix Electra Fork Choice Tests (sigp#5764)

* Fix Electra Fork Choice Tests (sigp#5764)

* add epoch processing new tests

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix ef test compilation

* disable and enable ef test for epoch processing appropriately

* add consolidations to merkle calc for inclusion proof

* fix should_override_fc tests

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* get ssz tests compiling

* ssz merkle validity for only deneb

* im dumb af

* Merge remote-tracking branch 'upstream/ef-tests-electra' into ef-tests-electra

* enable tests

* Fix Consolidation Sigs & Withdrawals

* Merge pull request sigp#5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* comment out consolidation operation ef tests until they are added

* add signed consolidation ssz tests

* add as_iter to all List and Vector fields in the beacon state

* remove dbg

* Send unagg attestation based on fork

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Fix Epoch Processing Consolidations

* Fix Final Spec Tests!

* Fix ser/de

* Merge branch 'electra-engine-api' into beacon-api-electra

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* update electra readiness with new endpoints

* don't block mev boost till genesis finalization

* Publish all aggregates

* just one more check bro plz..

* Fix Bug In Block Processing with 0x02 Credentials

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request sigp#5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request sigp#5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request sigp#5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Rename deploy block electra (sigp#5853)

* Rename deploy_block.txt to deposit_contract_block.txt

* fmt

* Change for prater too

* fix: serde rename camle case for execution payload body (sigp#5846)

* Merge branch 'electra-engine-api' into beacon-api-electra

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Electra attestation changes rm decode impl (sigp#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (sigp#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (sigp#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (sigp#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Fix slasher tests (sigp#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (sigp#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* move committee bits to the last index in attestation (sigp#5924)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (sigp#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (sigp#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request sigp#5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations sigp#5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (sigp#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Fix Compilation Break

* Merge pull request sigp#5973 from ethDreamer/beacon-api-electra

Fix Compilation Break

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Rename Constants, Structures, and Fields

* Second Round

* Third Round

* Round Four

* Electra attestation changes sean review (sigp#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request sigp#5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request sigp#5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge pull request sigp#5974 from ethDreamer/electra-devnet-1

Electra: `devnet-1` Rename and Move Stuff

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* clean up enabled and disabled fork logic in ef tests

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Electra devnet 1 process operations (sigp#5994)

* Implement `process_consolidation_requests()`

* Finish Changes to `process_operations()`

* Fix Lint

* Fix test

* use electra_enabled()

* add light client ssz tests

* Update `engine-api` for `devnet-1`

* disable failing light client ssz tests

* correctly exclude light client struct from accessed files

* Fix Failing Tests

* Fix Calculations Relying on MAX_EB (sigp#5983)

* Fix Calculations Relying on MAX_EB

* Use spec fn for max_effective_balance

* The great renaming receipt -> request

* Address some more review comments

* Electra: Get `devnet-1` ef-tests Working

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Update beacon_node/beacon_chain/src/electra_readiness.rs

* Update consensus/types/src/chain_spec.rs

* Update pending balance deposit processing (sigp#6005)

* Update pending balance deposit processing

* Update single_pass.rs

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge pull request sigp#6001 from ethDreamer/electra-devnet-1-engine-api

Update `engine-api` for `devnet-1`

* Merge pull request sigp#6019 from ethDreamer/electra-devnet-1-ef-tests

Electra: Get `devnet-1` ef-tests Working

* Fix Bug in `engine_api` by Renaming Field

* Merge pull request sigp#6047 from ethDreamer/electra-devnet-1-engine-api-fix

Fix Bug in `engine_api` by Renaming Field

* one more stupid rename

* update GET requests

* update POST requests

* add client updates and test updates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* compile after merge

* unwrap -> unwrap_err

* self review

* fix tests

* convert op pool messages to electra in electra

* remove methods to post without content header

* Add BeaconBlocksByRange v3

* Revert "Add BeaconBlocksByRange v3"

This reverts commit e3ce7fc.

* filter instead of convert

* Add range sync metrics to track efficiency (sigp#6095)

* Add more range sync metrics to track efficiency

* Add ignored blocks metrics

* Enable the outbound rate limiter by default, and update blobs method quotas (sigp#6093)

* Enable the outbound rate limiter by default, and update blobs method quotas.

* Lint and book updates.

* Beacon api + validator electra (sigp#5744)

* Attestation superstruct changes for EIP 7549 (sigp#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (sigp#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (sigp#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (sigp#5713)

* Finish EF Test Fork Agnostic (sigp#5714)

* Superstruct `AggregateAndProof` (sigp#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request sigp#5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* process withdrawals updates

* cleanup withdrawals processing

* update `process_operations` deposit length check

* add apply_deposit changes

* add execution layer withdrawal request processing

* process deposit receipts

* add consolidation processing

* update process operations function

* exit updates

* clean up

* update slash_validator

* EIP7549 `get_attestation_indices` (sigp#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (sigp#5739)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* cargo fmt (sigp#5740)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* fix attestation verification

* Add new engine api methods

* Fix the versioning of v4 requests

* Handle new engine api methods in mock EL

* Note todo

* Fix todos

* Add support for electra fields in getPayloadBodies

* Add comments for potential versioning confusion

* udpates for aggregate attestation endpoint

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Sketch op pool changes

* fix get attesting indices (sigp#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Ef test fixes (sigp#5753)

* attestation related ef test fixes

* delete commented out stuff

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Aggregation Pool for Electra (sigp#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (sigp#5755)

* Get `electra_op_pool` up to date (sigp#5756)

* fix get attesting indices (sigp#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (sigp#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (sigp#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (sigp#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (sigp#5756)" (sigp#5757)

This reverts commit ab9e58a.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Compute on chain aggregate impl (sigp#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update beacon api aggregate attestationendpoint

* update the naive agg pool interface (sigp#5760)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* updates after merge

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Electra epoch processing

* add deposit limit for old deposit queue

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge pull request sigp#5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* don't fail on empty consolidations

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* update committee offset

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* update committee offset

* update committee offset

* update committee offset

* only increment the state deposit index on old deposit flow

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* use correct max eb in epoch cache initialization

* drop initiate validator ordering optimization

* fix initiate exit for single pass

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* accept new payload v4 in mock el

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Electra Fork Choice Tests (sigp#5764)

* Fix Electra Fork Choice Tests (sigp#5764)

* Fix Electra Fork Choice Tests (sigp#5764)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Consolidation Sigs & Withdrawals

* Merge pull request sigp#5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Send unagg attestation based on fork

* Fix ser/de

* Merge branch 'electra-engine-api' into beacon-api-electra

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (sigp#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* update electra readiness with new endpoints

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request sigp#5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request sigp#5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request sigp#5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix: serde rename camle case for execution payload body (sigp#5846)

* Merge branch 'electra-engine-api' into beacon-api-electra

* Electra attestation changes rm decode impl (sigp#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (sigp#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (sigp#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (sigp#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix slasher tests (sigp#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (sigp#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (sigp#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (sigp#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request sigp#5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations sigp#5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (sigp#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Compilation Break

* Merge pull request sigp#5973 from ethDreamer/beacon-api-electra

Fix Compilation Break

* Electra attestation changes sean review (sigp#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request sigp#5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request sigp#5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* The great renaming receipt -> request

* Address some more review comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Update beacon_node/beacon_chain/src/electra_readiness.rs

* Update consensus/types/src/chain_spec.rs

* update GET requests

* update POST requests

* add client updates and test updates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* compile after merge

* unwrap -> unwrap_err

* self review

* fix tests

* convert op pool messages to electra in electra

* remove methods to post without content header

* filter instead of convert

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix ef test compile

* use `enabled` fork methods

* get tests passing

* last updates

* Fix bug on engine api with consolidations (sigp#6101)

* Fix small bug in engine_api

* Merge pull request sigp#6116 from ethDreamer/electra-devnet-1

Fix payload_bodies bug in engine_api

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into ef-tests-electra

* add deposit request file

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* clean up after merge

* Fix Broken Test on `devnet-1`

* another small bug

* Merge pull request sigp#6132 from ethDreamer/devnet-1-fix-test

Fix Broken Test on `devnet-1`

* Fix Deserialization Bug in `engine-api`

* Merge pull request sigp#6235 from ethDreamer/fix_deserialization

Fix Deserialization Bug in `engine-api`

* Merge branch 'unstable' into electra-devnet-1

* remove import

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-devnet-1

* Remove false negative checks on `shuffling_is_compatible`

* Fix flaky validator monitor test
  • Loading branch information
ethDreamer authored Aug 22, 2024
1 parent 677f96a commit 9a295d0
Show file tree
Hide file tree
Showing 42 changed files with 874 additions and 624 deletions.
4 changes: 2 additions & 2 deletions beacon_node/beacon_chain/src/beacon_block_streamer.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::{metrics, BeaconChain, BeaconChainError, BeaconChainTypes, BlockProcessStatus};
use execution_layer::{ExecutionLayer, ExecutionPayloadBodyV1};
use execution_layer::{ExecutionLayer, ExecutionPayloadBody};
use slog::{crit, debug, error, Logger};
use std::collections::HashMap;
use std::sync::Arc;
Expand Down Expand Up @@ -57,7 +57,7 @@ struct BodiesByRange<E: EthSpec> {
struct BlockParts<E: EthSpec> {
blinded_block: Box<SignedBlindedBeaconBlock<E>>,
header: Box<ExecutionPayloadHeader<E>>,
body: Option<Box<ExecutionPayloadBodyV1<E>>>,
body: Option<Box<ExecutionPayloadBody<E>>>,
}

impl<E: EthSpec> BlockParts<E> {
Expand Down
2 changes: 0 additions & 2 deletions beacon_node/beacon_chain/src/beacon_chain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5531,8 +5531,6 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
bls_to_execution_changes: bls_to_execution_changes.into(),
blob_kzg_commitments: kzg_commitments
.ok_or(BlockProductionError::InvalidPayloadFork)?,
// TODO(electra): finish consolidations when they're more spec'd out
consolidations: Vec::new().into(),
},
}),
maybe_blobs_and_proofs,
Expand Down
12 changes: 4 additions & 8 deletions beacon_node/beacon_chain/tests/store_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1437,18 +1437,16 @@ fn check_shuffling_compatible(
head_state.current_epoch(),
|committee_cache, _| {
let state_cache = head_state.committee_cache(RelativeEpoch::Current).unwrap();
// We used to check for false negatives here, but had to remove that check
// because `shuffling_is_compatible` does not guarantee their absence.
//
// See: https://github.com/sigp/lighthouse/issues/6269
if current_epoch_shuffling_is_compatible {
assert_eq!(
committee_cache,
state_cache.as_ref(),
"block at slot {slot}"
);
} else {
assert_ne!(
committee_cache,
state_cache.as_ref(),
"block at slot {slot}"
);
}
Ok(())
},
Expand Down Expand Up @@ -1479,8 +1477,6 @@ fn check_shuffling_compatible(
let state_cache = head_state.committee_cache(RelativeEpoch::Previous).unwrap();
if previous_epoch_shuffling_is_compatible {
assert_eq!(committee_cache, state_cache.as_ref());
} else {
assert_ne!(committee_cache, state_cache.as_ref());
}
Ok(())
},
Expand Down
42 changes: 18 additions & 24 deletions beacon_node/beacon_chain/tests/validator_monitor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use beacon_chain::test_utils::{
use beacon_chain::validator_monitor::{ValidatorMonitorConfig, MISSED_BLOCK_LAG_SLOTS};
use logging::test_logger;
use std::sync::LazyLock;
use types::{Epoch, EthSpec, Keypair, MainnetEthSpec, PublicKeyBytes, Slot};
use types::{Epoch, EthSpec, ForkName, Keypair, MainnetEthSpec, PublicKeyBytes, Slot};

// Should ideally be divisible by 3.
pub const VALIDATOR_COUNT: usize = 48;
Expand Down Expand Up @@ -128,7 +128,7 @@ async fn produces_missed_blocks() {
let initial_blocks = slots_per_epoch * nb_epoch_to_simulate.as_u64() - 1;

// The validator index of the validator that is 'supposed' to miss a block
let mut validator_index_to_monitor = 1;
let validator_index_to_monitor = 1;

// 1st scenario //
//
Expand Down Expand Up @@ -201,34 +201,22 @@ async fn produces_missed_blocks() {
// Missed block happens when slot and prev_slot are not in the same epoch
// making sure that the cache reloads when the epoch changes
// in that scenario the slot that missed a block is the first slot of the epoch
validator_index_to_monitor = 7;
// We are adding other validators to monitor as these ones will miss a block depending on
// the fork name specified when running the test as the proposer cache differs depending on
// the fork name (cf. seed)
//
// If you are adding a new fork and seeing errors, print
// `validator_indexes[slot_in_epoch.as_usize()]` and add it below.
let validator_index_to_monitor_altair = 2;
// Same as above but for the merge upgrade
let validator_index_to_monitor_bellatrix = 4;
// Same as above but for the capella upgrade
let validator_index_to_monitor_capella = 11;
// Same as above but for the deneb upgrade
let validator_index_to_monitor_deneb = 3;
// Same as above but for the electra upgrade
let validator_index_to_monitor_electra = 6;

let harness2 = get_harness(
validator_count,
vec![
validator_index_to_monitor,
validator_index_to_monitor_altair,
validator_index_to_monitor_bellatrix,
validator_index_to_monitor_capella,
validator_index_to_monitor_deneb,
validator_index_to_monitor_electra,
],
);
let validator_index_to_monitor = match harness1.spec.fork_name_at_slot::<E>(Slot::new(0)) {
ForkName::Base => 7,
ForkName::Altair => 2,
ForkName::Bellatrix => 4,
ForkName::Capella => 11,
ForkName::Deneb => 3,
ForkName::Electra => 1,
};

let harness2 = get_harness(validator_count, vec![validator_index_to_monitor]);
let advance_slot_by = 9;
harness2
.extend_chain(
Expand Down Expand Up @@ -300,6 +288,12 @@ async fn produces_missed_blocks() {
duplicate_block_root = *_state2.block_roots().get(idx as usize).unwrap();
validator_indexes = _state2.get_beacon_proposer_indices(&harness2.spec).unwrap();
let not_monitored_validator_index = validator_indexes[slot_in_epoch.as_usize()];
// This could do with a refactor: https://github.com/sigp/lighthouse/issues/6293
assert_ne!(
not_monitored_validator_index,
validator_index_to_monitor,
"this test has a fragile dependency on hardcoded indices. you need to tweak some settings or rewrite this"
);

assert_eq!(
_state2.set_block_root(prev_slot, duplicate_block_root),
Expand Down
Loading

0 comments on commit 9a295d0

Please sign in to comment.