diff --git a/docs/_sidebar.md b/docs/_sidebar.md index d5e1afe4..02c0c28e 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -42,11 +42,11 @@ - [RLP Serialization](/wiki/EL/RLP.md) - Consensus Layer - [Overview](/wiki/CL/overview.md) + - [Client architecture](/wiki/CL/cl-architecture.md) + - [CL Networking](/wiki/CL/cl-networking.md) - [CL Specs](/wiki/CL/cl-specs.md) - - [Client architecture](/wiki/CL/client-architecture.md) - - [CL Clients](/wiki/CL/cl-clients.md) - [Beacon API](/wiki/CL/beacon-api.md) - - [CL Networking](/wiki/CL/cl-networking.md) + - [CL Clients](/wiki/CL/cl-clients.md) - [SSZ Serialization](/wiki/CL/SSZ.md) - [Merkleization](/wiki/CL/merkleization.md) - Development diff --git a/docs/images/cl/RANDAO.png b/docs/images/cl/RANDAO.png new file mode 100644 index 00000000..3648ed87 Binary files /dev/null and b/docs/images/cl/RANDAO.png differ diff --git a/docs/images/cl/blobs.png b/docs/images/cl/blobs.png new file mode 100644 index 00000000..cbf1e05c Binary files /dev/null and b/docs/images/cl/blobs.png differ diff --git a/docs/images/cl/blockchain.svg b/docs/images/cl/blockchain.svg new file mode 100644 index 00000000..9c7f7f29 --- /dev/null +++ b/docs/images/cl/blockchain.svg @@ -0,0 +1,491 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/cl/blocktree.svg b/docs/images/cl/blocktree.svg new file mode 100644 index 00000000..fcd5da3e --- /dev/null +++ b/docs/images/cl/blocktree.svg @@ -0,0 +1,657 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/cl/checkpoints.jpg b/docs/images/cl/checkpoints.jpg new file mode 100644 index 00000000..460b995b Binary files /dev/null and b/docs/images/cl/checkpoints.jpg differ diff --git a/docs/images/cl/cl-ideal.png b/docs/images/cl/cl-ideal.png new file mode 100644 index 00000000..ec67a64c Binary files /dev/null and b/docs/images/cl/cl-ideal.png differ diff --git a/docs/images/cl/cl-networking-overview.webp b/docs/images/cl/cl-networking-overview.webp new file mode 100644 index 00000000..1a91621c Binary files /dev/null and b/docs/images/cl/cl-networking-overview.webp differ diff --git a/docs/images/cl/cl-real.png b/docs/images/cl/cl-real.png new file mode 100644 index 00000000..5957b005 Binary files /dev/null and b/docs/images/cl/cl-real.png differ diff --git a/docs/images/cl/cl.png b/docs/images/cl/cl.png new file mode 100644 index 00000000..8d7376b7 Binary files /dev/null and b/docs/images/cl/cl.png differ diff --git a/docs/images/cl/client-architecture.png b/docs/images/cl/client-architecture.png new file mode 100644 index 00000000..30f0122c Binary files /dev/null and b/docs/images/cl/client-architecture.png differ diff --git a/docs/images/cl/committees.png b/docs/images/cl/committees.png new file mode 100644 index 00000000..c3c99e67 Binary files /dev/null and b/docs/images/cl/committees.png differ diff --git a/docs/images/cl/el.png b/docs/images/cl/el.png new file mode 100644 index 00000000..b8940c0d Binary files /dev/null and b/docs/images/cl/el.png differ diff --git a/docs/images/cl/finalization.png b/docs/images/cl/finalization.png new file mode 100644 index 00000000..57e33f0f Binary files /dev/null and b/docs/images/cl/finalization.png differ diff --git a/docs/images/cl/network.png b/docs/images/cl/network.png new file mode 100644 index 00000000..9e79be32 Binary files /dev/null and b/docs/images/cl/network.png differ diff --git a/docs/images/cl/reorg-0.svg b/docs/images/cl/reorg-0.svg new file mode 100644 index 00000000..c8eac812 --- /dev/null +++ b/docs/images/cl/reorg-0.svg @@ -0,0 +1,478 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/cl/reorg-1.svg b/docs/images/cl/reorg-1.svg new file mode 100644 index 00000000..62bb3ae8 --- /dev/null +++ b/docs/images/cl/reorg-1.svg @@ -0,0 +1,518 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/cl/slots-and-epochs.png b/docs/images/cl/slots-and-epochs.png new file mode 100644 index 00000000..ac2e9d71 Binary files /dev/null and b/docs/images/cl/slots-and-epochs.png differ diff --git a/docs/images/cl/validator-lifecycle.png b/docs/images/cl/validator-lifecycle.png new file mode 100644 index 00000000..16a98d85 Binary files /dev/null and b/docs/images/cl/validator-lifecycle.png differ diff --git a/docs/images/cl/validators.png b/docs/images/cl/validators.png new file mode 100644 index 00000000..e63721de Binary files /dev/null and b/docs/images/cl/validators.png differ diff --git a/docs/wiki/CL/beacon-api.md b/docs/wiki/CL/beacon-api.md index 53e59474..1e9701bb 100644 --- a/docs/wiki/CL/beacon-api.md +++ b/docs/wiki/CL/beacon-api.md @@ -1,7 +1,60 @@ -# Beacon API +# Beacon Chain Spec and APIs -> :warning: This article is a [stub](https://en.wikipedia.org/wiki/Wikipedia:Stub), help the wiki by [contributing](/contributing.md) and expanding it. +At the core of Ethereum Proof-of-Stake is a system chain called the "Beacon Chain". The beacon chain stores and manages the registry of validators. In the initial deployment phases of Proof-of-Stake, the only mechanism to become a validator is to make a one-way(withdrawable after Capella) ETH transaction to a deposit contract on the Ethereum proof-of-work chain. Activation as a validator happens when deposit receipts are processed by the Beacon Chain, the activation balance is reached, and a queuing process is completed. Exit is either voluntary or done forcibly as a penalty for misbehavior. The primary source of load on the beacon chain is "attestations". Attestations are simultaneously availability votes for a shard block (in a later upgrade) and proof-of-stake votes for a beacon block (Phase 0). -Beacon API is the endpoint provided by consensus layer clients. It's the interface for interacting with consensus for users and validators. +This section will cover some important parts of Beacon Chain Spec and APIs. Also checkout complete [Beacon Chain Spec](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md) and [Beacon API reference](https://ethereum.github.io/beacon-APIs/#/). -Check the [Beacon API reference](https://ethereum.github.io/beacon-APIs/#/). \ No newline at end of file +Beacon API is the REST endpoint provided by consensus layer clients (beacon nodes). It's the interface to read information about the consensus and used by validator clients. + +## Containers + +`BeaconState` + +```python +class BeaconState(Container): + # Versioning + genesis_time: uint64 + genesis_validators_root: Root + slot: Slot + fork: Fork + # History + latest_block_header: BeaconBlockHeader + block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] + # Eth1 + eth1_data: Eth1Data + eth1_deposit_index: uint64 + application_state_root: Bytes32 + # Registry + validators: List[Validator, VALIDATOR_REGISTRY_LIMIT] + balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT] + # Randomness + randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR] + # Slashings + slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] + # Attestations + previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH] + current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH] + # Finality + justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch + previous_justified_checkpoint: Checkpoint # Previous epoch snapshot + current_justified_checkpoint: Checkpoint + finalized_checkpoint: Checkpoint +``` + +#### `BeaconBlockBody` + +```python +class BeaconBlockBody(Container): + randao_reveal: BLSSignature + eth1_data: Eth1Data # Eth1 data vote + graffiti: Bytes32 # Arbitrary data + # Operations + proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS] + attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS] + attestations: List[Attestation, MAX_ATTESTATIONS] + deposits: List[Deposit, MAX_DEPOSITS] + voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS] + application_payload: ApplicationPayload +``` diff --git a/docs/wiki/CL/cl-architecture.md b/docs/wiki/CL/cl-architecture.md new file mode 100644 index 00000000..ef8b0d9a --- /dev/null +++ b/docs/wiki/CL/cl-architecture.md @@ -0,0 +1,246 @@ +# Consensus Layer (CL) architecture + +> Many blockchain consensus protocols are _forkful_. Forkful chains use a fork choice rule, and sometimes undergo reorganisations. +> +> Ethereum's consensus protocol combines two separate consensus protocols. _LMD GHOST_ essentially provides liveness. _Casper FFG_ provides finality. Together they are known as _Gasper_. +> +> In a _live"_ protocol, something good always happens. In a _safe_ protocol, nothing bad ever happens. No practical protocol can be always safe and always live. + +## Fork-choice Mechanism + +As described in [BFT](/wiki/CL/overview.md?id=byzantine-fault-tolerance-bft-and-byzantine-generals39-problem), for various reasons - like network delays, outages, out-of-order messages, or malicious behavior — nodes in the network can have different views of the network's state. Eventually, we want every honest node to agree on an identical, linear history and a common view of the system's state. The protocol's fork choice rule is what helps achieve this agreement. + +#### Block Tree +Given a block tree and decision criteria based on a node's local view of the network, the fork choice rule is designed to select the branch that is most likely to become the final, linear, canonical chain. It chooses the branch least likely to be pruned out as nodes converge on a common view. + + + +
+ +![Diagram for Block Tree](../../images/cl/blocktree.svg) + +
+ +_The fork choice rule picks a head block from the candidates. The head block identifies a unique linear blockchain back to the Genesis block._ + +
+
+ +#### Fork choice rules + +The fork choice rule implicitly selects a branch by choosing a block at the branch's tip, called the head block. For any correct node, the first rule of any fork choice is that the chosen block must be valid according to protocol rules, and all its ancestors must also be valid. Any invalid block is ignored, and blocks built on an invalid block are also invalid. + +There are several examples of different fork choice rules: + +- **Proof-of-work**: In Ethereum and Bitcoin, the "heaviest chain rule" (sometimes called "longest chain", though not strictly accurate) is used. The head block is the tip of the chain with the most cumulative "work" done. +> Note that contrary to popular belief, Ethereum's Proof-of-work protocol [did not use](https://ethereum.stackexchange.com/questions/38121/why-did-ethereum-abandon-the-ghost-protocol/50693#50693) any form of GHOST in its fork choice. This misconception is very persistent, probably due to the [Ethereum Whitepaper](https://ethereum.org/en/whitepaper/#modified-ghost-implementation). Eventually when Vitalik was asked about it, he confirmed that although GHOST had been planned under PoW it was never implemented due to concerns about some unspecified attacks. The heaviest chain rule was simpler and well tested. It worked fine. +- **Casper FFG (Proof-of-Stake)**: In Ethereum's PoS Casper FFG protocol, the fork-choice rule is to "follow the chain containing the justified checkpoint of the **greatest height**" and never revert a finalized block. +- **LMD GHOST (Proof-of-Stake)**: In Ethereum's PoS LMD GHOST protocol, the fork-choice rule is to take the "Greediest Heaviest Observed SubTree". It involves counting accumulated votes from validators for blocks and their descendent blocks. It also applies the same rule as Casper FFG. + +Each of these fork choice rules assigns a numeric score to a block. The winning block, or head block, has the highest score. The goal is that all correct nodes, when they see a certain block, will agree that it is the head and follow its branch. This way, all correct nodes will eventually agree on a single canonical chain that goes back to Genesis. + +#### Reorgs and Reversion + +As a node receives new votes (and new votes for blocks in Proof-of-stake), it re-evaluates the fork choice rule with this new information. Usually, a new block will be a child of the current head block, and it will become the new head block. + +Sometimes, however, the new block might be a descendant of a different block in the block tree. If the node doesn't have the parent block of the new block, it will ask its peers for it and any other missing blocks. + +Running the fork choice rule on the updated block tree might show that the new head block is on a different branch than the previous head block. When this happens, the node must perform a reorg (reorganisation). This means it will remove (revert) blocks it previously included and adopt the blocks on the new head's branch. + +For example, if a node has blocks $A, B, D, E,$ and $F$ in its chain, and it views $F$ as the head block, it knows about block $$ but it does not appear in its view of the chain; it is on a side branch. + + + +
+ +![Diagram for Reorg-0](../../images/cl/reorg-0.svg) + +
+ +_At this point, the node believes that block $F$ is the best head, therefore its chain is blocks $[A \leftarrow B \leftarrow D \leftarrow E \leftarrow F]$_ + +
+
+ +When the node later receives block $G$, which is built on block $C$, not on its current head block $F$, it must decide if $G$ should be the new head. Just for example, If the fork choice rule says $G$ is the better head block, the node will revert blocks $D, E,$ and $F$. It will remove them from its chain, as if they were never received, and go back to the state after block $B$. + +Then, the node will add blocks $C$ and $G$ to its chain and process them. After this reorg, the node's chain will be $A, B, C,$ and $G$. + + + +
+ +![Diagram for Reorg-1](../../images/cl/reorg-1.svg) + +
+ +_Now the node believes that block $G$ is the best head, therefore its chain must change to the blocks $[A \leftarrow B \leftarrow C \leftarrow G]$_ + +
+
+ +Later, perhaps, a block $H$ might appear, that's built on $F$, and the fork choice rule says $H$ should be the new head, the node will reorg again, reverting to block $B$ and replaying blocks on $H$'s branch. + +Short reorgs of one or two blocks are common due to network delays. Longer reorgs should be rare unless the chain is under attack or there is a bug in the fork choice rule or its implementation. + +### Safety and Liveness + +In consensus mechanisms, two key concepts are safety and liveness. + +**Safety** means "nothing bad ever happens," such as preventing double-spending or finalizing conflicting checkpoints. It ensures consistency, meaning all honest nodes should always agree on the state of the blockchain. + +**Liveness** means "something good eventually happens," ensuring the blockchain can always add new blocks and never gets stuck in a deadlock. + +**CAP Theorem** states that no distributed system can provide consistency, availability, and partition tolerance simultaneously. This means we can't design a system that is both safe and live under all circumstances when communication is unreliable. + +#### Ethereum Prioritizes Liveness + +Ethereum’s consensus protocol aims to offer both safety and liveness in good network conditions. However, it prioritizes liveness during network issues. In a network partition, nodes on each side will continue to produce blocks but won't achieve finality (a safety property). If the partition persists, each side may finalize different histories, leading to two irreconcilable, independent chains. + +Thus, while Ethereum strives for both safety and liveness, it leans towards ensuring the network remains live and continues to process transactions, even at the cost of potential safety issues during severe network disruptions. + +## The Ghosts in the Machine + +Ethereum's Proof-of-Stake consensus protocol combines two separate protocols: [LMD GHOST](/wiki/cl/gasper?id=lmd-ghost.md) and [Casper FFG](/wiki/cl/gasper?id=casper-ffg.md). Together, they form the consensus protocol known as "Gasper". Detailed Information about both protocols and how they work in combination are covered in the next section [Gasper]. + +Gasper aims to combine the strengths of both LMD GHOST and Casper FFG. LMD GHOST provides liveness, ensuring the chain keeps running by producing new blocks regularly. However, it is prone to forks and not formally safe. Casper FFG, on the other hand, provides safety by periodically finalizing the chain, protecting it from long reversions. + +In essence, LMD GHOST keeps the chain moving forward, while Casper FFG ensures stability by finalizing blocks. This combination allows Ethereum to prioritize liveness, meaning the chain continues to grow even if Casper FFG can't finalize blocks. Although this combined mechanism isn't always perfect and has some complexities, it is a practical engineering solution that works well in practice for Ethereum. + +## Architecture + +Ethereum is a decentralized network of nodes that communicate via peer-to-peer connections. These connections are formed by computers running Ethereum's client software. + + + +
+ +![Diagram for Network](../../images/cl/network.png) + +
+ +_Nodes aren't required to run a validator client (green ones) to be a part of the network, however to take part in consensus one needs to stake 32 ETH and run a validator client._ + +
+
+ +### Components of the Consensus Layer + +- **Beacon Node**: Beacon nodes use client software to coordinate Ethereum's proof-of-stake consensus. Examples include Prysm, Teku, Lighthouse, and Nimbus. Beacon nodes communicate with other beacon nodes, a local execution node, and optionally, a local validator. + +- **Validator**: Validator client is the software that allows people to stake 32 ETH in Ethereum's consensus layer. Validators propose blocks in the Proof-of-Stake system, which replaced Proof-of-work miners. Validators communicate only with a local beacon node, which instructs them and broadcasts their work to the network. + +The main Ethereum network hosting real-world applications is called Ethereum Mainnet. Ethereum Mainnet is the live, production instance of Ethereum that mints and manages real Ethereum (ETH) and holds real monetary value. + +There are also test networks that mint and manage test Ethereum for developers, node runners, and validators to test new functionality before using real ETH on Mainnet. Each Ethereum network has two layers: the execution layer (EL) and the consensus layer (CL). Every Ethereum node contains software for both layers: execution-layer client software (like Nethermind, Besu, Geth, and Erigon) and consensus-layer client software (like Prysm, Teku, Lighthouse, Nimbus, and Lodestar). + + + +
+ +![Diagram for CL](../../images/cl/cl.png) + +
+ +**Consensus Layer** is responsible for maintaining consensus chain (beacon chain) and processing the consensus blocks (beacon blocks) and attestations received from other peers. **Consensus clients** participate in a separate [peer-to-peer network](/wiki/cl/cl-networking.md) with a different specification from execution clients. They need to participate in block gossip to receive new blocks from peers and broadcast blocks when it's their turn to propose. + +Both EL and CL clients run in parallel and need to be connected for communication. The consensus client provides instructions to the execution client, and the execution client passes transaction bundles to the consensus client to include in Beacon blocks. Communication is achieved using a local RPC connection via the **Engine-API**. They share an [ENR](/wiki/cl/cl-networking?id=ethereum-enr) with separate keys for each client (eth1 key and eth2 key). + +### Control Flow + +**When the consensus client is not the block producer:** +1. Receives a block via the block gossip protocol. +2. Pre-validates the block. +3. Sends transactions in the block to the execution layer as an execution payload. +4. Execution layer executes transactions and validates the block state. +5. Execution layer sends validation data back to the consensus layer. +6. Consensus layer adds the block to its blockchain and attests to it, broadcasting the attestation over the network. + +**When the consensus client is the block producer:** +1. Receives notice of being the next block producer. +2. Calls the create block method in the execution client. +3. Execution layer accesses the transaction mempool. +4. Execution client bundles transactions into a block, executes them, and generates a block hash. +5. Consensus client adds transactions and block hash to the beacon block. +6. Consensus client broadcasts the block over the block gossip protocol. +7. Other clients validate the block and attest to it. +8. Once attested by sufficient validators, the block is added to the head of the chain, justified, and finalized. + + +### State Transitions + +The state transition function is essential in blockchains. Each node maintains a state that reflects its view of the world. + +Nodes update their state by applying blocks in order using a "state transition function". This function is "pure", meaning its output depends only on the input and has no side effects. Thus, if every node starts with the same state (Genesis state) and applies the same blocks, they all end up with the same state. If they don't, there's a consensus failure. + +If $S$ is a beacon state and $B$ a beacon block, the state transition function $f$ is: + +$$S' \equiv f(S, B)$$ + +Here, $S$ is the pre-state and $S'$ is the post-state. The function $f$ is iterated with each new block to update the state. + +### Beacon Chain State Transitions + +Unlike the block-driven Proof-of-work, the beacon chain is slot-driven. State updates depend on slot progress, regardless of block presence. + +The beacon chain's state transition function includes: + +1. **Per-slot transition**: $S' \equiv f_s(S)$ +2. **Per-block transition**: $S' \equiv f_b(S, B)$ +3. **Per-epoch transition**: $S' \equiv f_e(S)$ + +Each function updates the chain at specific times, as defined in the beacon chain specification. + +### Validity Conditions + +The post-state from a pre-state and a signed block is `state_transition(state, signed_block)`. Transitions causing unhandled exceptions (e.g., failed asserts or out-of-range accesses) or uint64 overflows/underflows are invalid. + +### Beacon chain state transition function + +The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. + +```python +def state_transition(state: BeaconState, signed_block: SignedBeaconBlock, validate_result: bool=True) -> None: + block = signed_block.message + # Process slots (including those with no blocks) since block + process_slots(state, block.slot) + # Verify signature + if validate_result: + assert verify_block_signature(state, signed_block) + # Process block + process_block(state, block) + # Verify state root + if validate_result: + assert block.state_root == hash_tree_root(state) +``` + +```python +def verify_block_signature(state: BeaconState, signed_block: SignedBeaconBlock) -> bool: + proposer = state.validators[signed_block.message.proposer_index] + signing_root = compute_signing_root(signed_block.message, get_domain(state, DOMAIN_BEACON_PROPOSER)) + return bls.Verify(proposer.pubkey, signing_root, signed_block.signature) +``` + +```python +def process_slots(state: BeaconState, slot: Slot) -> None: + assert state.slot < slot + while state.slot < slot: + process_slot(state) + # Process epoch on the start slot of the next epoch + if (state.slot + 1) % SLOTS_PER_EPOCH == 0: + process_epoch(state) + state.slot = Slot(state.slot + 1) +``` + + +## Resources + +- Vitalik Buterin, ["Parametrizing Casper: the decentralization/finality time/overhead tradeoff"](https://medium.com/@VitalikButerin/parametrizing-casper-the-decentralization-finality-time-overhead-tradeoff-3f2011672735) +- [Engine API spec](https://hackmd.io/@n0ble/consensus_api_design_space) +- [Vitalik's Annotated Ethereum 2.0 Spec](https://notes.ethereum.org/@vbuterin/SkeyEI3xv) +- Ethereum, ["Eth2: Annotated Spec"](https://github.com/ethereum/annotated-spec) +- Martin Kleppmann, [Distributed Systems.](https://www.youtube.com/playlist?list=PLeKd45zvjcDFUEv_ohr_HdUFe97RItdiB) +- Leslie Lamport et al., [The Byzantine Generals Problem.](https://lamport.azurewebsites.net/pubs/byz.pdf) +- Austin Griffith, [Byzantine Generals - ETH.BUILD.](https://www.youtube.com/watch?v=c7yvOlwBPoQ) +- Michael Sproul, ["Inside Ethereum"](https://www.youtube.com/watch?v=LviEOQD9e8c) +- [Eth2 Handbook by Ben Edgington](https://eth2book.info/capella/part2/consensus/) \ No newline at end of file diff --git a/docs/wiki/CL/cl-networking.md b/docs/wiki/CL/cl-networking.md index cb75d7b2..575f4d51 100644 --- a/docs/wiki/CL/cl-networking.md +++ b/docs/wiki/CL/cl-networking.md @@ -1,46 +1,33 @@ # Networking -The Consensus clients use [libp2p][libp2p] as the peer-to-peer protocol, -[discv5][discv5] for peer discovery, [libp2p-noise][libp2p-noise] for -encryption, [SSZ][ssz] for encoding, and, optionally, [Snappy][snappy] for -compression. +The Consensus clients use [libp2p][libp2p] as the peer-to-peer protocol, [discv5][discv5] for peer discovery, [libp2p-noise][libp2p-noise] for encryption, [SSZ][ssz] for encoding, and, optionally, [Snappy][snappy] for compression. + +## ENR (Ethereum Node Records) + +## discv5 + ## Specs -The [Phase 0 -- Networking][consensus-networking] page specifies the network -fundamentals, protocols, and rationale/design choices. +The [Phase 0 -- Networking][consensus-networking] page specifies the network fundamentals, protocols, and rationale/design choices. ## libp2p - P2P protocol -[libp2p][libp2p] is used as the peer-to-peer protocol. -[libp2p and Ethereum][libp2p-and-eth] is a great article for a deep-dive on the -history of libp2p, and its adoption in the Consensus clients. +[libp2p][libp2p] is used as the peer-to-peer protocol. [libp2p and Ethereum][libp2p-and-eth] is a great article for a deep-dive on the history of libp2p, and its adoption in the Consensus clients. ## libp2p-noise - Encryption -The [Noise framework][noise-framework] is not a protocol itself, but a framework -for designing key exchange protocols. The [specification][noise-specification] -is a great place to start. +The [Noise framework][noise-framework] is not a protocol itself, but a framework for designing key exchange protocols. The [specification][noise-specification] is a great place to start. -There are many [patterns][noise-patterns] which describe the key exchange -process. The pattern used in the consensus clients is [`XX`][noise-xx] -(transmit-transmit), meaning that both the initiator, and responder transmit -their public key in the initial stages of the key exchange. +There are many [patterns][noise-patterns] which describe the key exchange process. The pattern used in the consensus clients is [`XX`][noise-xx] (transmit-transmit), meaning that both the initiator, and responder transmit their public key in the initial stages of the key exchange. ## SSZ - Encoding -[Simple serialize (SSZ)][ssz] replaces the [RLP][rlp] serialization used on the -execution layer everywhere across the consensus layer except the peer discovery -protocol. SSZ is designed to be deterministic and also to Merkleize efficiently. -SSZ can be thought of as having two components: a serialization scheme and a -Merkleization scheme that is designed to work efficiently with the serialized -data structure. +[Simple serialize (SSZ)][ssz] replaces the [RLP][rlp] serialization used on the execution layer everywhere across the consensus layer except the peer discovery protocol. SSZ is designed to be deterministic and also to Merkleize efficiently. SSZ can be thought of as having two components: a serialization scheme and a Merkleization scheme that is designed to work efficiently with the serialized data structure. ## Snappy - Compression -[Snappy][snappy] is a compression scheme created by engineers at Google in 2011. -It's main design considerations prioritize compression/decompression speed, -while still having a reasonable compression ratio. +[Snappy][snappy] is a compression scheme created by engineers at Google in 2011. It's main design considerations prioritize compression/decompression speed, while still having a reasonable compression ratio. ## Related R&D @@ -67,3 +54,4 @@ while still having a reasonable compression ratio. [rlp]: https://ethereum.org/developers/docs/data-structures-and-encoding/rlp [snappy]: https://en.wikipedia.org/wiki/Snappy_(compression) [ssz]: https://ethereum.org/developers/docs/data-structures-and-encoding/ssz +[blog]: https://medium.com/coinmonks/dissecting-the-ethereum-networking-stack-node-discovery-4b3f7895f83f diff --git a/docs/wiki/CL/client-architecture.md b/docs/wiki/CL/client-architecture.md deleted file mode 100644 index bfad295b..00000000 --- a/docs/wiki/CL/client-architecture.md +++ /dev/null @@ -1,10 +0,0 @@ -# CL Client architecture - -> :warning: This article is a [stub](https://en.wikipedia.org/wiki/Wikipedia:Stub), help the wiki by [contributing](/contributing.md) and expanding it. - -Beacon Chain clients are implementing various fundamental features: - -- Forkchoice mechanism -- Engine API for communication with the execution client -- Beacon APIs for validators and users -- libp2p protocol for communication with other CL clients \ No newline at end of file diff --git a/docs/wiki/CL/gasper.md b/docs/wiki/CL/gasper.md new file mode 100644 index 00000000..6292dfbe --- /dev/null +++ b/docs/wiki/CL/gasper.md @@ -0,0 +1,40 @@ +# Gasper + +## LMD GHOST (Latest Message Driven -- Greediest Heaviest Observed SubTree) + + + +## Casper FFG (Friendly Finality Gadget) + + + + +- Hybrid Fork-choice (Refer pos-evolution in ethereum post/book) +Possible attacks +- simple ex-ante reorg +- weighted proposer boost + +Solutions: +- view-merge strategy + + +## Resources + +- [Combining GHOST and Casper](https://arxiv.org/pdf/2003.03052) +- [Yang X Zhang- Combining GHOST and Casper](https://www.youtube.com/watch?v=V0RjGmFE35U) +- [Introduction to Gasper from Ethereum.org](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/gasper/) +- [Evolution of Ethereum PoS Consensus Protocol](https://github.com/ethereum/pos-evolution/blob/master/pos-evolution.md) +- [Goldfish](https://arxiv.org/pdf/2209.03255) +- [Casper FFG](https://arxiv.org/pdf/1710.09437) +- [LMD Ghost](https://inevitableeth.com/home/ethereum/network/consensus/lmd-ghost) +- [RLMD GHOST with Luca Zanolini](https://www.youtube.com/watch?v=F2olypDSVnA) +- [Comparison of different LMD Ghost Implementations by ProtoLambda](https://github.com/protolambda/lmd-ghost) \ No newline at end of file diff --git a/docs/wiki/CL/overview.md b/docs/wiki/CL/overview.md index 0eba8dd2..dca4c7fa 100644 --- a/docs/wiki/CL/overview.md +++ b/docs/wiki/CL/overview.md @@ -1,20 +1,356 @@ -# Consensus layer +# Consensus Layer Overview -The Ethereum consensus layer defines the mechanism for nodes to agree on the network's state. This layer currently employs Proof-of-Stake (PoS), a crypto-economic system. PoS encourages honest behavior by requiring validators to lock ETH. These validators are responsible for proposing new blocks, validating existing ones, and processing transactions. The protocol enforces rewards and penalties to ensure validator integrity and deter malicious activity. +The main challenge a consensus protocol aims to solve is building a reliable distributed system on top of unreliable infrastructure. Research on consensus protocols dates back to the 1970s, but the scale of what Ethereum is trying to achieve is far more ambitious. -Validator selection, block voting, and fork resolution are governed by [consensus specification](/wiki/CL/cl-specs.md). In case of competing blocks, the "heaviest" chain, determined by validator support weighted by staked ETH, prevails. +In Ethereum's consensus layer, the goal is to ensure that tens of thousands of independent nodes around the world stay reasonably synchronized. Each node keeps a ledger with the state of every account, and all these ledgers must match exactly. There can be no differences; the nodes must agree and do so quickly. This is what we mean by "a reliable distributed system." -Ethereum consensus layer addresses the fundamental challenge of Byzantine fault tolerance in distributed computing. Similar to the Byzantine Generals Problem, geographically dispersed nodes in the blockchain network must agree on transaction validity despite potential communication issues or malicious actors. +These nodes often use [consumer grade hardware](https://stakefromhome.com/) and communicate over internet connections that may be slow, lose data, or disconnect unexpectedly. Node operators might misconfigure their software or fail to update it. Additionally, there could be many bad actors running rogue nodes or tampering with communications for personal gain. This is what we mean by "unreliable infrastructure." -Ethereum uses a consensus mechanism known as Gasper that combines [Casper proof-of-stake](https://arxiv.org/pdf/1710.09437.pdf) with LMD GHOST - an extension of [GHOST fork-choice rule](https://eprint.iacr.org/2013/881.pdf). +### Byzantine Fault Tolerance (BFT) and Byzantine Generals' Problem -## Resources +Byzantine Fault Tolerance (BFT) is a property of distributed systems that allows them to function correctly even when some components fail or act maliciously. BFT is crucial in decentralized networks, where trust among nodes cannot be assumed. In other words, a system exhibiting BFT can tolerate Byzantine faults, which are arbitrary failures that include malicious behavior. For a system to be Byzantine fault-tolerant, it must reach consensus despite these faults. For more on the problem and practical solutions read [this](https://hackmd.io/@kira50/SknuPZMIC). -- Vitalik Buterin et al., [Gasper: Combining GHOST and Casper.](https://arxiv.org/pdf/2003.03052.pdf) -- Alt Explainer, ["Ethereum's Proof of Stake consensus explained."](https://www.youtube.com/watch?v=5gfNUVmX3Es) -- Vitalik Buterin, ["Parametrizing Casper: the decentralization/finality time/overhead tradeoff"](https://medium.com/@VitalikButerin/parametrizing-casper-the-decentralization-finality-time-overhead-tradeoff-3f2011672735) -- Ethereum, ["Eth2: Annotated Spec"](https://github.com/ethereum/annotated-spec) -- Martin Kleppmann, [Distributed Systems.](https://www.youtube.com/playlist?list=PLeKd45zvjcDFUEv_ohr_HdUFe97RItdiB) -- Leslie Lamport et al., [The Byzantine Generals Problem.](https://lamport.azurewebsites.net/pubs/byz.pdf) -- Austin Griffith, [Byzantine Generals - ETH.BUILD.](https://www.youtube.com/watch?v=c7yvOlwBPoQ) -- Michael Sproul, ["Inside Ethereum"](https://www.youtube.com/watch?v=LviEOQD9e8c) +## Introduction to Consensus Layer (CL) + +> Consensus is a way to build reliable distributed systems with unreliable components. Blockchain-based distributed systems aim to agree on a single history of transactions. +> +> Proof-of-work and Proof-of-stake are not consensus protocols, but enable consensus protocols. In Ethereum, Nodes and validators are the actors of the consensus system. Slots and epochs regulate consensus time. Blocks and attestations are the currency of consensus. + +The Consensus Layer (CL) is a fundamental component that ensures the network's security, reliability, and efficiency. Originally, Ethereum utilized Proof-of-work (PoW) as its consensus mechanism, similar to Bitcoin. PoW, while effective in maintaining decentralization and security, has significant drawbacks, including high energy consumption and limited scalability. To address these issues, Ethereum has transitioned to Proof-of-Stake (PoS), a more sustainable and scalable consensus mechanism. + +The Ethereum network consists of many individual nodes. Each node operates independently and communicates over the Internet, which is often unreliable and asynchronous. + +Users send transactions to this network of nodes, and the consensus protocol ensures that all honest nodes eventually agree on a single, consistent transaction history. This means they agree on the order of transactions and their outcomes. For example, if I have 1 ETH and tell the network to send it to both Alice and Bob at the same time, the network must eventually agree on whether I sent it to Alice or Bob. It would be a failure if both Alice and Bob received the Ether, or if neither did. A consensus protocol is the process that enables this agreement on transaction order. + +Ethereum's consensus protocol actually _bolts together_ two different consensus protocols. One is called [LMD GHOST](/wiki/CL/gasper.md?lmd-ghost), the other [Casper FFG](/wiki/CL/gasper.md?casper-ffg). The combination has become known as [Gasper](/wiki/CL/gasper.md). In subsequent sections we will be looking at these both separately and in combination. + +## Proof-of-work and Proof-of-stake + +This is a good point to clarify that neither Proof-of-work (PoW) nor Proof-of-Stake (PoS) are consensus protocols by themselves. They are often incorrectly referred to as such, but they are actually mechanisms that enable consensus protocols. Both PoW and PoS primarily serve as Sybil resistance mechanisms, placing a cost on participating in the protocol. This prevents attackers from overwhelming the system cheaply. + +However, both PoW and PoS are closely linked to the consensus mechanisms they support through [fork choice](/wiki/CL/cl-architecture.md?id=fork-choice-rules) rules. They help assign a weight or score to a chain of blocks: in PoW, it's the total computational work done; in PoS, it's the total value staked that supports a particular chain. Beyond these basics, PoW and PoS can support various consensus protocols, each with its own dynamics and trade-offs. + +### Blockchains + +The basic element of a blockchain is the block. A block contains a set of transactions assembled by a leader (block proposer). The contents of a block can vary based on the protocol. + +- The payload of a block on Ethereum's execution chain is a list of user transactions. +- In the pre-Merge PoS Beacon Chain, a block's payload was mostly a set of attestations by validators. +- Post-Merge Beacon Chain blocks also include the execution payload (user transactions). +- After [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844)(Deneb upgrade), blocks now also contain commitments to opaque data blobs alongside user transactions. + +Except for the Genesis block, each block builds on and points to a parent block, forming a chain of blocks: a blockchain. The goal is for all nodes on the network to agree on the same canonical blockchain history. + + + +
+ +![Blockchain](../../images/cl/blockchain.svg) + +
+ +_Time moves from left to right and, except for the Genesis block, each block points to the parent block it builds on._ + +
+
+ +The chain grows as nodes add new blocks to its tip. This is done by temporarily selecting a "leader", the node that extends the chain. In PoW, the leader is the miner who first solves the PoW puzzle for its block. In Ethereum's PoS, the proposer(leader) is pseudo-randomly selected from active validator set. + +The leader (block proposer) adds a block to the chain, choosing and ordering its contents. The block must be valid according to protocol rules, or the network will ignore it. Using blocks is an optimization. Adding individual transactions one by one would create a huge consensus overhead. So blocks are batches of transactions, In Ethereum's execution chain, block size is limited by the block gas limit (the amount of work needed to process the transactions). [Beacon block](/wiki/CL/beacon-api.md?id=beaconblockbody) sizes are limited by hard-coded constants. + +## Transition to Proof-of-Stake + +> The engine was changed mid-flight! September 15, 2022 — the day Ethereum switched to Proof-of-Stake. That new engine is the Consensus Layer, formerly known as Ethereum 2.0’s Beacon Chain. + +The Paris hard fork (The merge) in Ethereum was activated based on "terminal total difficulty" (TTD) instead of block height to avoid risks like malicious forks. This ensures the transition to Proof-of-Stake (PoS) occurs only when the cumulative difficulty reaches a critical threshold. The terminal block is the last Proof-of-work (PoW) block where its total difficulty surpasses a predefined threshold, ensuring security. The total difficulty is calculated recursively, reflecting the computational effort in the blockchain. + +This is relevant because testnets, devnets and any Ethereum network running the latest software needs to activate the Merge - not by block height but by Total Terminal Difficulty (TTD). More details on the [transition criteria](https://hackmd.io/@kira50/rJ2y7jImR) and [Total Terminal Difficulty](https://bordel.wtf). + +The merge introduces the following: + +- **Beacon Chain Takes Over**: The Beacon Chain, already running in parallel with the Ethereum mainnet, assumes the responsibility for processing new blocks. Under PoS, blocks are validated by validators who stake their ETH to participate in the consensus mechanism, rather than by miners solving cryptographic puzzles. + +- **Security and Efficiency**: This transition not only aims to enhance the security of the Ethereum network by making it more decentralized but also significantly reduces its energy consumption, addressing one of the major criticisms of traditional PoW systems. + +- **New Consensus Mechanism**: The consensus under PoS is achieved through a combination of staking, attestation by validators, and algorithms that randomly select block proposers and committees to ensure the network remains secure and transactions are processed efficiently. + +### Beacon Chain Introduction +The Beacon Chain plays a crucial role in managing the PoS consensus. It oversees validators who propose and attest to new blocks, ensuring the network’s integrity and security. Validators are selected based on a number of criteria, one of them being the amount of ETH they stake, which also acts as collateral against dishonest behavior. Some high level responsibilities of validators are: + +- **Staking ETH**: Validators must stake a minimum of 32 ETH to participate. +- **Proposing Blocks**: A Validator is randomly selected to propose a new block. They must construct valid blocks and broadcast them to the network +- **Attesting Blocks**: Validators attest to the validity of blocks proposed by others. Attestations are essentially votes on the validity of the blocks, ensuring consensus. +- **Participating in Consensus**: Validators participate in consensus by voting on the state of the blockchain at regular intervals, helping to finalize the blockchain's state. + +The Paris hard fork was a pivotal event in Ethereum's history, setting the stage for more scalable, sustainable, and secure operations. It represents Ethereum's commitment to innovation and its responsiveness to the broader societal concerns about the environmental impact of cryptocurrency mining. + +## Beacon Chain and its Preliminaries + +The Beacon Chain is the backbone of Ethereum’s consensus. It coordinates validators, manages the PoS protocol, and ensures consensus across the network. This section with cover the anatomy of Beacon chain. + +### Validators + +Validators are essentially the participants in the PoS Protocol. They propose and validate new blocks, ensuring the integrity and security of the blockchain. Validators must stake ETH as collateral, aligning their interests with the network’s health. Validators are chosen to propose blocks based on several factors: + +- **Staked Ether**: Each validator can stake a maximum of 32 ETH. Stakers with more ETH can increase their influence by running multiple validator nodes, each staking 32 ETH. This system ensures decentralization and aligns the interests of validators with the network's security and integrity. +- **Randomness**: The selection process incorporates cryptographic randomness to prevent predictability and manipulation. This is achieved through the [RANDAO](https://inevitableeth.com/home/ethereum/network/consensus/randao) and [VDF (Verifiable Delay Function)](https://inevitableeth.com/home/ethereum/upgrades/consensus-updates/vdf) mechanisms. +- **Committees**: Validators are grouped into committees for block proposal and attestation. Each committee is responsible for validating and attesting to blocks, ensuring a decentralized and secure validation process. +- **Staking Requirements**: To become a validator, an individual must deposit a minimum of 32 ETH into the official deposit contract. This ETH acts as collateral to incentivize honest behavior. The validator's ETH is at risk if they fail to perform their duties or engage in malicious activities. + +### Slots and Epochs + +Each slot is 12 seconds and an epoch is 32 slots: 384 seconds or 6.4 minutes. Each slot has a validator assigned to propose a block, while committees of validators attest to the block’s validity. + +A slot is a chance for a block to be added to the Beacon Chain. Every 12 seconds, one block is added. Validators need to be roughly [synchronized with time](https://ethresear.ch/t/network-adjusted-timestamps/4187). A slot is like the block time, but slots can be empty. The Beacon Chain genesis block is at Slot 0. + + + + +
+ +![Diagram for slots and epoch](../../images/cl/slots-and-epochs.png) + +
+ +_The first 32 slots are in Epoch 0. Genesis block is at Slot 0._ + +
+
+ + +### Validators and Attestations + +A block proposer is a validator that has been pseudo-randomly selected to build a block. Validators propose blocks and attest to the blocks proposed by others. Most of the time, validators are attesters that vote on blocks. These votes are recorded in the Beacon Chain and determine the head of the Beacon Chain. + +Attestations are votes on the validity of the blocks, which are aggregated into the Beacon Chain to ensure consensus. + + + +
+ +![Diagram for Validator selection](../../images/cl/validators.png) + +
+ +_A slot can be missed as you can see in this diagram on 28th slot_ + +
+
+ +An **attestation** is a validator’s vote, weighted by the validator’s stake. Attestations are broadcasted by validators in addition to blocks. Validators also police each other and are rewarded for reporting other validators that make conflicting votes, or propose multiple blocks. + +The contents of the Beacon Chain is primarily a registry of validator addresses, the state of each validator, and attestations. Validators are activated by the Beacon Chain and can transition to states + +**IMPORTANT NOTE on Staking Validators Semantics:** *In Ethereum's PoS, users activate validators by staking ETH, similar to buying hardware in PoW. Stakers are associated with the amount staked, while validators have a maximum balance of 32 ETH each. For every 32 ETH staked, one validator is activated. Validators are run by validator clients, which use a beacon node to follow and read the Beacon Chain. A single validator client can manage multiple validators.* + +### Committees + +Committees are groups of at least 128 validators assigned to each slot for added security. An attacker has less than a 1 in a trillion chance of controlling ⅔ of a committee. + +The concept of a randomness beacon that emits random numbers for the public, is how Beacon Chain got it's name. The Beacon Chain enforces consensus on a pseudorandom process called RANDAO. + + +
+ +![Diagram for Validator selection](../../images/cl/RANDAO.png) + +
+ +_At every epoch, a pseudorandom process RANDAO selects proposers for each slot, and shuffles validators to committees._ + +
+
+ +**Validator Selection:** +- As mentioned earlier, Proposers are chosen by RANDAO, weighted by validator balance. +- A validator can be both a proposer and a committee member for the same slot, but this is rare (1/32 probability). + +The sketch depicts a scenario with less than 8,192 validators, otherwise there would be at least two committees per slot. + + + +
+ +![Diagram for Committees](../../images/cl/committees.png) + +
+ +The diagram is a combined depiction of what happened in 3 slots: +- Slot 1: A block is proposed and attested by two validators; one validator is offline. +- Slot 2: A block is proposed, but one validator misses it and attests to the previous block. +- Slot 3: All validators in Committee C attest to the same head, following the LMD GHOST rule. + +Validators attest to their view of the Beacon Chain head using the LMD GHOST rule. +Attestations help finalize blocks by reaching consensus on the blockchain’s state. + +**Committee Size and Security:** +- With more than 8,192 validators, multiple committees per slot are formed. +- Committees must be at least 128 validators for optimal security. +- Security decreases with fewer than 4,096 validators, as committee sizes drop below 128. + +> At every epoch, validators are evenly divided across slots and then subdivided into committees of appropriate size. All of the validators from that slot attest to the Beacon Chain head. A shuffling algorithm scales up or down the number of committees per slot to get at least 128 validators per committee. More details on shuffling can be found in [proto's repo.](https://github.com/protolambda/eth2-docs#shuffling) + +### Blobs + +[EIP-4844](https://eips.ethereum.org/EIPS/eip-4844), also known as proto-danksharding, is part of the Deneb/Cancun hardfork. It introduces a data availability layer to Ethereum, allowing for the temporary storage of arbitrary data on the blockchain. This arbitrary data stored this way are called `blobs`, and each block can have 3 ~ 6 blob sidecars (wrappers for blobs). EIP-4844 marks Ethereum's first step towards sharding and scalability, enabling Layer 2 solutions (L2s) to use this data availability layer to lower gas fees and process more transactions. + +### Design and Implementation + +A key design decision in EIP-4844 is the use of [KZG commitments](/wiki/Cryptography/kzg.md) to verify blobs and support future proposer-builder separation. To use KZG commitments, a Trusted Setup is needed. For the Deneb hardfork, a [KZG Ceremony](https://github.com/ethereum/kzg-ceremony) was conducted to create this Trusted Setup. + + + +
+ +![Diagram for Blobs](../../images/cl/blobs.png) + +
+ +### Storage Requirements + +The most significant impact on node operators is the increased storage requirement. Node runners will need more storage: + +``` +131,928 ssz bytes per blob * 4096 blobs retention period * +32 potential blocks per epoch * 3~6 blob sidecars per block + += 52~104GB +``` + +By default, these blobs will be retained for 4096 epochs, and clients would prune the oldest blobs once the retention period is reached. + +### Checkpoints and Finality + +At the end of each epoch, checkpoints are created. A checkpoint is a block in the first slot of an epoch. If there is no such block, then the checkpoint is the preceding most recent block. There is always one checkpoint block per epoch. A block can be the checkpoint for multiple epochs. + +A block becomes a checkpoint if it receives attestations from a majority of validators. Checkpoints are used to finalize the blockchain's state. A block is considered final when it is included in two-thirds of the most recent checkpoint attestations, ensuring it cannot be reverted. + + +
+ +![Diagram for checkpoints](../../images/cl/checkpoints.jpg) + +
+ +_Checkpoints for a scenario where epoch contain 64 slots_ + +
+
+ +For example, if Slots 65 to 128 are empty, the Epoch 2 checkpoint defaults to the block at Slot 64. Similarly, if Slot 192 is empty, the Epoch 3 checkpoint is the block at Slot 180. **Epoch boundary blocks (EBB)** is a term in some literature (such as the [Gasper](https://arxiv.org/abs/2003.03052) paper, the source of the diagram above and a later one), and they can be considered synonymous with checkpoints. + +Validators cast two types of votes: **LMD GHOST** votes for blocks and **Casper FFG** votes for checkpoints. An **FFG** vote includes a source checkpoint from a previous epoch and a target checkpoint from the current epoch. For example, a validator in Epoch 1 might vote for a source checkpoint at the genesis block and a target checkpoint at Slot 64, repeating the same vote in Epoch 2. Only Validators assigned to a slot cast LMD GHOST votes, while all validators cast FFG votes for epoch checkpoints. + +#### Supermajority and Finality + +A supermajority, defined as ⅔ of the total validator balance, is required for a checkpoint to be justified. For instance, if validators have balances of 8 ETH, 8 ETH, and 32 ETH, a supermajority needs the vote of the 32 ETH validator. Once a checkpoint receives a supermajority, it becomes justified. If the subsequent epoch's checkpoint also achieves justification, the previous checkpoint is finalized, securing all preceding blocks. Typically, this process spans two epochs (12.8 minutes). + +When a user transaction is included in a block, On average it would be somewhere in the middle of an epoch. It takes half an epoch (about 3.2 minutes) to reach the next checkpoint, suggesting transaction finality of 2.5 epochs: 16 minutes. Optimally, more than ⅔ of attestations will have been included by the 22nd (2/3rd of 32) slot of an epoch. Thus, transaction finality is an average of 14 minutes (16+32+22 slots). Block confirmations emerge from a block’s attestations, then move to its justification, to its finality. Use cases can decide whether they need finality or an earlier safety threshold is sufficient. + + +
+ +![Diagram for Finality](../../images/cl/finalization.png) + +
+ +_Example of one checkpoint getting justified (Slot 64) and finalizing a prior checkpoint (Slot 32)._ + +
+
+ +**What happened at the Beacon Chain head:** +At Slot 96, a block is proposed that includes attestations (votes) for the Epoch 2 checkpoint. These attestations reach the required two-thirds supermajority, justifying the Epoch 2 checkpoint. This action finalizes the previously justified Epoch 1 checkpoint. When the Epoch 1 checkpoint is finalized, all preceding blocks (up to Slot 32) also become final. Finality calculations happen at epoch boundaries, but attestations accumulate with each block. + +**What could have happened from genesis to the head:** +- **Scenario 1:** + - Proposers from Slot 1 to Slot 63 propose blocks. + - Each block in Epoch 1 contributes attestations for the checkpoint at Slot 32, eventually reaching 55%. + - The block at Slot 64 includes additional attestations, bringing support for the Slot 32 checkpoint to 70%, causing its justification. + - Throughout Epoch 2, the Slot 64 checkpoint gathers attestations but doesn't reach the two-thirds threshold until Slot 96, where it is justified. + - Justifying the Epoch 2 checkpoint finalizes the Epoch 1 checkpoint and all preceding blocks. + +- **Scenario 2:** + - The checkpoint at Epoch 1 could reach the two-thirds supermajority before the next epoch. + - For example, blocks from Slot 32 to Slot 54 could provide enough attestations to justify the checkpoint at Slot 32. + - In this case, the Slot 32 checkpoint would be justified within its current epoch but would need the next epoch to finalize. + +**Special Cases:** +The justification of a checkpoint can sometimes finalize blocks from two or more epochs ago, especially during periods of high latency, network partitions, or attacks, You can find more such cases, discussed in the Gasper paper. These scenarios are exceptional and not the norm. + + +#### Closer Look on Attestations + +Validators submit one attestation per epoch, containing both an LMD GHOST and an FFG vote. These attestations have 32 chances per epoch for inclusion on-chain, with earlier inclusions receiving higher rewards. This means a validator may have two attestations included on-chain in a single epoch. Validators are rewarded the most when their attestation is included on-chain at their assigned slot; later inclusion has a decayed reward. To give validators time to prepare, they are assigned to committees one epoch in advance. Proposers are only assigned to slots once the epoch starts. Nonetheless, [secret leader election](https://ethresear.ch/t/low-overhead-secret-single-leader-election/5994) research aims to mitigate attacks or bribing of proposers. + +Consider a block proposed at Slot 64 containing attestations for the Epoch 2 checkpoint. This scenario can finalize the checkpoint at Slot 32. The finality of the Slot 32 checkpoint, once achieved, propagates backward, securing all preceding blocks. + +In essence, Committees allow for the technical optimization of combining signatures from each attester into a single aggregate signature. When validators in the same committee make the same LMD GHOST and FFG votes, their signatures can be aggregated. + +### Staking Rewards and Penalties + +Ethereum’s PoS system employs a comprehensive set of rewards and penalties to incentivize validator behavior and maintain network security. This section covers six key aspects of these incentives: + +**1. Attester Rewards:** +Validators earn rewards for making attestations (LMD GHOST and FFG votes) that align with the majority of other validators. Attestations included in finalized blocks are more valuable. + +**2. Attester Penalties:** +Validators are penalized for failing to attest or for attesting to blocks that do not get finalized. These penalties ensure validators remain active and aligned with the network’s consensus. + +**3. Typical Downside Risk for Stakers:** +Stakers can estimate their downside risk by comparing potential earnings and penalties. An honest validator earning 10% in a year could lose up to 7.5% for poor performance. Minor penalties apply for short-term inactivity, while prolonged offline periods incur larger penalties. + +**4. Slashings and Whistleblower Rewards:** +Slashing penalizes validators for serious protocol violations. Penalties range from over 0.5 ETH up to the entire stake. For example, a validator committing a slashable offense loses at least 1/32 of their balance and is deactivated. Additional penalties are proportional to the number of validators slashed simultaneously. A whistleblower who reports a slashable offense receives a reward, which currently goes to the block proposer. + +**5. Proposer Rewards:** +Block proposers receive substantial rewards for proposing blocks that get finalized. Consistently performing validators gain approximately a 1/8 boost to their total rewards. Additionally, proposers receive small rewards for including slashing evidence in their blocks. + +**6. Inactivity Leak Penalty:** +The inactivity leak is a severe penalty designed to ensure the network’s finality. If finality is delayed for more than four epochs, validators suffer increasing penalties until a checkpoint is finalized. This mechanism drains the balances of inactive validators, leading to their forced exit, thus allowing active validators to form a ⅔ majority to resume finality. During an inactivity leak, only proposer and whistleblower rewards are earned, while attester rewards are zero. + + +### **Slashable Offenses:** +There are four conditions under which a validator can be slashed: +- **Double Proposal:** Proposing more than one block for their assigned slot. +- **LMD GHOST Double Vote:** Attesting to different Beacon Chain heads for the same slot. +- **FFG Surround Vote:** Casting an FFG vote that surrounds or is surrounded by a previous FFG vote by the same validator. +- **FFG Double Vote:** Casting two FFG votes for different targets in the same epoch. + +## Beacon Chain Validator Activation and Lifecycle: + +A validator requires 32 ETH to be activated. Validators are deactivated if their balance falls to 16 ETH, with any remaining balance withdrawable. Validators can also voluntarily exit after serving 2,048 epochs (approximately nine days). + +Upon exit, there is a delay of four epochs before withdrawal, during which validators can still be slashed. + +Honest validators can withdraw their balance in about 27 hours, whereas slashed validators face a delay of approximately 36 days (8,192 epochs). + + +
+ +![Diagram for Validator Lifecycle](../../images/cl/validator-lifecycle.png) + +
+ +To prevent rapid changes in the validator set, mechanisms limit how many validators can be activated or exited per epoch. The Beacon Chain also employs effective balances for technical optimization, which change less frequently than actual validator balances. + +#### Overall Effects +At every epoch, validators are evenly divided across slots and then subdivided into committees of appropriate size. Validators can only be in one slot, and in one committee. Collectively: + +- All validators in an epoch attempt to finalize the same checkpoint: FFG vote +- All validators assigned to a slot attempt to vote on the same Beacon Chain head: LMD GHOST vote +Optimal behavior rewards validators the most. + +The Beacon Chain's introduction on December 1, 2020, began with 21,063 validators. The number of validators can decrease with slashings or voluntary exits, or more stakers can join and be activated. Fast forward to today(15th May, 2024) there are more than 1,000,000 validators that are active on Ethereum Network. The world has never seen a scalable platform for decentralized systems and applications like Ethereum. + + + + +### References + +- [Beacon Chain Explainer from ethos.dev](https://ethos.dev/beacon-chain) +- [Evolution of Ethereum Proof-of-Stake](https://github.com/ethereum/pos-evolution/blob/master/pos-evolution.md) +- Alt Explainer, [Ethereum's Proof-of-Stake consensus explained](https://www.youtube.com/watch?v=5gfNUVmX3Es) +- [Eth2 Handbook by Ben Edgington](https://eth2book.info/capella/part2/consensus/) diff --git a/docs/wiki/EL/el-architecture.md b/docs/wiki/EL/el-architecture.md index 487e2c49..e053f0a1 100644 --- a/docs/wiki/EL/el-architecture.md +++ b/docs/wiki/EL/el-architecture.md @@ -184,7 +184,7 @@ The execution layer has its own consensus engine to work with its own copy of th | $H_{difficulty} = 0 $ | validate_header-> [ensure](https://github.com/ethereum/execution-specs/blob/9fc7925c80ff2f3949e1cc340a4a0d36fcd4161c/src/ethereum/cancun/fork.py#L327) | | | | | | | $H_{nonce} = 0x0000000000000000 $ | validate_header-> [ensure](https://github.com/ethereum/execution-specs/blob/9fc7925c80ff2f3949e1cc340a4a0d36fcd4161c/src/ethereum/cancun/fork.py#L328) | | | | | | | $H_{prevRandao} = PREVRANDAO() $ (this is stale , beacon chain provides this now) | | | | | | | -| $H_{withdrawlsHash} \neq nil $ | | | | | | | +| $H_{withdrawalHash} \neq nil $ | | | | | | | | $H_{blobGasUsed} \neq nil $ | | | | | | | | $H_{blobGasUsed} \leq MaxBlobGasPerBlock_{=786432} $ | | | | | | | | $H_{blobGasUsed} \% GasPerBlob_{=2^{17}} = 0 $ | | | | | | | diff --git a/docs/wiki/EL/el-specs.md b/docs/wiki/EL/el-specs.md index cdfd683b..5fc8ac84 100644 --- a/docs/wiki/EL/el-specs.md +++ b/docs/wiki/EL/el-specs.md @@ -95,7 +95,7 @@ $$H_{difficulty} = 0\qquad (57k)$$ $$\land $$ $$H_{nonce} = 0x0000000000000000 \qquad (57l)$$ $$\land$$ -$$H_{withdrawlsHash} \neq nil \qquad (57n)$$ +$$H_{withdrawalHash} \neq nil \qquad (57n)$$ $$\land$$ $$H_{blobGasUsed} \neq nil \qquad (57o)$$ $$\land$$ diff --git a/docs/wiki/protocol/design-rationale.md b/docs/wiki/protocol/design-rationale.md index da310377..dce685ba 100644 --- a/docs/wiki/protocol/design-rationale.md +++ b/docs/wiki/protocol/design-rationale.md @@ -96,7 +96,7 @@ The [Casper FFG](https://arxiv.org/abs/1710.09437v4) is an overlay atop a propos Simply put, each validator votes on the checkpoint, and after two rounds of voting, the checkpoint is **finalized**. All finalized checkpoints become part of the canonical chain (part of the blockchain history). While Casper guarantees **finality** through attestations to the latest block addition to the canonical chain, it requires a fork-choice rule where validators attest to blocks to signal support for those blocks. - ***LMD GHOST*** -Latest Message Driven Greediest Heaviest Observed Sub-Tree (LMD-GHOST) is a *fork choice rule* where validators attests to blocks to signal support for those blocks. This similar in some ways to the fork choice rule used in Proof-of-Work network, where the fork with the most work done is selected as the canonical chain. +Latest Message Driven Greediest Heaviest Observed Sub-Tree (LMD-GHOST) is a *fork choice rule* where validators attests to blocks to signal support for those blocks. This similar in some ways to the fork choice rule used in Proof-of-work network, where the fork with the most work done is selected as the canonical chain. ![LMD-GHOST-Algorithm](./img/lmt-ghost.png) diff --git a/wordlist.txt b/wordlist.txt index 8e7657a6..649dbaf9 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -16,6 +16,7 @@ aggregative Aleth allowfullscreen amidst +amongst Andreas Antonopoulos API @@ -50,6 +51,7 @@ Barnabe Barnabé Barnabé's BDN +beaconblock Beiko Bertoni Besu @@ -173,12 +175,14 @@ decrementation decrementing DeFi Degatchi +dht DELEGATECALL delegator delegators deliverables Dencun deployer +descendent deserialization Deserialize deserialized @@ -238,6 +242,7 @@ ELs emptyset Encodings Endian +enr env EOA EOAs @@ -266,6 +271,7 @@ ethone's ethresear ethresearch ethroadmap +ETHW EVM evmlab EVMONE @@ -276,6 +282,7 @@ exchangeTransitionConfigurationV Explainer Extractable extraData +fanout farcaster fastssz faytey @@ -327,6 +334,7 @@ Goldwasser Goomy Goron Gorondan +gossipsub Gottfried Gottlob gpg @@ -341,6 +349,7 @@ hackathon hackmd Hager halmos +hardfork HashedStorages Herc’s hevm @@ -382,6 +391,7 @@ interop invariants IOP IPC +ipfs IRTF ISA Jitsi @@ -390,6 +400,7 @@ JSON JUMPDEST JVM JWT +Kademlia Karapetsas Katex KEC @@ -411,6 +422,7 @@ Lamport's lceil ldots Lefteris +leftarrow leq leveldb lfloor @@ -436,6 +448,7 @@ LSP LST Lua LuaVM +Luca Lyubashevsky mainnet Mana @@ -461,6 +474,7 @@ mevboost Michaël middleware minimalistic +misconfigure Mitigations mload MMPTs @@ -534,6 +548,7 @@ pepc's performence permissionless permissionlessness +pex PGA Pigovian Pilipovic @@ -543,6 +558,7 @@ pmod POC POS POSIX +postel's possibile Potuz's POV @@ -573,6 +589,7 @@ privateKey probabilistically programmability proto +proto's protobuf Protolambda prover @@ -583,6 +600,8 @@ Prysm Prysmatic PSE PSE's +pseudorandom +pseudorandomly ptc publically pubsub @@ -605,6 +624,8 @@ README reciept referrerpolicy remerkleable +reorganisation +reorganisations replayable repo responder @@ -619,7 +640,9 @@ Rikard RIPEMD Ritchie rK +RLMD RLP +RLPx roadmap rollup rollup's @@ -674,6 +697,7 @@ solvm SPHINCS Sproul src +ssd SSF SSLE SSTORE @@ -701,11 +725,13 @@ substate Substate subtrees Summa +Supermajority systemd Szabo Takenobu Tani tbhl +TCP Teku testnet testnets @@ -746,8 +772,11 @@ unaggregated unbundle Unbundling underbrace +underflow +underflows Unformatted unguessable +unhandled unincentivized upstreamed utils @@ -761,6 +790,7 @@ validators Vanstone Varun VB's +VDF VDFs Vec verifications @@ -779,9 +809,11 @@ walkthrough WB webkit WebRTC +Whistleblower Whitepaper WIP -withdrawlsHash +withdrawable +withdrawalHash WSS Xatu xff @@ -792,7 +824,10 @@ Yan Yellowpaper Yoni Yoichi +Zanolini Zaverucha +Zhang +Zipfian zk zkEVMs ZKSNARK