Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consensus Layer Wiki Page #246

Merged
merged 63 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
a0ae153
CL init, Update overview of CL
shyam-patel-kira May 2, 2024
dd773ff
Update ordering
shyam-patel-kira May 2, 2024
d10c76e
Update to current changes
shyam-patel-kira May 14, 2024
ae4124c
Add validators section to overview
shyam-patel-kira May 14, 2024
e82b698
add iamges; added Beacon chain explainer; checkpoints and finality; s…
shyam-patel-kira May 15, 2024
1fef51a
Added validator life cylce
shyam-patel-kira May 15, 2024
abed432
fix typo; update wordlist
shyam-patel-kira May 15, 2024
17656e0
Update state of validators
shyam-patel-kira May 17, 2024
55e6a24
Improve flow of the page;added simpler explanations; added some links
shyam-patel-kira May 18, 2024
b4fd8c7
add introduction; minor fixes
shyam-patel-kira May 19, 2024
e044716
Minor typos ffix
shyam-patel-kira May 19, 2024
93e3671
add introduction; added byzantine generals problem
shyam-patel-kira May 19, 2024
b0f53d7
revamp overview structure
shyam-patel-kira May 20, 2024
af6f604
fix dark background in svg
shyam-patel-kira May 20, 2024
bec30b3
complete overview of CL; added cl-architecture structure
shyam-patel-kira May 20, 2024
8a6c63d
Added Blocktree and fork-choice rules
shyam-patel-kira May 21, 2024
ad71957
Merge branch 'main' of https://github.com/eth-protocol-fellows/protoc…
shyam-patel-kira May 21, 2024
9705975
fix some typos; update wordlist
shyam-patel-kira May 21, 2024
68087f0
Merge branch 'main' of https://github.com/eth-protocol-fellows/protoc…
shyam-patel-kira May 21, 2024
11dbc63
add reorgs and reversion
shyam-patel-kira May 21, 2024
107038f
Add liveness and safey comparision
shyam-patel-kira May 21, 2024
2750896
Add some more details on consensus protocol
shyam-patel-kira May 23, 2024
89d042d
Add architecture and blobs
shyam-patel-kira May 23, 2024
d516ade
stf; control flow
shyam-patel-kira May 23, 2024
9c34f94
fix a broken link; added gasper file
shyam-patel-kira May 26, 2024
51cbbbe
Use consistent naming for PoW and PoS
shyam-patel-kira May 26, 2024
5d62c91
Complete cl-architecture
shyam-patel-kira May 30, 2024
1e1c000
Update structure of cl-networking
shyam-patel-kira Jun 19, 2024
8488571
fix spelling mistake
shyam-patel-kira Jun 19, 2024
9fe39bb
fix typos; added words to wordlist
shyam-patel-kira Jun 19, 2024
a0477c2
remove whitespace
shyam-patel-kira Jun 19, 2024
36ad561
address some nits
shyam-patel-kira Jun 20, 2024
2f86faf
Omit some redudant content
shyam-patel-kira Jun 20, 2024
32dd223
remove redundant content; fix broken links
shyam-patel-kira Jun 20, 2024
e8d87ec
Update proposer and validator set wording
shyam-patel-kira Jun 20, 2024
e4eabfd
Address some more nits
shyam-patel-kira Jun 20, 2024
2fad7dd
Merge branch 'feat/consensus-layer' of https://github.com/eth-protoco…
shyam-patel-kira Jun 20, 2024
e67c1e1
Add resources; omit whitespace
shyam-patel-kira Jun 20, 2024
d6b78d8
fix typos
shyam-patel-kira Jun 20, 2024
528b7e2
Merge branch 'main' into feat/consensus-layer
shyam-patel-kira Jun 20, 2024
42f3617
Merge branch 'main' into feat/consensus-layer
shyam-patel-kira Jun 28, 2024
b664169
Update wordlist
shyam-patel-kira Jul 4, 2024
9d34f18
nit: grammar
shyam-patel-kira Jul 24, 2024
ae6fed1
nit: space
shyam-patel-kira Jul 24, 2024
502402b
nit: word
shyam-patel-kira Jul 24, 2024
bcd3adc
nit: word
shyam-patel-kira Jul 24, 2024
fa8de57
clean up
shyam-patel-kira Jul 24, 2024
45d4a3d
nit: spell
shyam-patel-kira Jul 24, 2024
8669733
Merge branch 'main' of https://github.com/eth-protocol-fellows/protoc…
shyam-patel-kira Jul 24, 2024
bec3641
nit: content captilization
shyam-patel-kira Jul 24, 2024
a661f2a
Merge branch 'feat/consensus-layer' of https://github.com/eth-protoco…
shyam-patel-kira Jul 24, 2024
3110ef2
Update wordlist
shyam-patel-kira Jul 24, 2024
a9bd4ad
nit: diagram name
shyam-patel-kira Jul 25, 2024
fd996f4
nit: word
shyam-patel-kira Jul 25, 2024
a671998
nit: clean up
shyam-patel-kira Jul 25, 2024
3b5d0bc
nit: clean up
shyam-patel-kira Jul 25, 2024
7bc5acc
nit: title
shyam-patel-kira Jul 25, 2024
8973b79
nit: wording
shyam-patel-kira Jul 25, 2024
0eb994c
nit: title
shyam-patel-kira Jul 25, 2024
b25a252
fix flow for the transition
shyam-patel-kira Jul 26, 2024
67061b5
Merge branch 'feat/consensus-layer' of https://github.com/eth-protoco…
shyam-patel-kira Jul 26, 2024
c941698
Merge branch 'main' of https://github.com/eth-protocol-fellows/protoc…
shyam-patel-kira Jul 26, 2024
bdf947d
Update beacon-api.md
taxmeifyoucan Jul 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/_sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@
- [RLP Serialization](/wiki/EL/RLP.md)
- Consensus Layer
- [Overview](/wiki/CL/overview.md)
- [Client architecture](/wiki/CL/cl-architecture.md)
- [CL Networking](/wiki/CL/cl-networking.md)
- [CL Specs](/wiki/CL/cl-specs.md)
- [Client architecture](/wiki/CL/client-architecture.md)
- [CL Clients](/wiki/CL/cl-clients.md)
- [Beacon API](/wiki/CL/beacon-api.md)
- [CL Networking](/wiki/CL/cl-networking.md)
- [CL Clients](/wiki/CL/cl-clients.md)
- [SSZ Serialization](/wiki/CL/SSZ.md)
- [Merkleization](/wiki/CL/merkleization.md)
- Development
Expand Down
Binary file added docs/images/cl/RANDAO.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/blobs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
491 changes: 491 additions & 0 deletions docs/images/cl/blockchain.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
657 changes: 657 additions & 0 deletions docs/images/cl/blocktree.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/checkpoints.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/cl-ideal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/cl-networking-overview.webp
Binary file not shown.
Binary file added docs/images/cl/cl-real.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/cl.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/client-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/committees.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/el.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/finalization.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/network.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
478 changes: 478 additions & 0 deletions docs/images/cl/reorg-0.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
518 changes: 518 additions & 0 deletions docs/images/cl/reorg-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/slots-and-epochs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/validator-lifecycle.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cl/validators.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
61 changes: 57 additions & 4 deletions docs/wiki/CL/beacon-api.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,60 @@
# Beacon API
# Beacon Chain Spec and APIs

> :warning: This article is a [stub](https://en.wikipedia.org/wiki/Wikipedia:Stub), help the wiki by [contributing](/contributing.md) and expanding it.
At the core of Ethereum Proof-of-Stake is a system chain called the "Beacon Chain". The beacon chain stores and manages the registry of validators. In the initial deployment phases of Proof-of-Stake, the only mechanism to become a validator is to make a one-way(withdrawable after Capella) ETH transaction to a deposit contract on the Ethereum proof-of-work chain. Activation as a validator happens when deposit receipts are processed by the Beacon Chain, the activation balance is reached, and a queuing process is completed. Exit is either voluntary or done forcibly as a penalty for misbehavior. The primary source of load on the beacon chain is "attestations". Attestations are simultaneously availability votes for a shard block (in a later upgrade) and proof-of-stake votes for a beacon block (Phase 0).

Beacon API is the endpoint provided by consensus layer clients. It's the interface for interacting with consensus for users and validators.
This section will cover some important parts of Beacon Chain Spec and APIs. Also checkout complete [Beacon Chain Spec](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md) and [Beacon API reference](https://ethereum.github.io/beacon-APIs/#/).

Check the [Beacon API reference](https://ethereum.github.io/beacon-APIs/#/).
Beacon API is the REST endpoint provided by consensus layer clients (beacon nodes). It's the interface to read information about the consensus and used by validator clients.

## Containers

`BeaconState`

```python
class BeaconState(Container):
# Versioning
genesis_time: uint64
genesis_validators_root: Root
slot: Slot
fork: Fork
# History
latest_block_header: BeaconBlockHeader
block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT]
# Eth1
eth1_data: Eth1Data
eth1_deposit_index: uint64
application_state_root: Bytes32
# Registry
validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
# Randomness
randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]
# Slashings
slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR]
# Attestations
previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
# Finality
justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch
previous_justified_checkpoint: Checkpoint # Previous epoch snapshot
current_justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
```

#### `BeaconBlockBody`

```python
class BeaconBlockBody(Container):
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Operations
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
application_payload: ApplicationPayload
```
246 changes: 246 additions & 0 deletions docs/wiki/CL/cl-architecture.md

Large diffs are not rendered by default.

38 changes: 13 additions & 25 deletions docs/wiki/CL/cl-networking.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,33 @@
# Networking

The Consensus clients use [libp2p][libp2p] as the peer-to-peer protocol,
[discv5][discv5] for peer discovery, [libp2p-noise][libp2p-noise] for
encryption, [SSZ][ssz] for encoding, and, optionally, [Snappy][snappy] for
compression.
The Consensus clients use [libp2p][libp2p] as the peer-to-peer protocol, [discv5][discv5] for peer discovery, [libp2p-noise][libp2p-noise] for encryption, [SSZ][ssz] for encoding, and, optionally, [Snappy][snappy] for compression.

## ENR (Ethereum Node Records)

## discv5


## Specs

The [Phase 0 -- Networking][consensus-networking] page specifies the network
fundamentals, protocols, and rationale/design choices.
The [Phase 0 -- Networking][consensus-networking] page specifies the network fundamentals, protocols, and rationale/design choices.

## libp2p - P2P protocol

[libp2p][libp2p] is used as the peer-to-peer protocol.
[libp2p and Ethereum][libp2p-and-eth] is a great article for a deep-dive on the
history of libp2p, and its adoption in the Consensus clients.
[libp2p][libp2p] is used as the peer-to-peer protocol. [libp2p and Ethereum][libp2p-and-eth] is a great article for a deep-dive on the history of libp2p, and its adoption in the Consensus clients.

## libp2p-noise - Encryption

The [Noise framework][noise-framework] is not a protocol itself, but a framework
for designing key exchange protocols. The [specification][noise-specification]
is a great place to start.
The [Noise framework][noise-framework] is not a protocol itself, but a framework for designing key exchange protocols. The [specification][noise-specification] is a great place to start.

There are many [patterns][noise-patterns] which describe the key exchange
process. The pattern used in the consensus clients is [`XX`][noise-xx]
(transmit-transmit), meaning that both the initiator, and responder transmit
their public key in the initial stages of the key exchange.
There are many [patterns][noise-patterns] which describe the key exchange process. The pattern used in the consensus clients is [`XX`][noise-xx] (transmit-transmit), meaning that both the initiator, and responder transmit their public key in the initial stages of the key exchange.

## SSZ - Encoding

[Simple serialize (SSZ)][ssz] replaces the [RLP][rlp] serialization used on the
execution layer everywhere across the consensus layer except the peer discovery
protocol. SSZ is designed to be deterministic and also to Merkleize efficiently.
SSZ can be thought of as having two components: a serialization scheme and a
Merkleization scheme that is designed to work efficiently with the serialized
data structure.
[Simple serialize (SSZ)][ssz] replaces the [RLP][rlp] serialization used on the execution layer everywhere across the consensus layer except the peer discovery protocol. SSZ is designed to be deterministic and also to Merkleize efficiently. SSZ can be thought of as having two components: a serialization scheme and a Merkleization scheme that is designed to work efficiently with the serialized data structure.

## Snappy - Compression

[Snappy][snappy] is a compression scheme created by engineers at Google in 2011.
It's main design considerations prioritize compression/decompression speed,
while still having a reasonable compression ratio.
[Snappy][snappy] is a compression scheme created by engineers at Google in 2011. It's main design considerations prioritize compression/decompression speed, while still having a reasonable compression ratio.

## Related R&D

Expand All @@ -67,3 +54,4 @@ while still having a reasonable compression ratio.
[rlp]: https://ethereum.org/developers/docs/data-structures-and-encoding/rlp
[snappy]: https://en.wikipedia.org/wiki/Snappy_(compression)
[ssz]: https://ethereum.org/developers/docs/data-structures-and-encoding/ssz
[blog]: https://medium.com/coinmonks/dissecting-the-ethereum-networking-stack-node-discovery-4b3f7895f83f
10 changes: 0 additions & 10 deletions docs/wiki/CL/client-architecture.md

This file was deleted.

40 changes: 40 additions & 0 deletions docs/wiki/CL/gasper.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Gasper
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is WIP? I would remove this from the PR if so or at least added some resources

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is not complete but wanted to get the structure out there, I've added some resources for this


## LMD GHOST (Latest Message Driven -- Greediest Heaviest Observed SubTree)

<!--
- What is lmd-ghost
- How subtree is selected
- fork-choice
- protocol
-->

## Casper FFG (Friendly Finality Gadget)

<!--
- justified chain
- checkpoints and finality
- gst, gat, synchronised
-->


- Hybrid Fork-choice (Refer pos-evolution in ethereum post/book)
Possible attacks
- simple ex-ante reorg
- weighted proposer boost

Solutions:
- view-merge strategy


## Resources

- [Combining GHOST and Casper](https://arxiv.org/pdf/2003.03052)
- [Yang X Zhang- Combining GHOST and Casper](https://www.youtube.com/watch?v=V0RjGmFE35U)
- [Introduction to Gasper from Ethereum.org](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/gasper/)
- [Evolution of Ethereum PoS Consensus Protocol](https://github.com/ethereum/pos-evolution/blob/master/pos-evolution.md)
- [Goldfish](https://arxiv.org/pdf/2209.03255)
- [Casper FFG](https://arxiv.org/pdf/1710.09437)
- [LMD Ghost](https://inevitableeth.com/home/ethereum/network/consensus/lmd-ghost)
- [RLMD GHOST with Luca Zanolini](https://www.youtube.com/watch?v=F2olypDSVnA)
- [Comparison of different LMD Ghost Implementations by ProtoLambda](https://github.com/protolambda/lmd-ghost)
Loading
Loading