From 0cd43cca76fbe5490670350bdae75428eee7ddc7 Mon Sep 17 00:00:00 2001 From: Phil Ngo <58080811+philknows@users.noreply.github.com> Date: Tue, 30 Jul 2024 12:02:18 -0400 Subject: [PATCH] docs: update dependency graph (#6981) * add external dependencies * add lodestar prover * add flare * add lodestar-logger * add reqresp * Update reqresp description * Lint * Add light-client and prover to liraries that are useful individually * Consistently use Req/Resp (as per CL spec) in docs * Add Hashicorp to wordlist * Sort wordlist * Update wordlist * Sort * Fix header size of packages * Consistent with readme text * Consistent spelling of TypeScript in depgraph page * Fix package names * Update diagram * Remove extra lines * More visible arrows * Use note instead of info label * update readme.md * Fix formatting * update wordlist.txt * update description of reqresp on readme.md Co-authored-by: Nico Flaig * Remove extra whiespace * alignment * light client alignment * Lint README.md --------- Co-authored-by: Nico Flaig --- .wordlist.txt | 7 + README.md | 34 +-- docs/pages/contribution/depgraph.md | 238 ++++++++++++++---- .../pages/run/beacon-management/networking.md | 10 +- docs/pages/supporting-libraries/libraries.md | 2 + packages/reqresp/README.md | 2 +- packages/reqresp/package.json | 2 +- 7 files changed, 221 insertions(+), 74 deletions(-) diff --git a/.wordlist.txt b/.wordlist.txt index 7ba20c86be99..11a5a3746476 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -45,6 +45,7 @@ Grafana Grandine HTTPS HackMD +Hashicorp Homebrew IPFS IPv @@ -103,6 +104,7 @@ backfill beaconcha blockRoot blockchain +blst bootnode bootnodes bundlers @@ -140,6 +142,7 @@ ephemery ethers flamegraph flamegraphs +floodsub getNetworkIdentity gnosis heapdump @@ -161,6 +164,7 @@ malloc mdns merkle merkleization +mmeshsub monorepo multiaddr multifork @@ -172,6 +176,7 @@ orchestrator osx overriden params +performant pid plaintext pre @@ -179,6 +184,7 @@ premined produceBlockV protolambda prover +pubsub repo repos req @@ -205,6 +211,7 @@ util utils validator validators +verifier vite vitest webpack diff --git a/README.md b/README.md index 4afb92d4e92d..e03de683a9de 100644 --- a/README.md +++ b/README.md @@ -44,21 +44,25 @@ yarn build - :package: This mono-repository contains a suite of Ethereum Consensus packages. - :balance_scale: The mono-repository is released under [LGPLv3 license](./LICENSE). Note, that the packages contain their own licenses. -| Package | Version | License | Docs | Description | -| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | -| [`@lodestar/beacon-node`](./packages/beacon-node) | [![npm](https://img.shields.io/npm/v/@lodestar/beacon-node)](https://www.npmjs.com/package/@lodestar/beacon-node) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/beacon-node) | :rotating_light: Beacon-chain client | -| [`@lodestar/validator`](./packages/validator) | [![npm](https://img.shields.io/npm/v/@lodestar/validator)](https://www.npmjs.com/package/@lodestar/validator) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/validator) | :bank: Validator client | -| [`@lodestar/light-client`](./packages/light-client) | [![npm](https://img.shields.io/npm/v/@lodestar/light-client)](https://www.npmjs.com/package/@lodestar/light-client) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/light-client) | :bird: Ethereum Light client | -| [`@lodestar/api`](./packages/api) | [![npm](https://img.shields.io/npm/v/@lodestar/api)](https://www.npmjs.com/package/@lodestar/api) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/api) | :clipboard: REST Client for the Ethereum Beacon API | -| [`@chainsafe/lodestar`](./packages/cli) | [![npm](https://img.shields.io/npm/v/@chainsafe/lodestar)](https://www.npmjs.com/package/@chainsafe/lodestar) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/cli/) | :computer: Command-line tool for Lodestar | -| [`@lodestar/state-transition`](./packages/state-transition) | [![npm](https://img.shields.io/npm/v/@lodestar/state-transition)](https://www.npmjs.com/package/@lodestar/state-transition) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/state-transition) | :mag_right: Eth Consensus beacon-state transition | -| [`@lodestar/types`](./packages/types) | [![npm](https://img.shields.io/npm/v/@lodestar/types)](https://www.npmjs.com/package/@lodestar/types) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/types) | :spiral_notepad: Eth Consensus TypeScript and SSZ types | -| [`@lodestar/params`](./packages/params) | [![npm](https://img.shields.io/npm/v/@lodestar/params)](https://www.npmjs.com/package/@lodestar/params) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/params) | :spider_web: Eth Consensus network parameters | -| [`@lodestar/utils`](./packages/utils) | [![npm](https://img.shields.io/npm/v/@lodestar/utils)](https://www.npmjs.com/package/@lodestar/utils) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/utils) | :toolbox: Miscellaneous utility functions used across Lodestar | -| [`@lodestar/config`](./packages/config) | [![npm](https://img.shields.io/npm/v/@lodestar/config)](https://www.npmjs.com/package/@lodestar/config) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/config) | :spiral_notepad: Eth Consensus types and params bundled together | -| [`@lodestar/spec-test-util`](./packages/spec-test-util) | [![npm](https://img.shields.io/npm/v/@lodestar/spec-test-util)](https://www.npmjs.com/package/@lodestar/spec-test-util) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/spec-test-util) | :test_tube: Test harness for Eth Consensus spec tests | -| [`@lodestar/db`](./packages/db) | [![npm](https://img.shields.io/npm/v/@lodestar/db)](https://www.npmjs.com/package/@lodestar/db) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/db) | :floppy_disk: Read/write persistent Eth Consensus data | -| [`@lodestar/fork-choice`](./packages/fork-choice) | [![npm](https://img.shields.io/npm/v/@lodestar/fork-choice)](https://www.npmjs.com/package/@lodestar/fork-choice) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/fork-choice) | :fork_and_knife: Beacon-chain fork choice | +| Package | Version | License | Docs | Description | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | +| [`@chainsafe/lodestar`](./packages/cli) | [![npm](https://img.shields.io/npm/v/@chainsafe/lodestar)](https://www.npmjs.com/package/@chainsafe/lodestar) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/cli/) | :computer: Command-line tool for Lodestar | +| [`@lodestar/api`](./packages/api) | [![npm](https://img.shields.io/npm/v/@lodestar/api)](https://www.npmjs.com/package/@lodestar/api) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/api) | :clipboard: REST Client for the Ethereum Beacon API | +| [`@lodestar/beacon-node`](./packages/beacon-node) | [![npm](https://img.shields.io/npm/v/@lodestar/beacon-node)](https://www.npmjs.com/package/@lodestar/beacon-node) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/beacon-node) | :rotating_light: Beacon-chain client | +| [`@lodestar/config`](./packages/config) | [![npm](https://img.shields.io/npm/v/@lodestar/config)](https://www.npmjs.com/package/@lodestar/config) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/config) | :spiral_notepad: Eth Consensus types and params bundled together | +| [`@lodestar/db`](./packages/db) | [![npm](https://img.shields.io/npm/v/@lodestar/db)](https://www.npmjs.com/package/@lodestar/db) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/db) | :floppy_disk: Read/write persistent Eth Consensus data | +| [`@lodestar/flare`](./packages/flare) | [![npm](https://img.shields.io/npm/v/@lodestar/flare)](https://www.npmjs.com/package/@lodestar/flare) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/flare) | :boom: Command tool for triggering non-standard actions | +| [`@lodestar/fork-choice`](./packages/fork-choice) | [![npm](https://img.shields.io/npm/v/@lodestar/fork-choice)](https://www.npmjs.com/package/@lodestar/fork-choice) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/fork-choice) | :fork_and_knife: Beacon-chain fork choice | +| [`@lodestar/light-client`](./packages/light-client) | [![npm](https://img.shields.io/npm/v/@lodestar/light-client)](https://www.npmjs.com/package/@lodestar/light-client) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/light-client) | :bird: Ethereum Light client | +| [`@lodestar/logger`](./packages/logger) | [![npm](https://img.shields.io/npm/v/@lodestar/logger)](https://www.npmjs.com/package/@lodestar/logger) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/logger) | :memo: NodeJS logger for Lodestar binaries | +| [`@lodestar/params`](./packages/params) | [![npm](https://img.shields.io/npm/v/@lodestar/params)](https://www.npmjs.com/package/@lodestar/params) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/params) | :spider_web: Eth Consensus network parameters | +| [`@lodestar/prover`](./packages/prover) | [![npm](https://img.shields.io/npm/v/@lodestar/prover)](https://www.npmjs.com/package/@lodestar/prover) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/prover) | :white_check_mark: Ethereum Light client verifier for execution JSON-RPC calls | +| [`@lodestar/reqresp`](./packages/reqresp) | [![npm](https://img.shields.io/npm/v/@lodestar/reqresp)](https://www.npmjs.com/package/@lodestar/reqresp) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/reqresp) | :telephone_receiver: Eth Consensus Req/Resp protocol | +| [`@lodestar/spec-test-util`](./packages/spec-test-util) | [![npm](https://img.shields.io/npm/v/@lodestar/spec-test-util)](https://www.npmjs.com/package/@lodestar/spec-test-util) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/spec-test-util) | :test_tube: Test harness for Eth Consensus spec tests | +| [`@lodestar/state-transition`](./packages/state-transition) | [![npm](https://img.shields.io/npm/v/@lodestar/state-transition)](https://www.npmjs.com/package/@lodestar/state-transition) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/state-transition) | :mag_right: Eth Consensus beacon-state transition | +| [`@lodestar/types`](./packages/types) | [![npm](https://img.shields.io/npm/v/@lodestar/types)](https://www.npmjs.com/package/@lodestar/types) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/types) | :spiral_notepad: Eth Consensus TypeScript and SSZ types | +| [`@lodestar/utils`](./packages/utils) | [![npm](https://img.shields.io/npm/v/@lodestar/utils)](https://www.npmjs.com/package/@lodestar/utils) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/utils) | :toolbox: Miscellaneous utility functions used across Lodestar | +| [`@lodestar/validator`](./packages/validator) | [![npm](https://img.shields.io/npm/v/@lodestar/validator)](https://www.npmjs.com/package/@lodestar/validator) | [![License: LGPL v3](https://img.shields.io/badge/License-LGPL%20v3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) | [![documentation](https://img.shields.io/badge/readme-blue)](./packages/validator) | :bank: Validator client | ## Contributors diff --git a/docs/pages/contribution/depgraph.md b/docs/pages/contribution/depgraph.md index c0e3d232660f..17675cf47ff0 100644 --- a/docs/pages/contribution/depgraph.md +++ b/docs/pages/contribution/depgraph.md @@ -2,118 +2,252 @@ title: Dependency Graph --- -## Lodestar monorepo dependency graph +## Lodestar Dependency Graph -This is a diagram of the various `lodestar-*` packages in the Lodestar monorepo and how they fit together: +This is a diagram of the various Lodestar packages in the monorepo, how they fit together and interact with major external dependencies: -:::info -note: this dependency graph only applies to dependencies as they are used in the `src/` folders of each package, not in `test/` +:::note +This dependency graph only applies to dependencies as they are used in the `src/` folders of each package, not in `test/` ::: ```mermaid graph TD - lodestar["lodestar"]:::nodemodule - cli["lodestar-cli"]:::nodemodule - config["lodestar-config"]:::nodemodule - db["lodestar-db"]:::nodemodule - fork-choice["lodestar-fork-choice"]:::nodemodule - params["lodestar-params"]:::nodemodule - types["lodestar-types"]:::nodemodule - utils["lodestar-utils"]:::nodemodule - validator["lodestar-validator"]:::nodemodule - state-trans["lodestar-state-transition"]:::nodemodule - + api["api"]:::nodemodule + light-client["light-client"]:::nodemodule + prover["prover"]:::nodemodule + logger["logger"]:::nodemodule + reqresp["reqresp"]:::nodemodule + beacon-node["beacon-node"]:::nodemodule + cli["cli"]:::nodemodule + config["config"]:::nodemodule + db["db"]:::nodemodule + fork-choice["fork-choice"]:::nodemodule + params["params"]:::nodemodule + types["types"]:::nodemodule + utils["utils"]:::nodemodule + validator["validator"]:::nodemodule + state-transition["state-transition"]:::nodemodule + ssz["ssz"]:::nodemodule + blst["blst"]:::nodemodule + discv5["discv5"]:::nodemodule + libp2p["libp2p"]:::nodemodule + libp2p-gossipsub["libp2p-gossipsub"]:::nodemodule + libp2p-noise["libp2p-noise"]:::nodemodule + libp2p-yamux["libp2p-yamux"]:::nodemodule + + ssz-->api + ssz-->config + ssz-->types + ssz-->beacon-node + ssz-->validator + ssz-->light-client + ssz-->state-transition + + blst-->beacon-node + blst-->state-transition + + discv5-->beacon-node + + libp2p-->beacon-node + + libp2p-gossipsub-->libp2p + libp2p-noise-->libp2p + libp2p-yamux-->libp2p + + api-->beacon-node + api-->validator + api-->light-client + + light-client-->prover + + params-->api params-->config params-->types + params-->beacon-node + params-->validator + params-->light-client + params-->prover - types-->lodestar + types-->api + types-->beacon-node types-->cli types-->config types-->validator types-->fork-choice + types-->light-client + types-->prover - config-->lodestar + config-->api + config-->beacon-node config-->cli config-->validator config-->fork-choice - config-->state-trans + config-->state-transition config-->db + config-->light-client + config-->prover - utils-->lodestar + utils-->api + utils-->beacon-node utils-->db utils-->cli utils-->validator utils-->fork-choice - utils-->state-trans + utils-->state-transition + utils-->light-client + + logger-->beacon-node + logger-->validator + logger-->light-client + logger-->prover + logger-->cli + + reqresp-->beacon-node - state-trans-->lodestar - state-trans-->validator - state-trans-->fork-choice + state-transition-->beacon-node + state-transition-->validator + state-transition-->fork-choice - db-->lodestar + db-->beacon-node db-->validator - fork-choice-->lodestar + fork-choice-->beacon-node - lodestar-->cli + beacon-node-->cli validator-->cli + light-client-->cli + click api "https://github.com/ChainSafe/lodestar/tree/unstable/packages/api" + click light-client "https://github.com/ChainSafe/lodestar/tree/unstable/packages/light-client" + click prover "https://github.com/ChainSafe/lodestar/tree/unstable/packages/prover" + click logger "https://github.com/ChainSafe/lodestar/tree/unstable/packages/logger" + click reqresp "https://github.com/ChainSafe/lodestar/tree/unstable/packages/reqresp" click cli "https://github.com/ChainSafe/lodestar/tree/unstable/packages/cli" - click lodestar "https://github.com/ChainSafe/lodestar/tree/unstable/packages/beacon-node" + click beacon-node "https://github.com/ChainSafe/lodestar/tree/unstable/packages/beacon-node" click validator "https://github.com/ChainSafe/lodestar/tree/unstable/packages/validator" click db "https://github.com/ChainSafe/lodestar/tree/unstable/packages/db" click params "https://github.com/ChainSafe/lodestar/tree/unstable/packages/params" - click state-trans "https://github.com/ChainSafe/lodestar/tree/unstable/packages/state-transition" + click state-transition "https://github.com/ChainSafe/lodestar/tree/unstable/packages/state-transition" click fork-choice "https://github.com/ChainSafe/lodestar/tree/unstable/packages/fork-choice" click types "https://github.com/ChainSafe/lodestar/tree/unstable/packages/types" click utils "https://github.com/ChainSafe/lodestar/tree/unstable/packages/utils" click config "https://github.com/ChainSafe/lodestar/tree/unstable/packages/config" - - classDef nodemodule fill:grey,stroke-width:2px,stroke:black,color:white; - linkStyle default stroke:grey, fill:none,stroke-width:1.5px; + click ssz "https://github.com/ChainSafe/ssz" + click blst "https://github.com/ChainSafe/blst-ts" + click discv5 "https://github.com/ChainSafe/discv5" + click libp2p "https://github.com/libp2p/js-libp2p" + click libp2p-gossipsub "https://github.com/ChainSafe/js-libp2p-gossipsub" + click libp2p-noise "https://github.com/ChainSafe/js-libp2p-noise" + click libp2p-yamux "https://github.com/ChainSafe/js-libp2p-yamux" + + classDef nodemodule fill:grey,stroke-width:4px,font-size:48px,stroke:black,color:white; + linkStyle default stroke:grey,fill:none,stroke-width:4px; ``` -For a list of all the packages in the monorepo and a description for each, click [here](https://github.com/ChainSafe/lodestar#packages). +## Lodestar Monorepo -Let's talk about how each package fits together in finer detail, from top to bottom, following the chart. +For a list of all the packages in the monorepo and a description for each, click [here](https://github.com/ChainSafe/lodestar#architecture-overview). +Below is a brief summary of each package in alphabetical order. -## `@lodestar/params` +### `@chainsafe/lodestar` -[@lodestar/params](https://github.com/ChainSafe/lodestar/tree/unstable/packages/params) contains the parameters for configuring an Ethereum Consensus network. For example, the [mainnet params](https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/beacon-chain.md#configuration) +[@chainsafe/lodestar](https://github.com/ChainSafe/lodestar/tree/unstable/packages/cli) combines everything together for CLI usage and configuration of the beacon node and validator. -## `@lodestar/types` +### `@lodestar/api` -[@lodestar/types](https://github.com/ChainSafe/lodestar/tree/unstable/packages/types) contains Ethereum Consensus ssz types and data structures. +[@lodestar/api](https://github.com/ChainSafe/lodestar/tree/unstable/packages/api) contains a TypeScript REST client for the [Ethereum Consensus API](https://github.com/ethereum/beacon-apis). -## `@lodestar/config` +### `@lodestar/beacon-node` -[@lodestar/config](https://github.com/ChainSafe/lodestar/tree/unstable/packages/config) combines `@lodestar/params` and `@lodestar/types` together to be used as a single config object across the other Lodestar packages. +[@lodestar/beacon-node](https://github.com/ChainSafe/lodestar/tree/unstable/packages/beacon-node) contains the actual beacon node process itself, which is the aggregate of all the above packages and the "brain" of the Lodestar beacon chain implementation. All of the node modules live in this package as well. -## `@lodestar/utils` +### `@lodestar/config` -[@lodestar/utils](https://github.com/ChainSafe/lodestar/tree/unstable/packages/utils) contains various utilities that are common among the various Lodestar monorepo packages. +[@lodestar/config](https://github.com/ChainSafe/lodestar/tree/unstable/packages/config) combines `@lodestar/params` and `@lodestar/types` together to be used as a single config object across the other Lodestar packages. -## `@lodestar/state-transition` +### `@lodestar/db` -[@lodestar/state-transition](https://github.com/ChainSafe/lodestar/tree/unstable/packages/state-transition) contains the Lodestar implementation of the [beacon state transition function](https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/beacon-chain.md#beacon-chain-state-transition-function), which is used by `@lodestar/beacon-node` to perform the actual beacon state transition. This package also contains various functions used to calculate info about the beacon chain (such as `computeEpochAtSlot`) which are used by `@lodestar/fork-choice` and `@lodestar/validator` +[@lodestar/db](https://github.com/ChainSafe/lodestar/tree/unstable/packages/db) is where all persistent data about the beacon node is stored. Any package that needs to read or write persistent beacon node data depends on `@lodestar/db`. -## `@lodestar/db` +### `@lodestar/flare` -[@lodestar/db](https://github.com/ChainSafe/lodestar/tree/unstable/packages/db) is where all persistent data about the beacon node is stored. Any package that needs to read or write persistent beacon node data depends on `lodestar-db`. +[@lodestar/flare](https://github.com/ChainSafe/lodestar/tree/unstable/packages/flare) is a command tool used for easily triggering non-standard actions and debugging for researchers, developers and testers. Use with care. -## `@lodestar/fork-choice` +### `@lodestar/fork-choice` [@lodestar/fork-choice](https://github.com/ChainSafe/lodestar/tree/unstable/packages/fork-choice) holds the methods for reading/writing the fork choice DAG. The `@lodestar/beacon-node` package is the sole consumer of this package because the beacon node itself is what controls when the fork choice DAG is updated. -For a good explanation on how the fork choice itself works, see the [annotated fork choice spec](https://github.com/ethereum/annotated-spec/blob/master/phase0/fork-choice.md). This is an annotated version of the [Ethereum Consensus fork choice spec](https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/fork-choice.md) which `lodestar-fork-choice` is based on. +For a good explanation on how the fork choice itself works, see the [annotated fork choice spec](https://github.com/ethereum/annotated-spec/blob/master/phase0/fork-choice.md). This is an annotated version of the [Ethereum Consensus fork choice spec](https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/fork-choice.md) which `@lodestar/fork-choice` is based on. + +### `@lodestar/light-client` + +[@lodestar/light-client](https://github.com/ChainSafe/lodestar/tree/unstable/packages/light-client) is our light client designed to interact with the Ethereum blockchain in a trust-minimized matter via the sync committee and the [light-client protocol](https://github.com/ethereum/consensus-specs/tree/v1.4.0/specs/altair/light-client). + +### `@lodestar/logger` + +[@lodestar/logger](https://github.com/ChainSafe/lodestar/tree/unstable/packages/logger) is a common NodeJS logger for Lodestar binaries, which is required for worker threads to instantiate new loggers with consistent settings. + +### `@lodestar/params` + +[@lodestar/params](https://github.com/ChainSafe/lodestar/tree/unstable/packages/params) contains the parameters for configuring an Ethereum Consensus network. For example, the [mainnet params](https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/beacon-chain.md#configuration). + +### `@lodestar/prover` + +[@lodestar/prover](https://github.com/ChainSafe/lodestar/tree/unstable/packages/prover) is a web3 provider and a proxy to enable verification of JSON-RPC calls to the execution client using the [light-client protocol](https://github.com/ethereum/consensus-specs/tree/v1.4.0/specs/altair/light-client). + +### `@lodestar/reqresp` + +[@lodestar/reqresp](https://github.com/ChainSafe/lodestar/tree/unstable/packages/reqresp) contains the TypeScript implementation of the [Ethereum Consensus Req/Resp protocol](https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/p2p-interface.md#reqresp). + +### `@lodestar/spec-test-util` + +[@lodestar/spec-test-util](https://github.com/ChainSafe/lodestar/tree/unstable/packages/spec-test-util) is a Vitest test utility harness used for adhering to the [Ethereum Consensus specification tests](https://github.com/ethereum/consensus-specs/tree/v1.4.0/tests). + +### `@lodestar/state-transition` -## `@lodestar/validator` +[@lodestar/state-transition](https://github.com/ChainSafe/lodestar/tree/unstable/packages/state-transition) contains the Lodestar implementation of the [beacon state transition function](https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/beacon-chain.md#beacon-chain-state-transition-function), which is used by `@lodestar/beacon-node` to perform the actual beacon state transition. This package also contains various functions used to calculate info about the beacon chain (such as `computeEpochAtSlot`) which are used by `@lodestar/fork-choice` and `@lodestar/validator` + +### `@lodestar/types` + +[@lodestar/types](https://github.com/ChainSafe/lodestar/tree/unstable/packages/types) contains Ethereum Consensus ssz types and data structures. + +### `@lodestar/utils` + +[@lodestar/utils](https://github.com/ChainSafe/lodestar/tree/unstable/packages/utils) contains various utilities that are common among the various Lodestar monorepo packages. + +### `@lodestar/validator` [@lodestar/validator](https://github.com/ChainSafe/lodestar/tree/unstable/packages/validator) contains the validator client. The sole consumer of this package is `@chainsafe/lodestar`, which provides CLI access to run and configure the validator client. However, the validator client communicates to a REST API that is contained in `@lodestar/beacon-node` (specifically in the `api` module) to perform the validator duties. -## `@lodestar/beacon-node` +--- -[@lodestar/beacon-node](https://github.com/ChainSafe/lodestar/tree/unstable/packages/beacon-node) contains the actual beacon node process itself, which is the aggregate of all the above packages and the "brain" of the Lodestar beacon chain implementation. All of the node modules live in this package as well. +## External Dependencies -## `@chainsafe/lodestar` +Below is a brief summary, listed alphabetically, of each of our main external dependencies managed externally from our monorepo. -[@chainsafe/lodestar](https://github.com/ChainSafe/lodestar/tree/unstable/packages/cli) combines everything together for CLI usage and configuration of the beacon node and validator. +### `@chainsafe/blst-ts` + +[@chainsafe/blst-ts`](https://github.com/ChainSafe/blst-ts) is our TypeScript wrapper for [@supranational/blst](https://github.com/supranational/blst) native bindings, a highly performant BLS12-381 signature library. + +### `@chainsafe/discv5` + +[@chainsafe/discv5](https://github.com/ChainSafe/discv5) is our monorepo containing our TypeScript implementation of the [discv5 Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md). + +### `@chainsafe/js-libp2p-gossipsub` + +[@chainsafe/js-libp2p-gossipsub](https://github.com/ChainSafe/js-libp2p-gossipsub) is an implementation of pubsub based on mmeshsub and floodsub. Specified under [@libp2p/specs/pubsub/gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub). + +### `@chainsafe/js-libp2p-noise` + +[@chainsafe/js-libp2p-noise](https://github.com/ChainSafe/js-libp2p-noise) contains the TypeScript implementation of the Noise protocol, an encryption protocol used in [@libp2p/specs/noise](https://github.com/libp2p/specs/blob/master/noise). + +### `@chainsafe/js-libp2p-yamux` + +[@chainsafe/js-libp2p-yamux](https://github.com/ChainSafe/js-libp2p-yamux) contains the JavaScript implementation of the [Yamux multiplexer from Hashicorp](https://github.com/hashicorp/yamux/blob/master/spec.md) designed for usage with js-libp2p. + +### `@chainsafe/ssz` + +[@chainsafe/ssz](https://github.com/ChainSafe/ssz) contains the packages as a monorepo related to the [Simple Serialize](https://github.com/ethereum/consensus-specs/blob/v1.4.0/ssz/simple-serialize.md). + +### `@libp2p/js-libp2p` + +[@libp2p/js-libp2p](https://github.com/libp2p/js-libp2p) is the JavaScript implementation of the libp2p networking stack used in Ethereum's networking stack. diff --git a/docs/pages/run/beacon-management/networking.md b/docs/pages/run/beacon-management/networking.md index 75f463a9b8f7..aaa202dc0af3 100644 --- a/docs/pages/run/beacon-management/networking.md +++ b/docs/pages/run/beacon-management/networking.md @@ -51,9 +51,9 @@ Alternatively, the ENR can also be retrieved from the beacon node API by queryin [ENR Viewer](https://enr-viewer.com/) provides a simple and convenient option to decode and inspect ENRs. -## Peer Communication (gossipsub and ReqResp) +## Peer Communication (gossipsub and Req/Resp) -Gossipsub and ReqResp are the two mechanisms that beacon nodes use to exchange chain data. Gossipsub is used disseminate the most recent relevant data proactively throughout the network. ReqResp is used to directly ask specific peers for specific information (eg: during syncing). +Gossipsub and Req/Resp are the two mechanisms that beacon nodes use to exchange chain data. Gossipsub is used disseminate the most recent relevant data proactively throughout the network. Req/Resp is used to directly ask specific peers for specific information (eg: during syncing). ### Gossipsub @@ -63,13 +63,13 @@ In GossipSub, nodes can subscribe to topics, effectively joining the correspondi Messages are propagated through a blend of eager-push and lazy-pull models. Specifically, the protocol employs "mesh links" to carry full messages actively and "gossip links" to carry only message identifiers (lazy-pull propagation model). This hybrid approach allows for both active message propagation and reactive message retrieval​ which is an extension of the traditional hub-and-spoke pub/sub model. -### ReqResp +### Req/Resp -ReqResp is the domain of protocols that establish a flexible, on-demand mechanism to retrieve historical data and data missed by gossip. This family of methods, implemented as separate libp2p protocols, operate between a single requester and responder. A method is initiated via a libp2p protocol ID, with the initiator sending a request message and the responder sending a response message. Every method defines a specific request and response message type, and a specific protocol ID. This framework also facilitates streaming responses and robust error handling. +Req/Resp is the domain of protocols that establish a flexible, on-demand mechanism to retrieve historical data and data missed by gossip. This family of methods, implemented as separate libp2p protocols, operate between a single requester and responder. A method is initiated via a libp2p protocol ID, with the initiator sending a request message and the responder sending a response message. Every method defines a specific request and response message type, and a specific protocol ID. This framework also facilitates streaming responses and robust error handling. ## Data Transport (libp2p) -Libp2p is a modular and extensible network stack that serves as the data transport layer below both gossipsub and ReqResp and facilitates the lower-level peer-to-peer communications. It provides a suite of protocols for various networking functionalities including network transports, connection encryption and protocol multiplexing. Its modular design allows for the easy addition, replacement, or upgrading of protocols, ensuring an adaptable and evolving networking stack. +Libp2p is a modular and extensible network stack that serves as the data transport layer below both gossipsub and Req/Resp and facilitates the lower-level peer-to-peer communications. It provides a suite of protocols for various networking functionalities including network transports, connection encryption and protocol multiplexing. Its modular design allows for the easy addition, replacement, or upgrading of protocols, ensuring an adaptable and evolving networking stack. Libp2p operates at the lower levels of the OSI model, particularly at the Transport and Network layers. Libp2p supports both TCP and UDP protocols for establishing connections and data transmission. Combined with libp2p's modular design it can integrate with various networking technologies to facilitating both routing and addressing. diff --git a/docs/pages/supporting-libraries/libraries.md b/docs/pages/supporting-libraries/libraries.md index e1576ba542f3..e76ccc2f9ec7 100644 --- a/docs/pages/supporting-libraries/libraries.md +++ b/docs/pages/supporting-libraries/libraries.md @@ -11,6 +11,8 @@ Several useful Ethereum consensus libraries are developed as part of the [Lodest - [`config`](https://github.com/ChainSafe/lodestar/tree/unstable/packages/config) - Ethereum consensus run-time network configuration - [`api`](https://github.com/ChainSafe/lodestar/tree/unstable/packages/api) - Ethereum consensus REST API client - [`flare`](https://github.com/ChainSafe/lodestar/tree/unstable/packages/flare) - Beacon chain multi-purpose and debugging tool +- [`light-client`](https://github.com/ChainSafe/lodestar/tree/unstable/packages/light-client) - Ethereum light client +- [`prover`](https://github.com/ChainSafe/lodestar/tree/unstable/packages/prover) - A set of tools allowing to verify EL client JSON-RPC calls ## Other libraries diff --git a/packages/reqresp/README.md b/packages/reqresp/README.md index 38a6fbe39653..84205a5589c8 100644 --- a/packages/reqresp/README.md +++ b/packages/reqresp/README.md @@ -7,7 +7,7 @@ > This package is part of [ChainSafe's Lodestar](https://lodestar.chainsafe.io) project -Typescript REST client for the [Ethereum Consensus API spec](https://github.com/ethereum/beacon-apis) +Typescript implementation of the [Ethereum Consensus Req/Resp protocol](https://github.com/ethereum/consensus-specs/blob/v1.4.0/specs/phase0/p2p-interface.md#reqresp) ## Usage diff --git a/packages/reqresp/package.json b/packages/reqresp/package.json index 37c16318c5b3..5f72d47e4efb 100644 --- a/packages/reqresp/package.json +++ b/packages/reqresp/package.json @@ -1,6 +1,6 @@ { "name": "@lodestar/reqresp", - "description": "A Typescript implementation of the Ethereum Consensus ReqResp protocol", + "description": "A Typescript implementation of the Ethereum Consensus Req/Resp protocol", "license": "Apache-2.0", "author": "ChainSafe Systems", "homepage": "https://github.com/ChainSafe/lodestar#readme",