client: integrate snapsync on experimental basis #3031
Conversation
g11tech
added
type: feature
PR state: WIP
package: client
target: master
Codecov Report
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. |
g11tech
force-pushed
the
integrate-snapsync
branch
from
db292f5
to
2e13846
Compare
g11tech
force-pushed
the
integrate-snapsync
branch
4 times, most recently
from
7ac9e24
to
abdb9ac
Compare
g11tech
force-pushed
the
integrate-snapsync
branch
from
3c7be01
to
8408df7
Compare
g11tech
force-pushed
the
integrate-snapsync
branch
2 times, most recently
from
fab73aa
to
7e531e0
Compare
use statemanager in snap fetchers and fix the snapsync startup and snapsync test functional snapsync integration with static peer state with hacks rebase fixes get static snapsync working again typefix fix spec track safe and finalized in finalized integrate building stat e with skeleton integrate account fetcher with the skeleton
g11tech
force-pushed
the
integrate-snapsync
branch
from
d08c31d
to
d676230
Compare
| const addressHex = bytesToUnprefixedHex(address.bytes) | ||
| const storageTrie = this._storageTries[addressHex] | ||
| // TODO PR: have a better interface for hashed address pull? | ||
| protected _getStorageTrie(addressOrHash: Address | Uint8Array, account?: Account): Trie { |
There was a problem hiding this comment.
@holgerd77 for your attention and feedback 🙏
There was a problem hiding this comment.
Hmm, since this is an internal method I would assume this should be fine for now.
Just FYI: we experimented though lately to get rid of this method (and generally: mechanism) completely here #3117 to see if we can get rid of creating all these trie copies.
That's likely (?) another story though.
|
Have merged |
added the handling for keeping unfinalized non canonical blocks around in skeleton to handle some reorgs without backfill |
|
Have updated this via UI |
There was a problem hiding this comment.
Ok, have had another high level look here, mainly from a how-is-the-main codebase affected. Also did two sanity run checs, one mainnet sync up to block 80.000 or so and one Holesky sync.
Seems things are working fine. 🙂 Can't oversee all the changes in the Skeleton class, e.g., but I do think we should take this code base in. Then we are getting more familiar with this over time, so this is very much a thing where one needs to grow some understanding/feeling of the behavior over time anyhow.
Will merge. 🎉 (thanks @scorbajio for the base stack and @g11tech for the integration! 😍)
| @@ -680,7 +681,7 @@ export class Config { | |||
| */ | |||
| getDnsDiscovery(option: boolean | undefined): boolean { | |||
| if (option !== undefined) return option | |||
| const dnsNets = ['holesky', 'sepolia'] | |||
| const dnsNets = ['goerli', 'sepolia', 'holesky'] | |||
There was a problem hiding this comment.
Ah, HERE this was hidden!
(I wondered why this never made it to master weeks after we had discussed 😂)
this PR aims to integrate snap sync on experimental basis (and has quite a good number fixes/refacs all over client beyond snapsync)
How to test it:
from
packages/clientNETWORK=mainnet NETWORKID=1337903 ELCLIENT=geth EXTRA_CL_PARAMS="--params.CAPELLA_FORK_EPOCH 0" DATADIR=/usr/app/ethereumjs/packages/client/data test/sim/single-run.shlet it run for
32+slots so that finalization happens, else you might end up waiting on step 2 anywayrm -rf ./datadir; EXTERNAL_RUN=true NETWORK=mainnet NETWORKID=1337903 ELCLIENT=geth SNAP_SYNC=true DATADIR=/usr/app/ethereumjs/packages/client/data npx vitest run test/sim/snapsync.spec.tsOR if you are feeling lucky then just try it on holesky (modify your data and jwtsecret args accordingly):
npm run client:start:ts -- --dataDir /data/lodestar-quickstart/holedata/ethereumjs --rpcEngineAddr 0.0.0.0 --jwtSecret /data/lodestar-quickstart/holesky-data/jwtsecret --network holesky --rpcEngine --snapi.e. just add
--snapadditonal arg to whatever is your usual run command