Snap sync moving root/healing implementation

[scorbajio]

This change will add the ability to the snap sync fetchers to have their target roots updated and will explore adding continual healing to the snap sync process.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 33.63%. Comparing base (fb50628) to head (edd0866).
Report is 13 commits behind head on master.

Additional details and impacted files

Flag	Coverage Δ
client	?
devp2p	0.00% <ø> (?)
trie	52.25% <0.00%> (?)
tx	78.77% <ø> (+0.33%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

[scorbajio]

Update: Currently, with #3354 merged, I am working on implementing functionality for the snap sync fetchers to change the state root that they are using for fetching state data.

In order to run the client manually for syncing Holesky using snap sync, use commands similar to the following:

DEBUG=ethjs,client:* npm run client:start -- --network holesky --dataDir /Volumes/Untitled/data/holesky-2/ethereumjs --jwtSecret /Volumes/Untitled/data/holesky-2/jwtsecret --saveReceipts --rpcEngine --
rpcEnginePort=8551 --rpcDebug --saveReceipts --snap

---

./lodestar beacon --network holesky --dataDir /Volumes/Untitled/data/holesky-2/lodestar --execution.urls http://localhost:8551 --rest.address 0.0.0.0 --rest.namespace '*' --jwt-secret /Volumes/Untitled/data/holesky-2/jwtsecret --logFileDailyRotate 5 --checkpointSyncUrl https://beaconstate-holesky.chainsafe.io

In order to run the client manually for syncing the smaller, more manageable edev1 devnet that has been setup for developing snap sync for EthereumJS, follow the following steps:

1. First, clone the lodestar-quickstart repo and run ./setup.sh --network ethjsdev1 --dataDir edev1-data --elClient ethereumjs from inside the root repo folder in order to setup the edev1-data folder containing the network configurations necessary for running the network inside of the edev1-data folder created (named using the dataDir option)
2. Next, run commands similar to the following, substituting the path to the network-configs directory found in the repo created by the quickstart scripts in step (1):

DEBUG=ethjs,client:* npm run client:start -- --dataDir /Volumes/Untitled/data/edev0-data-1/ethereumjs/ --jwtSecret /Volumes/Untitled/data/edev0-data-1/jwtsecret --saveReceipts --rpcEngine --rpcEnginePort=8552 --rpcDebug --gethGenesis /Volumes/Untitled/data/edev0-data-1/network-configs/devnet-1/genesis.json --snap --bootnodes=enode://2d38c72f7dfb7b68258dff2d899a4616ef8f3b8d094ddf480ee0d6455a8b4bbe639e42023be010206f1b69ce5c679ceb2f5d5b53edbe40f4507b81687e1842a8@116.203.135.75:30303?discport=30303,enode://d31d956e328e2de8f56df1604e74c0ebe147fc5417ec2726fd4b51956165232d7e2e169b91e7c132ca8b953ddf472456b4737e4c3a1bb1a2126c0dc87269c082@49.12.232.231:30303?discport=30303,enode://15267e3bd7d9ca3c16ca0556ef8aacdcb3b1f1c5fd812e0acae541e3945a769fd0a16b7a78d8c7355737124db5240912330001a4d5ac146544434d4775f01717@116.203.80.35:30303?discport=30303,enode://31dde0ddb2a5088f84bfcd73575f8703c98c24db9dad8904cd4c923dbbc4cd4ad589b3323affc62e170b2d80adc4633b17ff48fc045b029aebbe9800ebf0beca@116.203.148.252:30303?discport=30303

./lodestar beacon --paramsFile /Volumes/Untitled/data/edev1-data-1/network-configs/devnet-1/config.yaml --genesisStateFile /Volumes/Untitled/data/edev1-data-1/network-configs/devnet-1/genesis.ssz --eth1.depositContractDeployBlock 0 --bootnodesFile /Volumes/Untitled/data/edev1-data-1/network-configs/devnet-1/boot_enr.yaml --dataDir /Volumes/Untitled/data/edev1-data-1/lodestar2 --execution.urls http://localhost:8552 --rest.address 0.0.0.0 --rest.namespace '*' --jwt-secret /Volumes/Untitled/data/edev1-data-1/jwtsecret --logFileDailyRotate 5 --port 9000 --rest.port 9596

In both cases, the jwtsecret file should contain a valid jwt token, e.g.: 0xdc6457099f127cf0bac78de8b297df04951281909db4f58b43def7c7151e765d

[g11tech]

no we don't need stateroot from peer, we already have latest stateroot coming in from CL

[scorbajio]

I thought #3354 was for tracking and accessing the latest root/height from the peer latest function. Are you suggesting passing down the latest root from FCU calls?

[g11tech]

this function with this case would/should anyway be called when heal phase completed successfully

[g11tech]

need to see that the root is not continually updated.

so the thing is: you start trie node fetcher with latest root, keep requesting data till peers give you, if they start replying with empty, then we restart the entire fetcher with the new root.

so this is to be implemented as a while loop in the account fetcher's fetch where we start and run the trie node fetcher (either fetcher resolves or errors with error "out of range root", which will cause account fetcher to spin a new trie node fetcher with new root)