feat: support fetching historical proposer duties #7130
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #7130 +/- ##
============================================
- Coverage 49.12% 49.12% -0.01%
============================================
Files 597 597
Lines 39718 39721 +3
Branches 2085 2085
============================================
- Hits 19512 19511 -1
- Misses 20165 20168 +3
- Partials 41 42 +1 |
Performance Report✔️ no performance regression detected Full benchmark results
|
twoeths
reviewed
Oct 9, 2024
twoeths
reviewed
Oct 9, 2024
twoeths
reviewed
Oct 9, 2024
twoeths
reviewed
Oct 9, 2024
twoeths
reviewed
Oct 9, 2024
There was a problem hiding this comment.
historical state regen is not efficient until state diff strategy is implemented, even for state has just finalized
in the worse case we may have to reprocess 31 epochs which could be up to 3 minutes just to regen given 120ms to process a block and 2s to process an epoch, which could be too much (I'm not sure the performance of other clients)
instead of that we can think about a caching strategy similar to our balances cache:
this shares the same proposers memory with CachedBeaconState so does not cost us much
I was looking at extending the caches and storing proposers by epoch in
For the most common case, i.e. querying the finalized state we don't even need to trigger historical regen and it only takes around 2 seconds. For earlier states, you are right it takes a bit but at least we can serve this data, so depending on the use case it might be fine to wait 2 minutes for the response, we have the same limitation currently when serving the state via Imo we should use the same pattern everywhere to serve historical data and once we have #7005 it solves all the limitations. |
twoeths
previously approved these changes
Oct 11, 2024
twoeths
reviewed
Oct 11, 2024
twoeths
approved these changes
Oct 11, 2024
philknows
pushed a commit
that referenced
this pull request
Oct 18, 2024
* feat: support fetching historical proposer duties * Clone state before creating cached beacon state * Fix error message * Update test cases * Skip syncing pubkeys and sync committe cache * Update proposers tests epoch * Use switch/case instead of if/else * Add comment to clarify when head state can be used * Use loadState instead of creating a separate ViewDU * Clarify not yet initialized error for prev proposer duties * Assert loaded state epoch matches requested
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Fetching historical (mostly finalized) proposer duties has been a requirement by smoothing pools in the past. In addition, dappnode is building a staking rewards dashboard which also requires to pull finalized proposer duties from the node of the staker. Right now, Lodestar is the only beacon node that does not support serving this data.
Description
This change adds supports to fetch historical proposer duties by utilizing historical state regen added in #6033.
This has it's known limitations of being rather slow for far back epochs but for the most common use case which is fetching proposer duties for the finalized epoch it takes around 2 seconds to fetch duties which seems acceptable.
TODO