feat: add TrieCommitInterval configuration, commit trie every TrieCommitInterval blocks. #45
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Currently, all of our nodes can only be set to archive mode for gcMode. As the amount of data gradually increases, each of our nodes needs to be able to configure to full mode. This mode will delete Trie data that is lower than the latest block height minus TriesInMemory blocks, instead of committing and storing it in the database. This way, our nodes will occupy less disk space and have better performance. However, our official bridge and proposer components both need to use Trie data from about 1 hour ago. After enabling full mode, both the official bridge and proposer components will not function properly.
We need to enable full mode, and at the same time, we need to solve the problem of the official bridge and proposer using Trie data.
Rationale
It should be noted that the full mode does not discard all Trie data. It has a gcproc statistic that tracks the cumulative time spent processing blocks. When gcproc is greater than flushInterval (default is 1 hour), the full mode will commit and store the complete Trie data for the current block height minus TriesInMemory in the database. On the other hand, the submission frequency of our Mainnet and Testnet proposers is fixed and controlled by the configuration l2OutputOracleSubmissionInterval. We can use this configuration to calculate which block heights the official bridge and proposers need to use. Therefore, we add a TrieCommitInterval configuration in geth, whose value is equal to the value of l2OutputOracleSubmissionInterval for the corresponding network. In our code, we add a judgment condition that when the target block height (i.e. the latest block height minus TriesInMemory) divided by TrieCommitInterval results in a remainder of 0, it means that the target block height is the block height where we need to save the Trie data. We then commit the Trie data and store it in the database.
Example
add TrieCommitInterval config in config.toml
then add
gcMode=full
to geth param.All Trie data for block heights that are multiples of 240 will be available.
Changes
Notable changes: