Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is to open the discussion on adding multithreading to wallet3 to improve performance. After putting wallet3 through perf it highlighted 2 functions where wallet3 spent most of its time to scan the chain.
output_and_derivation_ours()
andgenerate_key_derivations()
Making some rough changes to wallet3 and adding threads around add_block resulted in a significant increase in block processing speed. Increasing wallet3 from around 2.5k blocks/second up to 13k blocks/second.
Before:
After:
However processing blocks out of order in this form isn't viable with the wallet outputting incorrect final balances after it finished scanning (in fact every time it resulted in a different balance). I suspect this is due to it ignoring outputs spent because the input hadnt been received yet
I do note that the two bottleneck functions are both in transaction scanning for outputs received and not outputs spent, so we possibly could split the scanning into first stage multithreaded scan of the outputs received, followed by a linear scan of outputs spent.
This is still pretty early days for wallet3 so adding multithreading might not be appropriate yet. But is good to know what the bottlenecks are atm