Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimization: (eventsyncer) don't remove QBFT instances to speed up historical syncing #1615

Merged
merged 8 commits into from
Aug 13, 2024

Conversation

AKorpusenko
Copy link
Contributor

@AKorpusenko AKorpusenko commented Aug 12, 2024

Closes #1593

Copy link
Contributor

@MatusKysel MatusKysel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job!

messageID := convert.NewMsgID(eh.networkConfig.DomainType(), share.ValidatorPubKey[:], role)
return store.CleanAllInstances(logger, messageID[:])
}
err := eh.storageMap.Each(removeDecidedMessages)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is storageMap still used elsewhere? if not we should remove it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not used, I will remove it (as @AKorpusenko is sick today)

@moshe-blox moshe-blox changed the title Delete usage of store.CleanAllInstances to make historical events sync fast optimization: (eventsyncer) don't remove QBFT instances to speed up historical syncing Aug 13, 2024
@moshe-blox moshe-blox merged commit db74a7d into stage Aug 13, 2024
6 checks passed
@moshe-blox moshe-blox deleted the alan/db-delete-optimize branch August 13, 2024 09:31
@moshe-blox
Copy link
Contributor

well done! @y0sher @AKorpusenko

@y0sher y0sher restored the alan/db-delete-optimize branch October 28, 2024 12:21
@y0sher y0sher deleted the alan/db-delete-optimize branch October 28, 2024 12:21
y0sher added a commit that referenced this pull request Oct 28, 2024
…istorical syncing (#1615)

* optimize badger delete action

* try another optimization

* use dropprefix instead of delete all keys

* drop messages only when own share or fullnode

* removed usage of store.CleanAllInstances

* remove storageMap from eventhandler

---------

Co-authored-by: y0sher <lyosher@gmail.com>
Co-authored-by: Matus Kysel <matus@ssvlabs.io>
y0sher added a commit that referenced this pull request Oct 28, 2024
…ync [main] (#1824)

* optimization: (eventsyncer) don't remove QBFT instances to speed up historical syncing (#1615)

* optimize badger delete action

* try another optimization

* use dropprefix instead of delete all keys

* drop messages only when own share or fullnode

* removed usage of store.CleanAllInstances

* remove storageMap from eventhandler

---------

Co-authored-by: y0sher <lyosher@gmail.com>
Co-authored-by: Matus Kysel <matus@ssvlabs.io>

* fix build and lint

---------

Co-authored-by: Anton Korpusenko <antokorp@gmail.com>
Co-authored-by: Matus Kysel <matus@ssvlabs.io>
Co-authored-by: Nikita Kryuchkov <nkryuchkov10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants