Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor AtCommitLog #1535

Open
4 of 5 tasks
sitaram-kalluri opened this issue Sep 5, 2023 · 3 comments
Open
4 of 5 tasks

Refactor AtCommitLog #1535

sitaram-kalluri opened this issue Sep 5, 2023 · 3 comments
Assignees

Comments

@sitaram-kalluri
Copy link
Member

sitaram-kalluri commented Sep 5, 2023

Rework the AtCommitLog and CommitLogKeyStore to segregate client and server-specific functionalities.

Within commit_log_keystore.dart, introduce a new abstract class named "BaseCommitLogKeyStore." This class should implement HiveBase and contain methods that are common between the client and server. Subsequently, transform the CommitLogKeyStore to extend "BaseCommitLogKeyStore" instead of implementing HiveBase, now containing server-specific methods. Simultaneously, introduce a new class called "ClientCommitLogKeyStore," extending "CommitLogKeyStore," to house client-specific methods.

  • Design and code changes
  • Unit and functional test
  • Raise PR
  • Address PR review comments
  • Merge and publish the changes
@sitaram-kalluri
Copy link
Member Author

Work done in PR-72:

  • Implement CommitChange method.
  • Refactored removeNullCommitEntries
  • Refactored removeMalformedCommitEntries
  • Started in the bench marking the performance.

@sitaram-kalluri
Copy link
Member Author

Work done in PR-73:

  • Spent 8 SP in PR-73
  • Moved the commit log to Lazy box. The changes are available in lazy_commit_log_changes.
  • @purnimavenkatasubbu started testing the changes and found issue in at_server and at_client and the issues are fixed the changes are pushed.

Pending issue in PR-74

  • Retest the changes and raise a PR.

@purnimavenkatasubbu
Copy link
Member

purnimavenkatasubbu commented Nov 27, 2023

Re-tested the changes, Previously reported issues are fixed. Comparison between the time taken for sync with trunk and this branch

The observation is sync is taking considerably longer time as the commits increase. For example

  1. For just 1000 commits, The Initial sync takes 6 seconds to complete
  2. For 10000 commits, the time to sync for the first 1000 keys is 1 minute 45 seconds
  3. For 30000 commits, the time to sync for the first 1000 keys is around 3 minutes

trunk branch

1)For just 1000 commits, The Initial sync takes 7 seconds
2) For 10000 commits, the time taken for the first 1000 keys is 7 seconds
3) For 30000 commits, the time taken for the first 1000 keys is 7 seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants