Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Sync skip deletes <do not merge> #2131

Open
wants to merge 13 commits into
base: trunk
Choose a base branch
from
Open

Conversation

murali-shris
Copy link
Member

@murali-shris murali-shris commented Oct 17, 2024

- What I did

  • added ability to skip delete commit entries in sync response. If skipDeletes flag is set to true, delete commits will not be returned in sync response from server to client.
    - How I did it
  • introduced skipDeletes flag in sync:from syntax
  • added optional flag skipDeletes=false in commitLog getEntries. If set to true, delete commits will be skipped. if not passed, then existing behaviour applies. Refactored the code to introduce a strategy pattern to fetch keys.
  • added unit tests
    - How to verify it
  • tests should pass

TODO:
publish at_commons and at_persistence changes.

@murali-shris murali-shris changed the title feat: Sync skip deletes feat: Sync skip deletes <do not merge> Oct 17, 2024
@@ -127,12 +127,13 @@ class AtCommitLog extends BaseAtCommitLog {
/// Returns the Iterator of [_commitLogCacheMap] from the commitId specified.
@server
Iterator<MapEntry<String, CommitEntry>> getEntries(int commitId,
{String? regex, int limit = 25}) {
{String? regex, int limit = 25, bool skipDeletes = false}) {
Copy link
Member

@VJag VJag Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this a strategy.. where FetchAllKeysStrategy is to get all and SkipDeletesStrategy is to skip deletes, instead of if-else code might look more readable.

@@ -43,7 +43,8 @@ class SyncProgressiveVerbHandler extends AbstractVerbHandler {
// Get entries to sync
var commitEntryIterator = atCommitLog!.getEntries(
int.parse(verbParams[AtConstants.fromCommitSequence]!) + 1,
regex: verbParams['regex']);
regex: verbParams['regex'],
skipDeletes: verbParams['skipDeletes'] == 'true');
Copy link
Member

@VJag VJag Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VerbSyntax also needs this parameter, otherwise regex validation would fail.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@override
bool shouldIncludeEntryInSyncResponse(
CommitEntry commitEntry, int commitId, String regex,
{List<String>? enrolledNamespace}) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dont we need call to super.shouldIncludeEntryInSyncResponse()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i kept the method in base class abstract

import 'package:at_persistence_secondary_server/at_persistence_secondary_server.dart';
import 'package:at_utils/at_utils.dart';

abstract class SyncKeysFetchStrategy {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call it NamespaceHonoringKeyFetchStrategy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honoring namespace is one of the checks done. regex match and skipping delete commits are the other two checks. In the future if we have any other checks not related to namespace we may have to rename this class

@murali-shris murali-shris marked this pull request as ready for review October 22, 2024 07:06
Copy link
Contributor

@gkc gkc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but have suggested adding couple more tests

@@ -840,6 +840,33 @@ void main() async {
expect(commitEntriesMap.containsKey('public:phone.wavi@alice'), false);
expect(commitEntriesMap.containsKey('public:location@alice'), true);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add another test which both uses a regex and sets skipDeletes: true

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -469,6 +469,39 @@ void main() {
expect(syncResponse[3]['operation'], '*');
});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add another test which both uses a regex and sets skipDeletes: true

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants