-
-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add PushPull benchmark test #658
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #658 +/- ##
==========================================
+ Coverage 48.89% 48.96% +0.07%
==========================================
Files 69 69
Lines 10076 10090 +14
==========================================
+ Hits 4927 4941 +14
- Misses 4629 4631 +2
+ Partials 520 518 -2 ☔ View full report in Codecov by Sentry. |
7892142
to
fdc2e1c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contribution.
I left a few comments.
c4b13a3
to
952b36a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. 👍
What this PR does / why we need it:
This PR introduces a benchmark test for PushPull in Yorkie.
While Yorkie already features gRPC benchmarks for
UpdateAndSync
andUpdateProject
, they may not provide accurate measurements of database I/O performance. As a result, these PushPull benchmarks are specifically designed to assess the performance of the server'sPushPull
functionality.To ensure accurate results, we make use of
b.StartTimer()
,b.StopTimer()
, andb.ResetTimer()
to exclude time spent on costly setup operations from the benchmark results.These benchmarks focus on various dimensions, including the type of operation (push/pull), the number of changes (100/1000) within a change pack, and the size of the snapshot in the change pack (3KB/30KB). It's important to note that the benchmark target database is currently only MongoDB.
We believe that these benchmarks will be valuable in estimating the potential improvements proposed in issue #597, assessing the effectiveness of database sharding (#530), and determining suitable snapshot intervals and thresholds for various CRDTs.
Plus, the benchmark task in CI is separated from the build task to run this in parallel.
Which issue(s) this PR fixes:
Fixes #600
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
Additional documentation:
Checklist: