Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PHPLIB-1183: Add benchmarks for codecs #1146

Merged
merged 3 commits into from
Aug 25, 2023
Merged

PHPLIB-1183: Add benchmarks for codecs #1146

merged 3 commits into from
Aug 25, 2023

Conversation

alcaeus
Copy link
Member

@alcaeus alcaeus commented Aug 25, 2023

PHPLIB-1183

The new benchmarks test the overhead of using a codec. They intentionally only pass through the Document in question or convert it to a PHP structure via toPHP so they remain comparable to the benchmarks that do this directly.

I also added another benchmark for BSON conversion using iteration, which will perform much better with the internal iterator made in PHPC-2269.

@alcaeus alcaeus requested a review from GromNaN August 25, 2023 10:58
@alcaeus alcaeus self-assigned this Aug 25, 2023
@GromNaN
Copy link
Member

GromNaN commented Aug 25, 2023 •
edited
Loading

Very nice to see that Codec (pass thru) is almost as fast as Raw BSON. But Codec (to array) is always the slowest.

Sharing the results for posterity.

benchmark::subject set mem_peak mode
ReadMultipleDocumentsBench::benchCursorToArray Driver default typemap 9.162mb 12.766ms
ReadMultipleDocumentsBench::benchCursorToArray Array typemap 8.946mb 12.436ms
ReadMultipleDocumentsBench::benchCursorToArray Library default typemap 10.107mb 16.344ms
ReadMultipleDocumentsBench::benchCursorToArray Raw BSON 1.564mb 3.189ms
ReadMultipleDocumentsBench::benchCursorToArray Codec (pass thru) 1.564mb 3.627ms
ReadMultipleDocumentsBench::benchCursorToArray Codec (to array) 9.315mb 14.101ms
ReadMultipleDocumentsBench::benchAccessId Driver default typemap 1.564mb 12.500ms
ReadMultipleDocumentsBench::benchAccessId Array typemap 1.564mb 12.243ms
ReadMultipleDocumentsBench::benchAccessId Library default typemap 1.565mb 16.228ms
ReadMultipleDocumentsBench::benchAccessId Raw BSON 1.564mb 3.235ms
ReadMultipleDocumentsBench::benchAccessId Codec (pass thru) 1.564mb 3.597ms
ReadMultipleDocumentsBench::benchAccessId Codec (to array) 1.564mb 13.869ms
ReadMultipleDocumentsBench::benchAccessNestedItem Driver default typemap 1.564mb 12.452ms
ReadMultipleDocumentsBench::benchAccessNestedItem Array typemap 1.564mb 12.274ms
ReadMultipleDocumentsBench::benchAccessNestedItem Library default typemap 1.565mb 16.423ms
ReadMultipleDocumentsBench::benchAccessNestedItem Raw BSON 1.564mb 4.004ms
ReadMultipleDocumentsBench::benchAccessNestedItem Codec (pass thru) 1.564mb 4.331ms
ReadMultipleDocumentsBench::benchAccessNestedItem Codec (to array) 1.564mb 13.816ms
ReadLargeDocumentBench::benchCursorToArray Driver default typemap 115.725mb 199.860ms
ReadLargeDocumentBench::benchCursorToArray Array typemap 111.936mb 182.977ms
ReadLargeDocumentBench::benchCursorToArray Library default typemap 115.726mb 199.056ms
ReadLargeDocumentBench::benchCursorToArray Raw BSON 1.564mb 25.477ms
ReadLargeDocumentBench::benchCursorToArray Codec (pass thru) 1.566mb 25.499ms
ReadLargeDocumentBench::benchCursorToArray Codec (to array) 144.818mb 206.599ms
ReadLargeDocumentBench::benchAccessId Driver default typemap 28.489mb 191.422ms
ReadLargeDocumentBench::benchAccessId Array typemap 24.701mb 176.181ms
ReadLargeDocumentBench::benchAccessId Library default typemap 28.489mb 191.676ms
ReadLargeDocumentBench::benchAccessId Raw BSON 1.564mb 21.655ms
ReadLargeDocumentBench::benchAccessId Codec (pass thru) 1.566mb 22.158ms
ReadLargeDocumentBench::benchAccessId Codec (to array) 30.325mb 204.137ms
ReadLargeDocumentBench::benchAccessFirstField Driver default typemap 28.489mb 192.469ms
ReadLargeDocumentBench::benchAccessFirstField Array typemap 24.701mb 175.848ms
ReadLargeDocumentBench::benchAccessFirstField Library default typemap 28.489mb 191.354ms
ReadLargeDocumentBench::benchAccessFirstField Raw BSON 1.564mb 22.553ms
ReadLargeDocumentBench::benchAccessFirstField Codec (pass thru) 1.566mb 22.411ms
ReadLargeDocumentBench::benchAccessFirstField Codec (to array) 30.325mb 203.086ms
ReadLargeDocumentBench::benchAccessLastField Driver default typemap 28.489mb 192.113ms
ReadLargeDocumentBench::benchAccessLastField Array typemap 24.701mb 175.583ms
ReadLargeDocumentBench::benchAccessLastField Library default typemap 28.489mb 192.206ms
ReadLargeDocumentBench::benchAccessLastField Raw BSON 1.564mb 38.488ms
ReadLargeDocumentBench::benchAccessLastField Codec (pass thru) 1.566mb 38.309ms
ReadLargeDocumentBench::benchAccessLastField Codec (to array) 30.325mb 200.955ms

GromNaN
GromNaN approved these changes Aug 25, 2023
View reviewed changes
Copy link
Member

@GromNaN GromNaN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after CS fixes.

@alcaeus alcaeus force-pushed the phplib-1183-codec-performance-tests branch from 759b6c1 to 1145036 Compare August 25, 2023 11:43
@alcaeus alcaeus force-pushed the phplib-1183-codec-performance-tests branch from 1145036 to 950546b Compare August 25, 2023 11:45
@alcaeus
Copy link
Member Author

alcaeus commented Aug 25, 2023 •
edited by GromNaN
Loading

Very nice to see that Codec (pass thru) is almost as fast as Raw BSON. But Codec (to array) is always the slowest.

I updated the data set to read Codec (to object), as the codec actually converts to an object. That data set is intended to be compared with the one that uses the driver default type map. It is expected to be considerably slower than the PassThruCodec one, as the former returns the Document while the latter invokes toPHP, replicating the behaviour of not passing a typeMap or codec option.

@GromNaN
Copy link
Member

GromNaN commented Aug 25, 2023

benchmark::subject set mem_peak mode
ReadMultipleDocumentsBench::benchCursorToArray Driver default typemap 9.162mb 20.221ms
ReadMultipleDocumentsBench::benchCursorToArray Array typemap 8.946mb 19.449ms
ReadMultipleDocumentsBench::benchCursorToArray Library default typemap 10.107mb 25.420ms
ReadMultipleDocumentsBench::benchCursorToArray Raw BSON 1.564mb 4.176ms
ReadMultipleDocumentsBench::benchCursorToArray Codec (pass thru) 1.564mb 4.884ms
ReadMultipleDocumentsBench::benchCursorToArray Codec (to object) 9.316mb 21.824ms
ReadMultipleDocumentsBench::benchAccessId Driver default typemap 1.564mb 19.256ms
ReadMultipleDocumentsBench::benchAccessId Array typemap 1.564mb 18.489ms
ReadMultipleDocumentsBench::benchAccessId Library default typemap 1.565mb 22.531ms
ReadMultipleDocumentsBench::benchAccessId Raw BSON 1.564mb 4.348ms
ReadMultipleDocumentsBench::benchAccessId Codec (pass thru) 1.564mb 5.467ms
ReadMultipleDocumentsBench::benchAccessId Codec (to object) 1.564mb 20.707ms
ReadMultipleDocumentsBench::benchAccessNestedItem Driver default typemap 1.564mb 19.025ms
ReadMultipleDocumentsBench::benchAccessNestedItem Array typemap 1.564mb 20.347ms
ReadMultipleDocumentsBench::benchAccessNestedItem Library default typemap 1.565mb 23.565ms
ReadMultipleDocumentsBench::benchAccessNestedItem Raw BSON 1.564mb 5.186ms
ReadMultipleDocumentsBench::benchAccessNestedItem Codec (pass thru) 1.564mb 5.721ms
ReadMultipleDocumentsBench::benchAccessNestedItem Codec (to object) 1.564mb 19.938ms
ReadLargeDocumentBench::benchCursorToArray Driver default typemap 115.725mb 298.247ms
ReadLargeDocumentBench::benchCursorToArray Array typemap 111.936mb 269.569ms
ReadLargeDocumentBench::benchCursorToArray Library default typemap 115.725mb 290.285ms
ReadLargeDocumentBench::benchCursorToArray Raw BSON 1.564mb 37.843ms
ReadLargeDocumentBench::benchCursorToArray Codec (pass thru) 1.567mb 39.509ms
ReadLargeDocumentBench::benchCursorToArray Codec (to object) 144.818mb 308.697ms
ReadLargeDocumentBench::benchAccessId Driver default typemap 28.489mb 283.525ms
ReadLargeDocumentBench::benchAccessId Array typemap 24.701mb 260.868ms
ReadLargeDocumentBench::benchAccessId Library default typemap 28.489mb 282.206ms
ReadLargeDocumentBench::benchAccessId Raw BSON 1.564mb 34.875ms
ReadLargeDocumentBench::benchAccessId Codec (pass thru) 1.567mb 34.487ms
ReadLargeDocumentBench::benchAccessId Codec (to object) 30.325mb 313.247ms
ReadLargeDocumentBench::benchAccessFirstField Driver default typemap 28.489mb 291.871ms
ReadLargeDocumentBench::benchAccessFirstField Array typemap 24.701mb 258.837ms
ReadLargeDocumentBench::benchAccessFirstField Library default typemap 28.489mb 287.565ms
ReadLargeDocumentBench::benchAccessFirstField Raw BSON 1.564mb 37.243ms
ReadLargeDocumentBench::benchAccessFirstField Codec (pass thru) 1.567mb 37.200ms
ReadLargeDocumentBench::benchAccessFirstField Codec (to object) 30.325mb 305.773ms
ReadLargeDocumentBench::benchAccessLastField Driver default typemap 28.489mb 294.900ms
ReadLargeDocumentBench::benchAccessLastField Array typemap 24.701mb 260.178ms
ReadLargeDocumentBench::benchAccessLastField Library default typemap 28.489mb 293.751ms
ReadLargeDocumentBench::benchAccessLastField Raw BSON 1.564mb 62.750ms
ReadLargeDocumentBench::benchAccessLastField Codec (pass thru) 1.567mb 61.813ms
ReadLargeDocumentBench::benchAccessLastField Codec (to object) 30.325mb 309.035ms

@alcaeus alcaeus merged commit a7a7682 into mongodb:master Aug 25, 2023
12 checks passed
@alcaeus alcaeus deleted the phplib-1183-codec-performance-tests branch August 25, 2023 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GromNaN GromNaN GromNaN approved these changes

Assignees

@alcaeus alcaeus

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alcaeus @GromNaN