Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GODRIVER-2914 x/mongo/driver: enable parallel zlib compression and improve zstd decompression #1320

Merged
merged 3 commits into from
Sep 5, 2023
Merged

GODRIVER-2914 x/mongo/driver: enable parallel zlib compression and improve zstd decompression #1320

merged 3 commits into from
Sep 5, 2023

Commits on Sep 5, 2023

  1. x/mongo/driver: benchmark decompression and fix benchmark data

    @charlievieth @matthewdale
    charlievieth authored and matthewdale committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    26566ed View commit details
    Browse the repository at this point in the history

  2. x/mongo/driver: enable parallel zlib compression and improve zstd dec… 

    …ompression

This commit fixes a bug where zlib compression was serialized across all
goroutines. This occurred because only one shared zlib decompresser was
instantiated for each compression level which had to be locked when used
and thus preventing concurrent compression.

The decompression performance of zstd is also improved by using a pool
of zstd decoders (instantiating a zstd encoded or decoder is fairly
expensive). It also slightly cleans up the logic used to store and
acquire zstd encoders.

The CompressPayload benchmark is changed to use a stable payload, which
is now stored in testdata/compression.go. Previously it used the
compression.go file itself, which made benchmarks results confusing and
liable to change whenever the compression/decompression logic changed.

```
goos: darwin
goarch: arm64
pkg: go.mongodb.org/mongo-driver/x/mongo/driver
                                    │ base.20.txt  │             new.20.txt              │
                                    │    sec/op    │   sec/op     vs base                │
CompressPayload/CompressorZLib-10     5387.4µ ± 0%   651.1µ ± 1%  -87.91% (p=0.000 n=20)
CompressPayload/CompressorZstd-10      64.56µ ± 1%   64.10µ ± 0%   -0.72% (p=0.000 n=20)
DecompressPayload/CompressorZLib-10    125.7µ ± 1%   123.7µ ± 0%   -1.60% (p=0.000 n=20)
DecompressPayload/CompressorZstd-10    70.13µ ± 1%   45.80µ ± 1%  -34.70% (p=0.000 n=20)
geomean                                235.3µ        124.0µ       -47.31%

                                    │ base.20.txt  │               new.20.txt               │
                                    │     B/s      │      B/s       vs base                 │
CompressPayload/CompressorZLib-10     365.2Mi ± 0%   3021.4Mi ± 1%  +727.41% (p=0.000 n=20)
CompressPayload/CompressorZstd-10     29.76Gi ± 1%    29.97Gi ± 0%    +0.73% (p=0.000 n=20)
DecompressPayload/CompressorZLib-10   15.28Gi ± 1%    15.53Gi ± 0%    +1.63% (p=0.000 n=20)
DecompressPayload/CompressorZstd-10   27.39Gi ± 1%    41.95Gi ± 1%   +53.13% (p=0.000 n=20)
geomean                               8.164Gi         15.49Gi        +89.77%

                                    │ base.20.txt  │               new.20.txt               │
                                    │     B/op     │     B/op      vs base                  │
CompressPayload/CompressorZLib-10     14.02Ki ± 0%   14.00Ki ± 0%   -0.10% (p=0.000 n=20)
CompressPayload/CompressorZstd-10     3.398Ki ± 0%   3.398Ki ± 0%        ~ (p=1.000 n=20) ¹
DecompressPayload/CompressorZLib-10   2.008Mi ± 0%   2.008Mi ± 0%        ~ (p=1.000 n=20) ¹
DecompressPayload/CompressorZstd-10   4.109Mi ± 0%   1.969Mi ± 0%  -52.08% (p=0.000 n=20)
geomean                               142.5Ki        118.5Ki       -16.82%
¹ all samples are equal

                                    │ base.20.txt  │              new.20.txt              │
                                    │  allocs/op   │ allocs/op   vs base                  │
CompressPayload/CompressorZLib-10       1.000 ± 0%   1.000 ± 0%        ~ (p=1.000 n=20) ¹
CompressPayload/CompressorZstd-10       4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=20) ¹
DecompressPayload/CompressorZLib-10     26.00 ± 0%   26.00 ± 0%        ~ (p=1.000 n=20) ¹
DecompressPayload/CompressorZstd-10   104.000 ± 0%   1.000 ± 0%  -99.04% (p=0.000 n=20)
geomean                                 10.20        3.193       -68.69%
¹ all samples are equal
```
    @charlievieth @matthewdale
    charlievieth authored and matthewdale committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    9bbdb31 View commit details
    Browse the repository at this point in the history

  3. Code review changes.

    @matthewdale
    matthewdale committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    51a6c7b View commit details
    Browse the repository at this point in the history