-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
perf(vector): updated marshalling of vector (#9109)
Earlier we were unmarshalling bytes to []float64 by iterating on each element and reading it little endian. But we are now doing it using unsafe pointers. This reduces thee time from O(size(bytes)) to O(1) basically. Benchmark stats: ``` goos: linux goarch: amd64 pkg: github.com/dgraph-io/dgraph/tok/index cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz BenchmarkEncodeDecodeUint64Matrix/Binary_Encoding/Decoding-8 185269 5991 ns/op BenchmarkEncodeDecodeUint64Matrix/Gob_Encoding/Decoding-8 46299 25547 ns/op BenchmarkEncodeDecodeUint64Matrix/JSON_Encoding/Decoding-8 77666 13723 ns/op BenchmarkEncodeDecodeUint64Matrix/PB_Encoding/Decoding-8 500092 2346 ns/op BenchmarkEncodeDecodeUint64Matrix/Unsafe_Encoding/Decoding-8 1465863 826.2 ns/op BenchmarkDotProduct/vek:size=96000-8 528499 2331 ns/op BenchmarkDotProduct/dotProduct:size=96000-8 170630 7765 ns/op BenchmarkDotProduct/dotProductT:size=96000-8 145855 8314 ns/op BenchmarkFloatConverstion/Current:size=96000-8 284873474 4.172 ns/op BenchmarkFloatConverstion/pointerFloat:size=96000-8 263052618 3.988 ns/op BenchmarkFloatConverstion/littleEndianFloat:size=96000-8 40446 35287 ns/op ``` Now indexing 500k vectors take about 5 minutes. (more than 5 hours before)
- Loading branch information
1 parent
a6e5233
commit 7e02491
Showing
16 changed files
with
844 additions
and
277 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.