Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

script: Fix comparison of non-ordered columns #61

Merged
merged 1 commit into from
Oct 22, 2024

Conversation

joamaki
Copy link
Contributor

@joamaki joamaki commented Oct 21, 2024

The takeColumns function assumed that the column indexes were ordered, e.g. if the table header was "Foo Bar Baz", then it'd always be referenced in order: "Foo" or "Foo Baz" but never "Baz Foo". There's no reason to assume that, so fix this and add tests.

The takeColumns function assumed that the column indexes
were ordered, e.g. if the table header was "Foo Bar Baz",
then it'd always be referenced in order: "Foo" or "Foo Baz"
but never "Baz Foo". There's no reason to assume that, so fix
this and add tests.

Signed-off-by: Jussi Maki <jussi.maki@isovalent.com>
@joamaki joamaki requested a review from a team as a code owner October 21, 2024 14:19
@joamaki joamaki requested review from dylandreimerink and removed request for a team October 21, 2024 14:19
Copy link

$ make test
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading golang.org/x/text v0.14.0
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
ok  	github.com/cilium/statedb	26.069s	coverage: 82.6% of statements
ok  	github.com/cilium/statedb/index	0.005s	coverage: 28.7% of statements
ok  	github.com/cilium/statedb/internal	0.014s	coverage: 46.7% of statements
ok  	github.com/cilium/statedb/part	2.735s	coverage: 82.9% of statements
ok  	github.com/cilium/statedb/reconciler	0.355s	coverage: 88.4% of statements
-----
$ make bench
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                    	  448622	      2641 ns/op	    378598 objects/sec	    2816 B/op	      32 allocs/op
BenchmarkDB_WriteTxn_10-4                   	 1223732	      1002 ns/op	    997771 objects/sec	     743 B/op	      10 allocs/op
BenchmarkDB_WriteTxn_100-4                  	 1503622	       772.8 ns/op	   1293994 objects/sec	     600 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                 	 1488218	       802.8 ns/op	   1245638 objects/sec	     550 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4   	  499228	      2287 ns/op	    437214 objects/sec	    1503 B/op	      37 allocs/op
BenchmarkDB_Modify-4                        	    1315	    926533 ns/op	   1079293 objects/sec	  775532 B/op	    8464 allocs/op
BenchmarkDB_GetInsert-4                     	    1221	    995599 ns/op	   1004421 objects/sec	  759304 B/op	    8463 allocs/op
BenchmarkDB_RandomInsert-4                  	    2361	    499093 ns/op	   2003635 objects/sec	  402439 B/op	    7098 allocs/op
BenchmarkDB_RandomReplace-4                 	     320	   3740781 ns/op	    267324 objects/sec	 2359132 B/op	   48574 allocs/op
BenchmarkDB_SequentialInsert-4              	    1518	    800728 ns/op	   1248864 objects/sec	  552093 B/op	    7289 allocs/op
BenchmarkDB_Changes_Baseline-4              	    1255	    943962 ns/op	   1059365 objects/sec	  553166 B/op	   10253 allocs/op
BenchmarkDB_Changes-4                       	     697	   1708319 ns/op	    585371 objects/sec	  993570 B/op	   14494 allocs/op
BenchmarkDB_RandomLookup-4                  	   22027	     54399 ns/op	  18382552 objects/sec	     160 B/op	       1 allocs/op
BenchmarkDB_SequentialLookup-4              	   27295	     44323 ns/op	  22561905 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4         	    7824	    145499 ns/op	   6872902 objects/sec	   93900 B/op	    1044 allocs/op
BenchmarkDB_FullIteration_All-4             	     979	   1260310 ns/op	  79345725 objects/sec	     480 B/op	      12 allocs/op
BenchmarkDB_FullIteration_Get-4             	     225	   5364546 ns/op	  18640947 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4              	  454852	      2328 ns/op	        20.00 50th_µs	        23.00 90th_µs	       102.0 99th_µs	    1590 B/op	      24 allocs/op
PASS
ok  	github.com/cilium/statedb	27.698s
PASS
ok  	github.com/cilium/statedb/index	0.003s
PASS
ok  	github.com/cilium/statedb/internal	0.003s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Insert_RootOnlyWatch-4    	    8971	    131963 ns/op	   7577870 objects/sec	  104167 B/op	    2041 allocs/op
Benchmark_Insert-4                  	    6165	    180833 ns/op	   5529964 objects/sec	  219070 B/op	    3065 allocs/op
Benchmark_Modify-4                  	    8226	    140058 ns/op	   7139926 objects/sec	  212502 B/op	    1205 allocs/op
Benchmark_GetInsert-4               	    6853	    171370 ns/op	   5835325 objects/sec	  212639 B/op	    1205 allocs/op
Benchmark_Replace-4                 	26961614	        44.83 ns/op	  22304026 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4   	27158880	        44.54 ns/op	  22452950 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                   	 3071902	       389.4 ns/op	   2568037 objects/sec	     448 B/op	       7 allocs/op
Benchmark_txn_10-4                  	 7506034	       158.0 ns/op	   6330925 objects/sec	     154 B/op	       2 allocs/op
Benchmark_txn_100-4                 	 8434983	       142.5 ns/op	   7018749 objects/sec	     224 B/op	       2 allocs/op
Benchmark_txn_1000-4                	 7403386	       161.6 ns/op	   6188126 objects/sec	     216 B/op	       2 allocs/op
Benchmark_txn_delete_1-4            	 3197378	       370.3 ns/op	   2700240 objects/sec	     856 B/op	       6 allocs/op
Benchmark_txn_delete_10-4           	 8481597	       141.2 ns/op	   7083929 objects/sec	     132 B/op	       1 allocs/op
Benchmark_txn_delete_100-4          	10441216	       114.5 ns/op	   8731049 objects/sec	      60 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4         	11088364	       107.6 ns/op	   9295403 objects/sec	      26 B/op	       1 allocs/op
Benchmark_Get-4                     	   39201	     31242 ns/op	  32008700 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterate-4                 	  177207	      6955 ns/op	 143771941 objects/sec	      80 B/op	       3 allocs/op
Benchmark_Hashmap_Insert-4          	   16006	     74845 ns/op	  13360913 objects/sec	   86556 B/op	      64 allocs/op
Benchmark_Hashmap_Get_Uint64-4      	  155406	      7744 ns/op	 129137381 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4       	  147145	      8066 ns/op	 123974425 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Uint64Map_Random-4        	    1340	    857144 ns/op	   1166665 items/sec	 2701851 B/op	    9019 allocs/op
Benchmark_Uint64Map_Sequential-4    	    1504	    790764 ns/op	   1264601 items/sec	 2492404 B/op	    9749 allocs/op
PASS
ok  	github.com/cilium/statedb/part	29.289s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 2.88 seconds (batch size 1000)
Throughput 347247.81 objects per second
Allocated 6011325 objects, 424761kB bytes, 515088kB bytes still in use

@joamaki joamaki merged commit 1adc61a into main Oct 22, 2024
1 check passed
@joamaki joamaki deleted the pr/joamaki/fix-take-columns branch October 22, 2024 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants