Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improve type.hashTreeRoot() using batch #409

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

feat: improve type.hashTreeRoot() using batch #409

wants to merge 3 commits into from

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Oct 15, 2024

Motivation

  • improve type.hashTreeRoot() using batch

Description

  • instead of getRoots() and compute root from there, this PR implement getChunkBytes()
    • this compute root using merkleizeInto() which use batch there
    • reuse chunkBytesBuffer memory in type, almost no Uint8Array allocations in the middle
  • new hashTreeRootInto() api. This is needed in case consumers want to reuse memory allocation there
  • use allocUnsafe() of as-sha256 where it makes sense

cherry picked from #378

@twoeths twoeths mentioned this pull request Oct 15, 2024
Closed
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 15, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: fcebdd6 Previous: e58781f Ratio
digestTwoHashObjects 50023 times 48.484 ms/op 48.243 ms/op 1.00
digest64 50023 times 53.706 ms/op 53.846 ms/op 1.00
digest 50023 times 55.770 ms/op 55.442 ms/op 1.01
input length 32 1.2130 us/op 1.2160 us/op 1.00
input length 64 1.3430 us/op 1.3420 us/op 1.00
input length 128 2.3090 us/op 2.2870 us/op 1.01
input length 256 3.3630 us/op 3.3970 us/op 0.99
input length 512 5.6020 us/op 5.6410 us/op 0.99
input length 1024 10.818 us/op 11.105 us/op 0.97
digest 1000000 times 869.89 ms/op 887.39 ms/op 0.98
hashObjectToByteArray 50023 times 1.7115 ms/op 1.7092 ms/op 1.00
byteArrayToHashObject 50023 times 2.3524 ms/op 2.3218 ms/op 1.01
digest64 200092 times 217.98 ms/op 219.70 ms/op 0.99
hash 200092 times using batchHash4UintArray64s 237.75 ms/op 237.40 ms/op 1.00
digest64HashObjects 200092 times 195.35 ms/op 197.01 ms/op 0.99
hash 200092 times using batchHash4HashObjectInputs 215.97 ms/op 202.27 ms/op 1.07
getGindicesAtDepth 4.2830 us/op 4.2090 us/op 1.02
iterateAtDepth 7.8710 us/op 7.6830 us/op 1.02
getGindexBits 465.00 ns/op 458.00 ns/op 1.02
gindexIterator 1.0830 us/op 1.0570 us/op 1.02
HashComputationLevel.push then loop 25.114 ms/op 26.744 ms/op 0.94
HashComputation[] push then loop 40.745 ms/op 49.699 ms/op 0.82
hash 2 Uint8Array 500000 times - as-sha256 548.62 ms/op 542.64 ms/op 1.01
hashTwoObjects 500000 times - as-sha256 504.59 ms/op 506.48 ms/op 1.00
executeHashComputations - as-sha256 47.816 ms/op 46.500 ms/op 1.03
hash 2 Uint8Array 500000 times - noble 1.2215 s/op 1.2005 s/op 1.02
hashTwoObjects 500000 times - noble 1.7110 s/op 1.7027 s/op 1.00
executeHashComputations - noble 36.917 ms/op 43.488 ms/op 0.85
hash 2 Uint8Array 500000 times - hashtree 231.79 ms/op 221.65 ms/op 1.05
hashTwoObjects 500000 times - hashtree 227.59 ms/op 210.11 ms/op 1.08
executeHashComputations - hashtree 11.412 ms/op 11.860 ms/op 0.96
getHashComputations 2.6357 ms/op 2.7888 ms/op 0.95
executeHashComputations 11.155 ms/op 12.028 ms/op 0.93
get root 18.106 ms/op 18.120 ms/op 1.00
getNodeH() x7812.5 avg hindex 13.089 us/op 11.963 us/op 1.09
getNodeH() x7812.5 index 0 6.3550 us/op 6.2460 us/op 1.02
getNodeH() x7812.5 index 7 6.3070 us/op 6.2880 us/op 1.00
getNodeH() x7812.5 index 7 with key array 6.3320 us/op 6.2660 us/op 1.01
new LeafNode() x7812.5 14.793 us/op 14.767 us/op 1.00
getHashComputations 250000 nodes 19.062 ms/op 19.834 ms/op 0.96
batchHash 250000 nodes 89.115 ms/op 88.667 ms/op 1.01
get root 250000 nodes 124.95 ms/op 115.88 ms/op 1.08
getHashComputations 500000 nodes 30.523 ms/op 26.421 ms/op 1.16
batchHash 500000 nodes 152.23 ms/op 167.13 ms/op 0.91
get root 500000 nodes 249.96 ms/op 229.90 ms/op 1.09
getHashComputations 1000000 nodes 53.876 ms/op 74.207 ms/op 0.73
batchHash 1000000 nodes 295.34 ms/op 359.23 ms/op 0.82
get root 1000000 nodes 505.45 ms/op 477.74 ms/op 1.06
multiproof - depth 15, 1 requested leaves 8.6390 us/op 8.2080 us/op 1.05
tree offset multiproof - depth 15, 1 requested leaves 18.930 us/op 18.153 us/op 1.04
compact multiproof - depth 15, 1 requested leaves 3.8490 us/op 3.5000 us/op 1.10
multiproof - depth 15, 2 requested leaves 12.225 us/op 11.886 us/op 1.03
tree offset multiproof - depth 15, 2 requested leaves 22.345 us/op 22.621 us/op 0.99
compact multiproof - depth 15, 2 requested leaves 3.9520 us/op 3.5070 us/op 1.13
multiproof - depth 15, 3 requested leaves 17.008 us/op 16.968 us/op 1.00
tree offset multiproof - depth 15, 3 requested leaves 29.812 us/op 29.267 us/op 1.02
compact multiproof - depth 15, 3 requested leaves 4.6780 us/op 4.1610 us/op 1.12
multiproof - depth 15, 4 requested leaves 22.368 us/op 23.212 us/op 0.96
tree offset multiproof - depth 15, 4 requested leaves 35.445 us/op 33.747 us/op 1.05
compact multiproof - depth 15, 4 requested leaves 5.5420 us/op 4.8340 us/op 1.15
packedRootsBytesToLeafNodes bytes 4000 offset 0 2.2770 us/op 1.9940 us/op 1.14
packedRootsBytesToLeafNodes bytes 4000 offset 1 2.3300 us/op 1.9880 us/op 1.17
packedRootsBytesToLeafNodes bytes 4000 offset 2 2.2720 us/op 2.0170 us/op 1.13
packedRootsBytesToLeafNodes bytes 4000 offset 3 2.2800 us/op 2.0030 us/op 1.14
subtreeFillToContents depth 40 count 250000 42.961 ms/op 44.179 ms/op 0.97
setRoot - gindexBitstring 8.9936 ms/op 9.0113 ms/op 1.00
setRoot - gindex 9.5305 ms/op 9.2688 ms/op 1.03
getRoot - gindexBitstring 2.7196 ms/op 2.4680 ms/op 1.10
getRoot - gindex 3.7234 ms/op 3.2010 ms/op 1.16
getHashObject then setHashObject 10.160 ms/op 9.8988 ms/op 1.03
setNodeWithFn 7.8742 ms/op 7.6070 ms/op 1.04
getNodeAtDepth depth 0 x100000 1.1217 ms/op 1.1146 ms/op 1.01
setNodeAtDepth depth 0 x100000 2.6093 ms/op 2.4256 ms/op 1.08
getNodesAtDepth depth 0 x100000 1.0558 ms/op 1.0526 ms/op 1.00
setNodesAtDepth depth 0 x100000 1.5485 ms/op 1.5199 ms/op 1.02
getNodeAtDepth depth 1 x100000 1.1815 ms/op 1.1876 ms/op 0.99
setNodeAtDepth depth 1 x100000 5.5882 ms/op 6.5339 ms/op 0.86
getNodesAtDepth depth 1 x100000 1.1772 ms/op 1.1782 ms/op 1.00
setNodesAtDepth depth 1 x100000 4.4687 ms/op 4.6101 ms/op 0.97
getNodeAtDepth depth 2 x100000 1.4586 ms/op 1.4553 ms/op 1.00
setNodeAtDepth depth 2 x100000 9.3601 ms/op 9.0335 ms/op 1.04
getNodesAtDepth depth 2 x100000 18.690 ms/op 17.148 ms/op 1.09
setNodesAtDepth depth 2 x100000 13.626 ms/op 12.973 ms/op 1.05
tree.getNodesAtDepth - gindexes 7.9077 ms/op 7.5545 ms/op 1.05
tree.getNodesAtDepth - push all nodes 1.9021 ms/op 1.8291 ms/op 1.04
tree.getNodesAtDepth - navigation 233.87 us/op 235.13 us/op 0.99
tree.setNodesAtDepth - indexes 439.93 us/op 402.14 us/op 1.09
set at depth 8 491.00 ns/op 457.00 ns/op 1.07
set at depth 16 638.00 ns/op 590.00 ns/op 1.08
set at depth 32 998.00 ns/op 932.00 ns/op 1.07
iterateNodesAtDepth 8 256 14.231 us/op 13.012 us/op 1.09
getNodesAtDepth 8 256 3.5610 us/op 3.3990 us/op 1.05
iterateNodesAtDepth 16 65536 4.4088 ms/op 4.2236 ms/op 1.04
getNodesAtDepth 16 65536 1.5758 ms/op 1.5425 ms/op 1.02
iterateNodesAtDepth 32 250000 15.980 ms/op 15.836 ms/op 1.01
getNodesAtDepth 32 250000 4.3117 ms/op 4.2444 ms/op 1.02
iterateNodesAtDepth 40 250000 15.919 ms/op 15.697 ms/op 1.01
getNodesAtDepth 40 250000 4.3804 ms/op 4.3415 ms/op 1.01
250000 validators root getter 126.71 ms/op 116.32 ms/op 1.09
250000 validators batchHash() 87.934 ms/op 83.153 ms/op 1.06
250000 validators hashComputations 14.561 ms/op 17.111 ms/op 0.85
bitlist bytes to struct (120,90) 1.0510 us/op 912.00 ns/op 1.15
bitlist bytes to tree (120,90) 3.9610 us/op 3.7060 us/op 1.07
bitlist bytes to struct (2048,2048) 1.4120 us/op 1.3000 us/op 1.09
bitlist bytes to tree (2048,2048) 4.4490 us/op 4.1160 us/op 1.08
ByteListType - deserialize 8.1545 ms/op 7.6292 ms/op 1.07
BasicListType - deserialize 18.955 ms/op 17.349 ms/op 1.09
ByteListType - serialize 8.7219 ms/op 7.6450 ms/op 1.14
BasicListType - serialize 10.911 ms/op 10.296 ms/op 1.06
BasicListType - tree_convertToStruct 31.146 ms/op 29.534 ms/op 1.05
List[uint8, 68719476736] len 300000 ViewDU.getAll() + iterate 5.2095 ms/op 4.5576 ms/op 1.14
List[uint8, 68719476736] len 300000 ViewDU.get(i) 3.9864 ms/op 3.9833 ms/op 1.00
Array.push len 300000 empty Array - number 7.5258 ms/op 6.2896 ms/op 1.20
Array.set len 300000 from new Array - number 2.3082 ms/op 2.1278 ms/op 1.08
Array.set len 300000 - number 6.7902 ms/op 5.9731 ms/op 1.14
Uint8Array.set len 300000 488.91 us/op 486.54 us/op 1.00
Uint32Array.set len 300000 569.48 us/op 562.82 us/op 1.01
Container({a: uint8, b: uint8}) getViewDU x300000 47.736 ms/op 47.349 ms/op 1.01
ContainerNodeStruct({a: uint8, b: uint8}) getViewDU x300000 10.896 ms/op 10.823 ms/op 1.01
List(Container) len 300000 ViewDU.getAllReadonly() + iterate 213.98 ms/op 210.41 ms/op 1.02
List(Container) len 300000 ViewDU.getAllReadonlyValues() + iterate 238.78 ms/op 241.62 ms/op 0.99
List(Container) len 300000 ViewDU.get(i) 6.4353 ms/op 6.6224 ms/op 0.97
List(Container) len 300000 ViewDU.getReadonly(i) 6.3232 ms/op 6.3623 ms/op 0.99
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonly() + iterate 34.533 ms/op 38.805 ms/op 0.89
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonlyValues() + iterate 5.9069 ms/op 4.5938 ms/op 1.29
List(ContainerNodeStruct) len 300000 ViewDU.get(i) 6.1454 ms/op 6.0350 ms/op 1.02
List(ContainerNodeStruct) len 300000 ViewDU.getReadonly(i) 5.9990 ms/op 5.9905 ms/op 1.00
Array.push len 300000 empty Array - object 6.8892 ms/op 6.0831 ms/op 1.13
Array.set len 300000 from new Array - object 2.0455 ms/op 2.0601 ms/op 0.99
Array.set len 300000 - object 5.7225 ms/op 5.8596 ms/op 0.98
cachePermanentRootStruct no cache 3.6160 us/op 5.4530 us/op 0.66
cachePermanentRootStruct with cache 214.00 ns/op 198.00 ns/op 1.08
epochParticipation len 250000 rws 7813 2.2468 ms/op 2.2599 ms/op 0.99
Deneb BeaconBlock.hashTreeRoot(), numTransaction=200 5.5946 ms/op
BeaconState ViewDU hashTreeRoot() vc=200000 108.86 ms/op 105.82 ms/op 1.03
BeaconState ViewDU recursive hash - commit step vc=200000 4.0561 ms/op 4.4781 ms/op 0.91
BeaconState ViewDU validator tree creation vc=10000 36.394 ms/op 36.296 ms/op 1.00
BeaconState ViewDU batchHashTreeRoot vc=200000 97.647 ms/op 95.770 ms/op 1.02
BeaconState ViewDU hashTreeRoot - commit step vc=200000 85.363 ms/op 83.425 ms/op 1.02
BeaconState ViewDU hashTreeRoot - hash step vc=200000 15.523 ms/op 15.702 ms/op 0.99
deserialize Attestation - tree 4.3500 us/op 4.6820 us/op 0.93
deserialize Attestation - struct 1.9660 us/op 1.8740 us/op 1.05
deserialize SignedAggregateAndProof - tree 3.8720 us/op 3.9380 us/op 0.98
deserialize SignedAggregateAndProof - struct 3.1120 us/op 2.8760 us/op 1.08
deserialize SyncCommitteeMessage - tree 1.1310 us/op 1.1190 us/op 1.01
deserialize SyncCommitteeMessage - struct 1.1160 us/op 1.0660 us/op 1.05
deserialize SignedContributionAndProof - tree 2.1970 us/op 2.1980 us/op 1.00
deserialize SignedContributionAndProof - struct 2.5090 us/op 2.4530 us/op 1.02
deserialize SignedBeaconBlock - tree 234.43 us/op 220.45 us/op 1.06
deserialize SignedBeaconBlock - struct 126.21 us/op 120.94 us/op 1.04
BeaconState vc 300000 - deserialize tree 628.04 ms/op 625.35 ms/op 1.00
BeaconState vc 300000 - serialize tree 152.06 ms/op 167.11 ms/op 0.91
BeaconState.historicalRoots vc 300000 - deserialize tree 757.00 ns/op 740.00 ns/op 1.02
BeaconState.historicalRoots vc 300000 - serialize tree 683.00 ns/op 661.00 ns/op 1.03
BeaconState.validators vc 300000 - deserialize tree 611.00 ms/op 594.08 ms/op 1.03
BeaconState.validators vc 300000 - serialize tree 103.22 ms/op 102.95 ms/op 1.00
BeaconState.balances vc 300000 - deserialize tree 23.985 ms/op 23.774 ms/op 1.01
BeaconState.balances vc 300000 - serialize tree 3.6822 ms/op 3.5483 ms/op 1.04
BeaconState.previousEpochParticipation vc 300000 - deserialize tree 405.52 us/op 407.23 us/op 1.00
BeaconState.previousEpochParticipation vc 300000 - serialize tree 302.40 us/op 300.47 us/op 1.01
BeaconState.currentEpochParticipation vc 300000 - deserialize tree 414.25 us/op 367.26 us/op 1.13
BeaconState.currentEpochParticipation vc 300000 - serialize tree 299.14 us/op 303.44 us/op 0.99
BeaconState.inactivityScores vc 300000 - deserialize tree 23.447 ms/op 23.701 ms/op 0.99
BeaconState.inactivityScores vc 300000 - serialize tree 4.2847 ms/op 3.5339 ms/op 1.21
hashTreeRoot Attestation - struct 16.071 us/op 19.038 us/op 0.84
hashTreeRoot Attestation - tree 9.9510 us/op 9.1270 us/op 1.09
hashTreeRoot SignedAggregateAndProof - struct 18.155 us/op 24.357 us/op 0.75
hashTreeRoot SignedAggregateAndProof - tree 13.844 us/op 12.946 us/op 1.07
hashTreeRoot SyncCommitteeMessage - struct 4.5690 us/op 6.2560 us/op 0.73
hashTreeRoot SyncCommitteeMessage - tree 3.3550 us/op 3.1830 us/op 1.05
hashTreeRoot SignedContributionAndProof - struct 10.922 us/op 16.508 us/op 0.66
hashTreeRoot SignedContributionAndProof - tree 9.5510 us/op 8.8250 us/op 1.08
hashTreeRoot SignedBeaconBlock - struct 863.73 us/op 1.2691 ms/op 0.68
hashTreeRoot SignedBeaconBlock - tree 836.34 us/op 763.79 us/op 1.09
hashTreeRoot Validator - struct 5.0040 us/op 7.6760 us/op 0.65
hashTreeRoot Validator - tree 6.9930 us/op 6.6460 us/op 1.05
BeaconState vc 300000 - hashTreeRoot tree 2.2697 s/op 2.1538 s/op 1.05
BeaconState vc 300000 - batchHashTreeRoot tree 3.5243 s/op 3.5603 s/op 0.99
BeaconState.historicalRoots vc 300000 - hashTreeRoot tree 1.0310 us/op 1.0680 us/op 0.97
BeaconState.validators vc 300000 - hashTreeRoot tree 2.2417 s/op 2.1477 s/op 1.04
BeaconState.balances vc 300000 - hashTreeRoot tree 35.121 ms/op 32.837 ms/op 1.07
BeaconState.previousEpochParticipation vc 300000 - hashTreeRoot tree 4.3713 ms/op 4.2665 ms/op 1.02
BeaconState.currentEpochParticipation vc 300000 - hashTreeRoot tree 4.3725 ms/op 4.1114 ms/op 1.06
BeaconState.inactivityScores vc 300000 - hashTreeRoot tree 36.292 ms/op 33.516 ms/op 1.08
hash64 x18 10.528 us/op 9.2410 us/op 1.14
hashTwoObjects x18 8.7580 us/op 8.7090 us/op 1.01
hash64 x1740 981.62 us/op 877.38 us/op 1.12
hashTwoObjects x1740 840.63 us/op 813.89 us/op 1.03
hash64 x2700000 1.5239 s/op 1.3210 s/op 1.15
hashTwoObjects x2700000 1.2891 s/op 1.2764 s/op 1.01
get_exitEpoch - ContainerType 392.00 ns/op 369.00 ns/op 1.06
get_exitEpoch - ContainerNodeStructType 398.00 ns/op 388.00 ns/op 1.03
set_exitEpoch - ContainerType 417.00 ns/op 388.00 ns/op 1.07
set_exitEpoch - ContainerNodeStructType 406.00 ns/op 390.00 ns/op 1.04
get_pubkey - ContainerType 896.00 ns/op 1.4530 us/op 0.62
get_pubkey - ContainerNodeStructType 394.00 ns/op 368.00 ns/op 1.07
hashTreeRoot - ContainerType 688.00 ns/op 653.00 ns/op 1.05
hashTreeRoot - ContainerNodeStructType 703.00 ns/op 675.00 ns/op 1.04
createProof - ContainerType 6.8690 us/op 6.4280 us/op 1.07
createProof - ContainerNodeStructType 31.467 us/op 27.037 us/op 1.16
serialize - ContainerType 2.2430 us/op 2.1310 us/op 1.05
serialize - ContainerNodeStructType 1.6640 us/op 1.6440 us/op 1.01
set_exitEpoch_and_hashTreeRoot - ContainerType 2.8720 us/op 2.7420 us/op 1.05
set_exitEpoch_and_hashTreeRoot - ContainerNodeStructType 7.8600 us/op 8.0520 us/op 0.98
Array - for of 5.7050 us/op 7.0080 us/op 0.81
Array - for(;;) 5.7940 us/op 7.1820 us/op 0.81
basicListValue.readonlyValuesArray() 4.6925 ms/op 4.3980 ms/op 1.07
basicListValue.readonlyValuesArray() + loop all 4.7354 ms/op 4.6682 ms/op 1.01
compositeListValue.readonlyValuesArray() 30.250 ms/op 30.292 ms/op 1.00
compositeListValue.readonlyValuesArray() + loop all 25.141 ms/op 30.032 ms/op 0.84
Number64UintType - get balances list 4.1574 ms/op 4.2023 ms/op 0.99
Number64UintType - set balances list 10.024 ms/op 9.9380 ms/op 1.01
Number64UintType - get and increase 10 then set 40.436 ms/op 40.902 ms/op 0.99
Number64UintType - increase 10 using applyDelta 16.401 ms/op 16.656 ms/op 0.98
Number64UintType - increase 10 using applyDeltaInBatch 16.494 ms/op 17.329 ms/op 0.95
tree_newTreeFromUint64Deltas 16.806 ms/op 17.262 ms/op 0.97
unsafeUint8ArrayToTree 34.068 ms/op 33.193 ms/op 1.03
bitLength(50) 234.00 ns/op 224.00 ns/op 1.04
bitLengthStr(50) 230.00 ns/op 207.00 ns/op 1.11
bitLength(8000) 240.00 ns/op 211.00 ns/op 1.14
bitLengthStr(8000) 277.00 ns/op 258.00 ns/op 1.07
bitLength(250000) 235.00 ns/op 214.00 ns/op 1.10
bitLengthStr(250000) 297.00 ns/op 291.00 ns/op 1.02
merkleize 32 chunks 16.599 us/op
merkleizeBlocksBytes 32 chunks 3.6060 us/op
merkleizeBlockArray 32 chunks 6.5680 us/op
merkleize 128 chunks 68.051 us/op
merkleizeBlocksBytes 128 chunks 7.8410 us/op
merkleizeBlockArray 128 chunks 18.997 us/op
merkleize 512 chunks 269.86 us/op
merkleizeBlocksBytes 512 chunks 23.290 us/op
merkleizeBlockArray 512 chunks 63.457 us/op
merkleize 1024 chunks 548.28 us/op
merkleizeBlocksBytes 1024 chunks 43.461 us/op
merkleizeBlockArray 1024 chunks 122.14 us/op
floor - Math.floor (53) 1.2365 ns/op 1.2370 ns/op 1.00
floor - << 0 (53) 1.2364 ns/op 1.2561 ns/op 0.98
floor - Math.floor (512) 1.2373 ns/op 1.2367 ns/op 1.00
floor - << 0 (512) 1.2371 ns/op 1.2368 ns/op 1.00
fnIf(0) 1.5459 ns/op 1.5464 ns/op 1.00
fnSwitch(0) 2.1698 ns/op 2.1655 ns/op 1.00
fnObj(0) 1.5514 ns/op 1.5556 ns/op 1.00
fnArr(0) 1.5477 ns/op 1.5466 ns/op 1.00
fnIf(4) 2.1655 ns/op 2.1664 ns/op 1.00
fnSwitch(4) 2.1639 ns/op 2.1706 ns/op 1.00
fnObj(4) 1.5590 ns/op 1.5537 ns/op 1.00
fnArr(4) 1.5472 ns/op 1.5485 ns/op 1.00
fnIf(9) 3.1430 ns/op 3.0934 ns/op 1.02
fnSwitch(9) 2.1649 ns/op 2.1642 ns/op 1.00
fnObj(9) 1.5556 ns/op 1.5494 ns/op 1.00
fnArr(9) 1.5458 ns/op 1.5478 ns/op 1.00
Container {a,b,vec} - as struct x100000 123.83 us/op 123.81 us/op 1.00
Container {a,b,vec} - as tree x100000 347.46 us/op 341.54 us/op 1.02
Container {a,vec,b} - as struct x100000 155.12 us/op 154.75 us/op 1.00
Container {a,vec,b} - as tree x100000 371.40 us/op 372.31 us/op 1.00
get 2 props x1000000 - rawObject 313.20 us/op 309.63 us/op 1.01
get 2 props x1000000 - proxy 73.753 ms/op 73.740 ms/op 1.00
get 2 props x1000000 - customObj 316.07 us/op 310.61 us/op 1.02
Simple object binary -> struct 922.00 ns/op 689.00 ns/op 1.34
Simple object binary -> tree_backed 1.9290 us/op 1.3410 us/op 1.44
Simple object struct -> tree_backed 2.6550 us/op 1.9630 us/op 1.35
Simple object tree_backed -> struct 2.4920 us/op 1.8540 us/op 1.34
Simple object struct -> binary 1.2060 us/op 933.00 ns/op 1.29
Simple object tree_backed -> binary 1.7030 us/op 1.5790 us/op 1.08
aggregationBits binary -> struct 693.00 ns/op 599.00 ns/op 1.16
aggregationBits binary -> tree_backed 2.6430 us/op 2.4590 us/op 1.07
aggregationBits struct -> tree_backed 3.0780 us/op 2.8520 us/op 1.08
aggregationBits tree_backed -> struct 1.3250 us/op 1.2490 us/op 1.06
aggregationBits struct -> binary 836.00 ns/op 798.00 ns/op 1.05
aggregationBits tree_backed -> binary 1.1390 us/op 1.0790 us/op 1.06
List(uint8) 100000 binary -> struct 1.5429 ms/op 1.4515 ms/op 1.06
List(uint8) 100000 binary -> tree_backed 102.48 us/op 96.619 us/op 1.06
List(uint8) 100000 struct -> tree_backed 1.1763 ms/op 1.2458 ms/op 0.94
List(uint8) 100000 tree_backed -> struct 1.0756 ms/op 1.0206 ms/op 1.05
List(uint8) 100000 struct -> binary 1.0711 ms/op 1.0383 ms/op 1.03
List(uint8) 100000 tree_backed -> binary 98.319 us/op 96.603 us/op 1.02
List(uint64Number) 100000 binary -> struct 1.2780 ms/op 1.2164 ms/op 1.05
List(uint64Number) 100000 binary -> tree_backed 2.6376 ms/op 3.3339 ms/op 0.79
List(uint64Number) 100000 struct -> tree_backed 4.8437 ms/op 4.8758 ms/op 0.99
List(uint64Number) 100000 tree_backed -> struct 2.3359 ms/op 2.4317 ms/op 0.96
List(uint64Number) 100000 struct -> binary 1.4686 ms/op 1.5224 ms/op 0.96
List(uint64Number) 100000 tree_backed -> binary 1.0149 ms/op 932.48 us/op 1.09
List(Uint64Bigint) 100000 binary -> struct 3.6581 ms/op 3.7286 ms/op 0.98
List(Uint64Bigint) 100000 binary -> tree_backed 3.5130 ms/op 3.6569 ms/op 0.96
List(Uint64Bigint) 100000 struct -> tree_backed 5.7023 ms/op 5.9455 ms/op 0.96
List(Uint64Bigint) 100000 tree_backed -> struct 4.6874 ms/op 4.6791 ms/op 1.00
List(Uint64Bigint) 100000 struct -> binary 2.0829 ms/op 2.1101 ms/op 0.99
List(Uint64Bigint) 100000 tree_backed -> binary 1.0420 ms/op 944.46 us/op 1.10
Vector(Root) 100000 binary -> struct 32.738 ms/op 33.530 ms/op 0.98
Vector(Root) 100000 binary -> tree_backed 27.258 ms/op 31.990 ms/op 0.85
Vector(Root) 100000 struct -> tree_backed 43.089 ms/op 42.422 ms/op 1.02
Vector(Root) 100000 tree_backed -> struct 51.151 ms/op 48.678 ms/op 1.05
Vector(Root) 100000 struct -> binary 2.7122 ms/op 2.7790 ms/op 0.98
Vector(Root) 100000 tree_backed -> binary 9.7044 ms/op 8.6419 ms/op 1.12
List(Validator) 100000 binary -> struct 112.50 ms/op 97.990 ms/op 1.15
List(Validator) 100000 binary -> tree_backed 305.93 ms/op 307.53 ms/op 0.99
List(Validator) 100000 struct -> tree_backed 322.61 ms/op 324.32 ms/op 0.99
List(Validator) 100000 tree_backed -> struct 213.90 ms/op 215.93 ms/op 0.99
List(Validator) 100000 struct -> binary 27.714 ms/op 27.645 ms/op 1.00
List(Validator) 100000 tree_backed -> binary 117.37 ms/op 114.41 ms/op 1.03
List(Validator-NS) 100000 binary -> struct 97.554 ms/op 104.51 ms/op 0.93
List(Validator-NS) 100000 binary -> tree_backed 153.99 ms/op 151.08 ms/op 1.02
List(Validator-NS) 100000 struct -> tree_backed 199.66 ms/op 193.18 ms/op 1.03
List(Validator-NS) 100000 tree_backed -> struct 170.80 ms/op 168.29 ms/op 1.01
List(Validator-NS) 100000 struct -> binary 27.832 ms/op 28.062 ms/op 0.99
List(Validator-NS) 100000 tree_backed -> binary 33.339 ms/op 32.705 ms/op 1.02
get epochStatuses - MutableVector 107.61 us/op 108.56 us/op 0.99
get epochStatuses - ViewDU 206.99 us/op 202.53 us/op 1.02
set epochStatuses - ListTreeView 1.6984 ms/op 2.4053 ms/op 0.71
set epochStatuses - ListTreeView - set() 440.06 us/op 454.94 us/op 0.97
set epochStatuses - ListTreeView - commit() 552.33 us/op 556.67 us/op 0.99
bitstring 640.52 ns/op 645.94 ns/op 0.99
bit mask 13.718 ns/op 13.679 ns/op 1.00
struct - increase slot to 1000000 927.84 us/op 928.36 us/op 1.00
UintNumberType - increase slot to 1000000 21.375 ms/op 21.413 ms/op 1.00
UintBigintType - increase slot to 1000000 169.96 ms/op 169.23 ms/op 1.00
UintBigint8 x 100000 tree_deserialize 5.0308 ms/op 5.0115 ms/op 1.00
UintBigint8 x 100000 tree_serialize 1.1628 ms/op 1.1549 ms/op 1.01
UintBigint16 x 100000 tree_deserialize 4.7061 ms/op 5.0486 ms/op 0.93
UintBigint16 x 100000 tree_serialize 1.2846 ms/op 1.2855 ms/op 1.00
UintBigint32 x 100000 tree_deserialize 5.2145 ms/op 5.4270 ms/op 0.96
UintBigint32 x 100000 tree_serialize 1.2724 ms/op 1.2925 ms/op 0.98
UintBigint64 x 100000 tree_deserialize 5.8736 ms/op 5.6486 ms/op 1.04
UintBigint64 x 100000 tree_serialize 1.6884 ms/op 1.7295 ms/op 0.98
UintBigint8 x 100000 value_deserialize 433.36 us/op 433.66 us/op 1.00
UintBigint8 x 100000 value_serialize 707.88 us/op 718.01 us/op 0.99
UintBigint16 x 100000 value_deserialize 464.26 us/op 464.07 us/op 1.00
UintBigint16 x 100000 value_serialize 741.82 us/op 757.82 us/op 0.98
UintBigint32 x 100000 value_deserialize 433.86 us/op 438.91 us/op 0.99
UintBigint32 x 100000 value_serialize 780.41 us/op 793.66 us/op 0.98
UintBigint64 x 100000 value_deserialize 495.89 us/op 497.38 us/op 1.00
UintBigint64 x 100000 value_serialize 958.94 us/op 980.81 us/op 0.98
UintBigint8 x 100000 deserialize 3.1456 ms/op 3.3464 ms/op 0.94
UintBigint8 x 100000 serialize 1.5481 ms/op 1.9069 ms/op 0.81
UintBigint16 x 100000 deserialize 3.3370 ms/op 3.2067 ms/op 1.04
UintBigint16 x 100000 serialize 1.5934 ms/op 1.5547 ms/op 1.02
UintBigint32 x 100000 deserialize 3.3944 ms/op 3.2863 ms/op 1.03
UintBigint32 x 100000 serialize 2.8950 ms/op 2.8053 ms/op 1.03
UintBigint64 x 100000 deserialize 4.0570 ms/op 4.1025 ms/op 0.99
UintBigint64 x 100000 serialize 1.5532 ms/op 1.5834 ms/op 0.98
UintBigint128 x 100000 deserialize 5.6293 ms/op 5.5703 ms/op 1.01
UintBigint128 x 100000 serialize 15.249 ms/op 14.282 ms/op 1.07
UintBigint256 x 100000 deserialize 8.8624 ms/op 8.4595 ms/op 1.05
UintBigint256 x 100000 serialize 44.046 ms/op 42.168 ms/op 1.04
Slice from Uint8Array x25000 1.3108 ms/op 1.3334 ms/op 0.98
Slice from ArrayBuffer x25000 17.753 ms/op 16.053 ms/op 1.11
Slice from ArrayBuffer x25000 + new Uint8Array 17.292 ms/op 17.122 ms/op 1.01
Copy Uint8Array 100000 iterate 2.6241 ms/op 2.7648 ms/op 0.95
Copy Uint8Array 100000 slice 92.045 us/op 100.46 us/op 0.92
Copy Uint8Array 100000 Uint8Array.prototype.slice.call 91.696 us/op 100.26 us/op 0.91
Copy Buffer 100000 Uint8Array.prototype.slice.call 91.512 us/op 101.05 us/op 0.91
Copy Uint8Array 100000 slice + set 160.19 us/op 174.30 us/op 0.92
Copy Uint8Array 100000 subarray + set 91.488 us/op 97.907 us/op 0.93
Copy Uint8Array 100000 slice arrayBuffer 93.379 us/op 101.14 us/op 0.92
Uint64 deserialize 100000 - iterate Uint8Array 1.9001 ms/op 1.7850 ms/op 1.06
Uint64 deserialize 100000 - by Uint32A 2.0112 ms/op 1.9250 ms/op 1.04
Uint64 deserialize 100000 - by DataView.getUint32 x2 1.9043 ms/op 1.8148 ms/op 1.05
Uint64 deserialize 100000 - by DataView.getBigUint64 4.9956 ms/op 4.8491 ms/op 1.03
Uint64 deserialize 100000 - by byte 40.853 ms/op 40.511 ms/op 1.01

by benchmarkbot/action

@twoeths
Copy link
Contributor Author

twoeths commented Oct 21, 2024

tested this on feat1, see ChainSafe/lodestar#7171 (comment)
ready to review

@twoeths twoeths marked this pull request as ready for review October 21, 2024 03:18
@twoeths twoeths requested a review from a team as a code owner October 21, 2024 03:18
@twoeths twoeths mentioned this pull request Oct 21, 2024
Draft
@twoeths twoeths marked this pull request as draft October 31, 2024 03:39
@twoeths
Copy link
Contributor Author

twoeths commented Oct 31, 2024 •
edited
Loading

sha256 works in blocks, each is 64 bytes so perhaps it's more meaningful to reflect that for chunkBytesBuffer variable

also with holesky, there are 1.7M validators. For every 8 deposits we have to reallocate the whole 1.7M * 8 bytes = 13.6MB for BeaconState.balances which is not ideal. Need to instead allocate another 64 bytes in this case. This applies for all list types.

Update:

  • the hash of BeaconState.balances and everything inside BeaconState work through ViewDU so it's not revelant
  • it's more related to the hash of BeaconBlock, for example transaction data and lists like number of transactions

@twoeths twoeths force-pushed the te/improve_type_dot_hash_tree_root branch from ab6cf01 to 9e32c5c Compare November 9, 2024 02:16
@twoeths twoeths force-pushed the te/improve_type_dot_hash_tree_root branch from 9e32c5c to 7ed3ced Compare November 9, 2024 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
as-sha256 ssz status-do-not-merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@twoeths