Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improve type.hashTreeRoot() using batch #409

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

feat: improve type.hashTreeRoot() using batch #409

wants to merge 1 commit into from

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Oct 15, 2024

Motivation

  • improve type.hashTreeRoot() using batch

Description

  • instead of getRoots() and compute root from there, this PR implement getChunkBytes()
    • this compute root using merkleizeInto() which use batch there
    • reuse chunkBytesBuffer memory in type, almost no Uint8Array allocations in the middle
  • new hashTreeRootInto() api. This is needed in case consumers want to reuse memory allocation there
  • use allocUnsafe() of as-sha256 where it makes sense

cherry picked from #378

@twoeths twoeths mentioned this pull request Oct 15, 2024
Closed
@github-actions GitHub Actions
Copy link

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: b831aea Previous: b4beed2 Ratio
digestTwoHashObjects 50023 times 47.973 ms/op 47.563 ms/op 1.01
digest64 50023 times 53.494 ms/op 53.231 ms/op 1.00
digest 50023 times 54.690 ms/op 54.726 ms/op 1.00
input length 32 1.2200 us/op 1.2230 us/op 1.00
input length 64 1.3470 us/op 1.3700 us/op 0.98
input length 128 2.2900 us/op 2.2950 us/op 1.00
input length 256 3.3920 us/op 3.4350 us/op 0.99
input length 512 5.6060 us/op 5.5950 us/op 1.00
input length 1024 10.763 us/op 10.736 us/op 1.00
digest 1000000 times 870.12 ms/op 920.41 ms/op 0.95
hashObjectToByteArray 50023 times 1.4297 ms/op 1.4270 ms/op 1.00
byteArrayToHashObject 50023 times 2.4564 ms/op 2.5095 ms/op 0.98
digest64 200092 times 218.50 ms/op 219.30 ms/op 1.00
hash 200092 times using batchHash4UintArray64s 241.37 ms/op 237.94 ms/op 1.01
digest64HashObjects 200092 times 194.44 ms/op 191.47 ms/op 1.02
hash 200092 times using batchHash4HashObjectInputs 198.11 ms/op 199.39 ms/op 0.99
getGindicesAtDepth 4.3010 us/op 4.3020 us/op 1.00
iterateAtDepth 7.5510 us/op 7.6230 us/op 0.99
getGindexBits 463.00 ns/op 478.00 ns/op 0.97
gindexIterator 1.0600 us/op 1.0660 us/op 0.99
HashComputationLevel.push then loop 25.307 ms/op 25.273 ms/op 1.00
HashComputation[] push then loop 47.903 ms/op 47.331 ms/op 1.01
hash 2 Uint8Array 500000 times - as-sha256 539.56 ms/op 538.91 ms/op 1.00
hashTwoObjects 500000 times - as-sha256 502.90 ms/op 496.24 ms/op 1.01
executeHashComputations - as-sha256 49.439 ms/op 46.232 ms/op 1.07
hash 2 Uint8Array 500000 times - noble 1.1967 s/op 1.0572 s/op 1.13
hashTwoObjects 500000 times - noble 1.7428 s/op 1.4922 s/op 1.17
executeHashComputations - noble 42.173 ms/op 40.539 ms/op 1.04
hash 2 Uint8Array 500000 times - hashtree 230.71 ms/op 224.17 ms/op 1.03
hashTwoObjects 500000 times - hashtree 215.20 ms/op 218.75 ms/op 0.98
executeHashComputations - hashtree 10.975 ms/op 10.730 ms/op 1.02
getHashComputations 2.8841 ms/op 2.6966 ms/op 1.07
executeHashComputations 11.628 ms/op 11.069 ms/op 1.05
get root 17.793 ms/op 17.276 ms/op 1.03
getNodeH() x7812.5 avg hindex 11.957 us/op 12.511 us/op 0.96
getNodeH() x7812.5 index 0 6.4320 us/op 6.2320 us/op 1.03
getNodeH() x7812.5 index 7 6.2750 us/op 6.3240 us/op 0.99
getNodeH() x7812.5 index 7 with key array 6.2770 us/op 6.2210 us/op 1.01
new LeafNode() x7812.5 14.668 us/op 14.723 us/op 1.00
getHashComputations 250000 nodes 20.050 ms/op 19.422 ms/op 1.03
batchHash 250000 nodes 85.686 ms/op 88.547 ms/op 0.97
get root 250000 nodes 117.53 ms/op 118.56 ms/op 0.99
getHashComputations 500000 nodes 28.164 ms/op 28.728 ms/op 0.98
batchHash 500000 nodes 162.11 ms/op 142.17 ms/op 1.14
get root 500000 nodes 246.28 ms/op 238.10 ms/op 1.03
getHashComputations 1000000 nodes 53.702 ms/op 70.132 ms/op 0.77
batchHash 1000000 nodes 287.17 ms/op 328.90 ms/op 0.87
get root 1000000 nodes 482.49 ms/op 477.38 ms/op 1.01
multiproof - depth 15, 1 requested leaves 8.5940 us/op 8.1730 us/op 1.05
tree offset multiproof - depth 15, 1 requested leaves 19.224 us/op 18.112 us/op 1.06
compact multiproof - depth 15, 1 requested leaves 3.5700 us/op 3.4020 us/op 1.05
multiproof - depth 15, 2 requested leaves 11.452 us/op 11.860 us/op 0.97
tree offset multiproof - depth 15, 2 requested leaves 20.965 us/op 21.715 us/op 0.97
compact multiproof - depth 15, 2 requested leaves 3.4550 us/op 3.4780 us/op 0.99
multiproof - depth 15, 3 requested leaves 16.059 us/op 16.901 us/op 0.95
tree offset multiproof - depth 15, 3 requested leaves 27.863 us/op 27.903 us/op 1.00
compact multiproof - depth 15, 3 requested leaves 4.0640 us/op 4.2200 us/op 0.96
multiproof - depth 15, 4 requested leaves 21.839 us/op 21.786 us/op 1.00
tree offset multiproof - depth 15, 4 requested leaves 33.948 us/op 35.400 us/op 0.96
compact multiproof - depth 15, 4 requested leaves 4.7330 us/op 5.0270 us/op 0.94
packedRootsBytesToLeafNodes bytes 4000 offset 0 1.7840 us/op 2.0580 us/op 0.87
packedRootsBytesToLeafNodes bytes 4000 offset 1 1.7880 us/op 2.0460 us/op 0.87
packedRootsBytesToLeafNodes bytes 4000 offset 2 1.7720 us/op 2.0460 us/op 0.87
packedRootsBytesToLeafNodes bytes 4000 offset 3 1.7790 us/op 2.0170 us/op 0.88
subtreeFillToContents depth 40 count 250000 42.703 ms/op 39.406 ms/op 1.08
setRoot - gindexBitstring 9.9167 ms/op 9.4713 ms/op 1.05
setRoot - gindex 10.488 ms/op 9.7799 ms/op 1.07
getRoot - gindexBitstring 2.7060 ms/op 2.3727 ms/op 1.14
getRoot - gindex 3.2586 ms/op 3.2045 ms/op 1.02
getHashObject then setHashObject 11.538 ms/op 10.076 ms/op 1.15
setNodeWithFn 9.1554 ms/op 7.6212 ms/op 1.20
getNodeAtDepth depth 0 x100000 1.1131 ms/op 1.1147 ms/op 1.00
setNodeAtDepth depth 0 x100000 2.7461 ms/op 2.4581 ms/op 1.12
getNodesAtDepth depth 0 x100000 1.0792 ms/op 1.0597 ms/op 1.02
setNodesAtDepth depth 0 x100000 1.5153 ms/op 1.5213 ms/op 1.00
getNodeAtDepth depth 1 x100000 1.2133 ms/op 1.1774 ms/op 1.03
setNodeAtDepth depth 1 x100000 6.0038 ms/op 5.1523 ms/op 1.17
getNodesAtDepth depth 1 x100000 1.1777 ms/op 1.1761 ms/op 1.00
setNodesAtDepth depth 1 x100000 4.8465 ms/op 4.3912 ms/op 1.10
getNodeAtDepth depth 2 x100000 1.4980 ms/op 1.4585 ms/op 1.03
setNodeAtDepth depth 2 x100000 10.360 ms/op 8.9137 ms/op 1.16
getNodesAtDepth depth 2 x100000 19.029 ms/op 17.966 ms/op 1.06
setNodesAtDepth depth 2 x100000 15.954 ms/op 12.902 ms/op 1.24
tree.getNodesAtDepth - gindexes 8.3864 ms/op 7.2954 ms/op 1.15
tree.getNodesAtDepth - push all nodes 2.2577 ms/op 1.8712 ms/op 1.21
tree.getNodesAtDepth - navigation 232.48 us/op 233.54 us/op 1.00
tree.setNodesAtDepth - indexes 430.43 us/op 389.71 us/op 1.10
set at depth 8 506.00 ns/op 455.00 ns/op 1.11
set at depth 16 668.00 ns/op 584.00 ns/op 1.14
set at depth 32 1.0760 us/op 931.00 ns/op 1.16
iterateNodesAtDepth 8 256 14.071 us/op 13.033 us/op 1.08
getNodesAtDepth 8 256 3.6210 us/op 3.3670 us/op 1.08
iterateNodesAtDepth 16 65536 4.4415 ms/op 4.2538 ms/op 1.04
getNodesAtDepth 16 65536 1.9499 ms/op 1.5348 ms/op 1.27
iterateNodesAtDepth 32 250000 17.869 ms/op 15.465 ms/op 1.16
getNodesAtDepth 32 250000 4.2769 ms/op 4.2977 ms/op 1.00
iterateNodesAtDepth 40 250000 15.206 ms/op 15.075 ms/op 1.01
getNodesAtDepth 40 250000 4.3072 ms/op 4.2544 ms/op 1.01
250000 validators root getter 119.33 ms/op 117.93 ms/op 1.01
250000 validators batchHash() 80.607 ms/op 81.978 ms/op 0.98
250000 validators hashComputations 17.515 ms/op 17.881 ms/op 0.98
bitlist bytes to struct (120,90) 1.0190 us/op 821.00 ns/op 1.24
bitlist bytes to tree (120,90) 3.9270 us/op 3.1870 us/op 1.23
bitlist bytes to struct (2048,2048) 1.4570 us/op 1.1530 us/op 1.26
bitlist bytes to tree (2048,2048) 4.6050 us/op 3.9570 us/op 1.16
ByteListType - deserialize 8.2210 ms/op 7.1776 ms/op 1.15
BasicListType - deserialize 18.153 ms/op 15.106 ms/op 1.20
ByteListType - serialize 8.2373 ms/op 7.3920 ms/op 1.11
BasicListType - serialize 11.503 ms/op 10.336 ms/op 1.11
BasicListType - tree_convertToStruct 30.721 ms/op 26.430 ms/op 1.16
List[uint8, 68719476736] len 300000 ViewDU.getAll() + iterate 4.7838 ms/op 4.3451 ms/op 1.10
List[uint8, 68719476736] len 300000 ViewDU.get(i) 3.9915 ms/op 4.0176 ms/op 0.99
Array.push len 300000 empty Array - number 6.4382 ms/op 5.7115 ms/op 1.13
Array.set len 300000 from new Array - number 1.7074 ms/op 1.8534 ms/op 0.92
Array.set len 300000 - number 5.2609 ms/op 5.6512 ms/op 0.93
Uint8Array.set len 300000 376.88 us/op 367.16 us/op 1.03
Uint32Array.set len 300000 449.67 us/op 420.72 us/op 1.07
Container({a: uint8, b: uint8}) getViewDU x300000 45.555 ms/op 45.041 ms/op 1.01
ContainerNodeStruct({a: uint8, b: uint8}) getViewDU x300000 10.693 ms/op 10.702 ms/op 1.00
List(Container) len 300000 ViewDU.getAllReadonly() + iterate 204.98 ms/op 198.38 ms/op 1.03
List(Container) len 300000 ViewDU.getAllReadonlyValues() + iterate 242.91 ms/op 230.48 ms/op 1.05
List(Container) len 300000 ViewDU.get(i) 6.3374 ms/op 6.3351 ms/op 1.00
List(Container) len 300000 ViewDU.getReadonly(i) 6.2274 ms/op 6.2013 ms/op 1.00
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonly() + iterate 38.665 ms/op 40.398 ms/op 0.96
List(ContainerNodeStruct) len 300000 ViewDU.getAllReadonlyValues() + iterate 4.8451 ms/op 5.0758 ms/op 0.95
List(ContainerNodeStruct) len 300000 ViewDU.get(i) 6.2359 ms/op 5.9316 ms/op 1.05
List(ContainerNodeStruct) len 300000 ViewDU.getReadonly(i) 6.1302 ms/op 5.8203 ms/op 1.05
Array.push len 300000 empty Array - object 5.9930 ms/op 5.8913 ms/op 1.02
Array.set len 300000 from new Array - object 2.1015 ms/op 2.0034 ms/op 1.05
Array.set len 300000 - object 6.0382 ms/op 6.2420 ms/op 0.97
cachePermanentRootStruct no cache 3.6020 us/op 5.1110 us/op 0.70
cachePermanentRootStruct with cache 207.00 ns/op 211.00 ns/op 0.98
epochParticipation len 250000 rws 7813 2.1987 ms/op 2.1637 ms/op 1.02
Deneb BeaconBlock.hashTreeRoot(), numTransaction=200 4.1141 ms/op
BeaconState ViewDU hashTreeRoot() vc=200000 104.74 ms/op 101.07 ms/op 1.04
BeaconState ViewDU recursive hash - commit step vc=200000 4.4910 ms/op 3.5608 ms/op 1.26
BeaconState ViewDU validator tree creation vc=10000 35.613 ms/op 33.278 ms/op 1.07
BeaconState ViewDU batchHashTreeRoot vc=200000 94.532 ms/op 89.382 ms/op 1.06
BeaconState ViewDU hashTreeRoot - commit step vc=200000 83.422 ms/op 80.229 ms/op 1.04
BeaconState ViewDU hashTreeRoot - hash step vc=200000 15.390 ms/op 14.314 ms/op 1.08
deserialize Attestation - tree 4.2040 us/op 3.9630 us/op 1.06
deserialize Attestation - struct 1.8610 us/op 1.7580 us/op 1.06
deserialize SignedAggregateAndProof - tree 3.9780 us/op 3.6310 us/op 1.10
deserialize SignedAggregateAndProof - struct 3.1910 us/op 2.9160 us/op 1.09
deserialize SyncCommitteeMessage - tree 1.1170 us/op 1.0250 us/op 1.09
deserialize SyncCommitteeMessage - struct 1.0940 us/op 1.0620 us/op 1.03
deserialize SignedContributionAndProof - tree 2.1380 us/op 2.0080 us/op 1.06
deserialize SignedContributionAndProof - struct 2.3150 us/op 2.1560 us/op 1.07
deserialize SignedBeaconBlock - tree 217.24 us/op 207.08 us/op 1.05
deserialize SignedBeaconBlock - struct 121.63 us/op 114.47 us/op 1.06
BeaconState vc 300000 - deserialize tree 635.95 ms/op 585.06 ms/op 1.09
BeaconState vc 300000 - serialize tree 154.85 ms/op 136.42 ms/op 1.14
BeaconState.historicalRoots vc 300000 - deserialize tree 770.00 ns/op 691.00 ns/op 1.11
BeaconState.historicalRoots vc 300000 - serialize tree 635.00 ns/op 650.00 ns/op 0.98
BeaconState.validators vc 300000 - deserialize tree 590.92 ms/op 566.56 ms/op 1.04
BeaconState.validators vc 300000 - serialize tree 118.91 ms/op 97.294 ms/op 1.22
BeaconState.balances vc 300000 - deserialize tree 22.892 ms/op 23.164 ms/op 0.99
BeaconState.balances vc 300000 - serialize tree 4.3301 ms/op 3.5235 ms/op 1.23
BeaconState.previousEpochParticipation vc 300000 - deserialize tree 535.84 us/op 334.74 us/op 1.60
BeaconState.previousEpochParticipation vc 300000 - serialize tree 281.23 us/op 271.56 us/op 1.04
BeaconState.currentEpochParticipation vc 300000 - deserialize tree 517.80 us/op 335.55 us/op 1.54
BeaconState.currentEpochParticipation vc 300000 - serialize tree 282.37 us/op 274.71 us/op 1.03
BeaconState.inactivityScores vc 300000 - deserialize tree 24.473 ms/op 23.268 ms/op 1.05
BeaconState.inactivityScores vc 300000 - serialize tree 5.5770 ms/op 3.3465 ms/op 1.67
hashTreeRoot Attestation - struct 13.301 us/op 19.610 us/op 0.68
hashTreeRoot Attestation - tree 8.8630 us/op 9.0690 us/op 0.98
hashTreeRoot SignedAggregateAndProof - struct 17.751 us/op 24.121 us/op 0.74
hashTreeRoot SignedAggregateAndProof - tree 13.184 us/op 12.891 us/op 1.02
hashTreeRoot SyncCommitteeMessage - struct 4.3770 us/op 6.0570 us/op 0.72
hashTreeRoot SyncCommitteeMessage - tree 3.1650 us/op 3.1200 us/op 1.01
hashTreeRoot SignedContributionAndProof - struct 10.532 us/op 16.609 us/op 0.63
hashTreeRoot SignedContributionAndProof - tree 8.9310 us/op 8.8530 us/op 1.01
hashTreeRoot SignedBeaconBlock - struct 986.79 us/op 1.2562 ms/op 0.79
hashTreeRoot SignedBeaconBlock - tree 791.62 us/op 759.84 us/op 1.04
hashTreeRoot Validator - struct 5.3640 us/op 7.5450 us/op 0.71
hashTreeRoot Validator - tree 7.2440 us/op 6.3470 us/op 1.14
BeaconState vc 300000 - hashTreeRoot tree 2.2079 s/op 2.0804 s/op 1.06
BeaconState vc 300000 - batchHashTreeRoot tree 3.7860 s/op 3.2767 s/op 1.16
BeaconState.historicalRoots vc 300000 - hashTreeRoot tree 1.0700 us/op 950.00 ns/op 1.13
BeaconState.validators vc 300000 - hashTreeRoot tree 2.2750 s/op 2.0641 s/op 1.10
BeaconState.balances vc 300000 - hashTreeRoot tree 35.614 ms/op 32.841 ms/op 1.08
BeaconState.previousEpochParticipation vc 300000 - hashTreeRoot tree 4.2177 ms/op 4.2966 ms/op 0.98
BeaconState.currentEpochParticipation vc 300000 - hashTreeRoot tree 4.1183 ms/op 4.1245 ms/op 1.00
BeaconState.inactivityScores vc 300000 - hashTreeRoot tree 37.920 ms/op 33.347 ms/op 1.14
hash64 x18 10.686 us/op 9.0690 us/op 1.18
hashTwoObjects x18 8.6660 us/op 8.7140 us/op 0.99
hash64 x1740 1.0021 ms/op 820.40 us/op 1.22
hashTwoObjects x1740 798.69 us/op 822.67 us/op 0.97
hash64 x2700000 1.5509 s/op 1.2905 s/op 1.20
hashTwoObjects x2700000 1.2472 s/op 1.2732 s/op 0.98
get_exitEpoch - ContainerType 294.00 ns/op 366.00 ns/op 0.80
get_exitEpoch - ContainerNodeStructType 316.00 ns/op 363.00 ns/op 0.87
set_exitEpoch - ContainerType 316.00 ns/op 381.00 ns/op 0.83
set_exitEpoch - ContainerNodeStructType 323.00 ns/op 372.00 ns/op 0.87
get_pubkey - ContainerType 999.00 ns/op 1.3880 us/op 0.72
get_pubkey - ContainerNodeStructType 299.00 ns/op 361.00 ns/op 0.83
hashTreeRoot - ContainerType 540.00 ns/op 614.00 ns/op 0.88
hashTreeRoot - ContainerNodeStructType 567.00 ns/op 645.00 ns/op 0.88
createProof - ContainerType 5.4390 us/op 6.3600 us/op 0.86
createProof - ContainerNodeStructType 26.079 us/op 24.930 us/op 1.05
serialize - ContainerType 2.1920 us/op 1.9060 us/op 1.15
serialize - ContainerNodeStructType 1.7640 us/op 1.4160 us/op 1.25
set_exitEpoch_and_hashTreeRoot - ContainerType 2.8510 us/op 2.7860 us/op 1.02
set_exitEpoch_and_hashTreeRoot - ContainerNodeStructType 7.2480 us/op 7.1890 us/op 1.01
Array - for of 5.5300 us/op 6.5340 us/op 0.85
Array - for(;;) 5.5230 us/op 6.3540 us/op 0.87
basicListValue.readonlyValuesArray() 4.3629 ms/op 4.1882 ms/op 1.04
basicListValue.readonlyValuesArray() + loop all 4.3628 ms/op 4.3806 ms/op 1.00
compositeListValue.readonlyValuesArray() 26.994 ms/op 30.292 ms/op 0.89
compositeListValue.readonlyValuesArray() + loop all 30.442 ms/op 29.651 ms/op 1.03
Number64UintType - get balances list 4.8852 ms/op 5.5447 ms/op 0.88
Number64UintType - set balances list 9.9975 ms/op 10.067 ms/op 0.99
Number64UintType - get and increase 10 then set 42.170 ms/op 39.157 ms/op 1.08
Number64UintType - increase 10 using applyDelta 17.627 ms/op 16.529 ms/op 1.07
Number64UintType - increase 10 using applyDeltaInBatch 17.458 ms/op 17.171 ms/op 1.02
tree_newTreeFromUint64Deltas 17.945 ms/op 16.725 ms/op 1.07
unsafeUint8ArrayToTree 32.114 ms/op 32.415 ms/op 0.99
bitLength(50) 247.00 ns/op 222.00 ns/op 1.11
bitLengthStr(50) 220.00 ns/op 210.00 ns/op 1.05
bitLength(8000) 240.00 ns/op 218.00 ns/op 1.10
bitLengthStr(8000) 269.00 ns/op 251.00 ns/op 1.07
bitLength(250000) 240.00 ns/op 220.00 ns/op 1.09
bitLengthStr(250000) 299.00 ns/op 289.00 ns/op 1.03
merkleizeInto 4 chunks 1.4310 us/op
merkleize 4 chunks 2.1200 us/op
merkleizeInto 8 chunks 1.9260 us/op
merkleize 8 chunks 4.3970 us/op
merkleizeInto 16 chunks 2.6740 us/op
merkleize 16 chunks 8.8380 us/op
merkleizeInto 32 chunks 3.6660 us/op
merkleize 32 chunks 17.465 us/op
floor - Math.floor (53) 1.2373 ns/op 1.2430 ns/op 1.00
floor - << 0 (53) 1.2400 ns/op 1.2366 ns/op 1.00
floor - Math.floor (512) 1.2377 ns/op 1.2373 ns/op 1.00
floor - << 0 (512) 1.2372 ns/op 1.2400 ns/op 1.00
fnIf(0) 1.5579 ns/op 1.5538 ns/op 1.00
fnSwitch(0) 2.1657 ns/op 2.1668 ns/op 1.00
fnObj(0) 1.5529 ns/op 1.5568 ns/op 1.00
fnArr(0) 1.5470 ns/op 1.5465 ns/op 1.00
fnIf(4) 2.1723 ns/op 2.1743 ns/op 1.00
fnSwitch(4) 2.1750 ns/op 2.1669 ns/op 1.00
fnObj(4) 1.5479 ns/op 1.5487 ns/op 1.00
fnArr(4) 1.5475 ns/op 1.5480 ns/op 1.00
fnIf(9) 3.1105 ns/op 3.0924 ns/op 1.01
fnSwitch(9) 2.1994 ns/op 2.1665 ns/op 1.02
fnObj(9) 1.5471 ns/op 1.5475 ns/op 1.00
fnArr(9) 1.5620 ns/op 1.5461 ns/op 1.01
Container {a,b,vec} - as struct x100000 124.02 us/op 123.84 us/op 1.00
Container {a,b,vec} - as tree x100000 340.30 us/op 340.09 us/op 1.00
Container {a,vec,b} - as struct x100000 155.04 us/op 154.32 us/op 1.00
Container {a,vec,b} - as tree x100000 371.31 us/op 371.18 us/op 1.00
get 2 props x1000000 - rawObject 310.29 us/op 309.88 us/op 1.00
get 2 props x1000000 - proxy 73.674 ms/op 72.741 ms/op 1.01
get 2 props x1000000 - customObj 315.03 us/op 307.92 us/op 1.02
Simple object binary -> struct 798.00 ns/op 567.00 ns/op 1.41
Simple object binary -> tree_backed 1.6320 us/op 986.00 ns/op 1.66
Simple object struct -> tree_backed 2.2860 us/op 1.5440 us/op 1.48
Simple object tree_backed -> struct 1.9010 us/op 1.4670 us/op 1.30
Simple object struct -> binary 946.00 ns/op 789.00 ns/op 1.20
Simple object tree_backed -> binary 1.5540 us/op 1.2600 us/op 1.23
aggregationBits binary -> struct 571.00 ns/op 444.00 ns/op 1.29
aggregationBits binary -> tree_backed 2.4100 us/op 1.9550 us/op 1.23
aggregationBits struct -> tree_backed 2.8240 us/op 2.3230 us/op 1.22
aggregationBits tree_backed -> struct 1.2150 us/op 926.00 ns/op 1.31
aggregationBits struct -> binary 784.00 ns/op 686.00 ns/op 1.14
aggregationBits tree_backed -> binary 1.0600 us/op 871.00 ns/op 1.22
List(uint8) 100000 binary -> struct 1.3395 ms/op 1.6812 ms/op 0.80
List(uint8) 100000 binary -> tree_backed 88.883 us/op 93.101 us/op 0.95
List(uint8) 100000 struct -> tree_backed 1.1788 ms/op 1.1024 ms/op 1.07
List(uint8) 100000 tree_backed -> struct 1.0392 ms/op 1.0546 ms/op 0.99
List(uint8) 100000 struct -> binary 1.0334 ms/op 989.83 us/op 1.04
List(uint8) 100000 tree_backed -> binary 92.547 us/op 89.572 us/op 1.03
List(uint64Number) 100000 binary -> struct 1.2879 ms/op 1.1648 ms/op 1.11
List(uint64Number) 100000 binary -> tree_backed 3.6211 ms/op 2.5600 ms/op 1.41
List(uint64Number) 100000 struct -> tree_backed 5.5800 ms/op 4.0852 ms/op 1.37
List(uint64Number) 100000 tree_backed -> struct 2.3130 ms/op 2.0743 ms/op 1.12
List(uint64Number) 100000 struct -> binary 1.3993 ms/op 1.3325 ms/op 1.05
List(uint64Number) 100000 tree_backed -> binary 871.59 us/op 853.53 us/op 1.02
List(Uint64Bigint) 100000 binary -> struct 3.5055 ms/op 3.5539 ms/op 0.99
List(Uint64Bigint) 100000 binary -> tree_backed 3.4988 ms/op 3.2431 ms/op 1.08
List(Uint64Bigint) 100000 struct -> tree_backed 5.7317 ms/op 5.5215 ms/op 1.04
List(Uint64Bigint) 100000 tree_backed -> struct 4.9643 ms/op 4.5138 ms/op 1.10
List(Uint64Bigint) 100000 struct -> binary 2.0656 ms/op 2.0407 ms/op 1.01
List(Uint64Bigint) 100000 tree_backed -> binary 933.00 us/op 934.11 us/op 1.00
Vector(Root) 100000 binary -> struct 28.277 ms/op 32.244 ms/op 0.88
Vector(Root) 100000 binary -> tree_backed 34.434 ms/op 27.173 ms/op 1.27
Vector(Root) 100000 struct -> tree_backed 42.085 ms/op 45.955 ms/op 0.92
Vector(Root) 100000 tree_backed -> struct 46.918 ms/op 48.127 ms/op 0.97
Vector(Root) 100000 struct -> binary 2.6716 ms/op 2.6239 ms/op 1.02
Vector(Root) 100000 tree_backed -> binary 9.2385 ms/op 8.2375 ms/op 1.12
List(Validator) 100000 binary -> struct 100.05 ms/op 104.93 ms/op 0.95
List(Validator) 100000 binary -> tree_backed 305.46 ms/op 285.76 ms/op 1.07
List(Validator) 100000 struct -> tree_backed 314.30 ms/op 309.56 ms/op 1.02
List(Validator) 100000 tree_backed -> struct 206.27 ms/op 210.31 ms/op 0.98
List(Validator) 100000 struct -> binary 28.845 ms/op 26.723 ms/op 1.08
List(Validator) 100000 tree_backed -> binary 117.99 ms/op 115.26 ms/op 1.02
List(Validator-NS) 100000 binary -> struct 104.63 ms/op 98.543 ms/op 1.06
List(Validator-NS) 100000 binary -> tree_backed 155.06 ms/op 144.42 ms/op 1.07
List(Validator-NS) 100000 struct -> tree_backed 197.45 ms/op 185.09 ms/op 1.07
List(Validator-NS) 100000 tree_backed -> struct 170.47 ms/op 165.67 ms/op 1.03
List(Validator-NS) 100000 struct -> binary 28.615 ms/op 27.056 ms/op 1.06
List(Validator-NS) 100000 tree_backed -> binary 34.070 ms/op 31.887 ms/op 1.07
get epochStatuses - MutableVector 94.910 us/op 106.38 us/op 0.89
get epochStatuses - ViewDU 203.84 us/op 203.15 us/op 1.00
set epochStatuses - ListTreeView 1.7334 ms/op 2.3891 ms/op 0.73
set epochStatuses - ListTreeView - set() 459.20 us/op 459.01 us/op 1.00
set epochStatuses - ListTreeView - commit() 587.82 us/op 555.56 us/op 1.06
bitstring 642.30 ns/op 641.14 ns/op 1.00
bit mask 13.929 ns/op 13.521 ns/op 1.03
struct - increase slot to 1000000 928.26 us/op 927.98 us/op 1.00
UintNumberType - increase slot to 1000000 21.976 ms/op 21.987 ms/op 1.00
UintBigintType - increase slot to 1000000 162.98 ms/op 159.36 ms/op 1.02
UintBigint8 x 100000 tree_deserialize 4.6952 ms/op 4.6560 ms/op 1.01
UintBigint8 x 100000 tree_serialize 1.1029 ms/op 1.0943 ms/op 1.01
UintBigint16 x 100000 tree_deserialize 4.7232 ms/op 4.7401 ms/op 1.00
UintBigint16 x 100000 tree_serialize 1.2226 ms/op 1.2154 ms/op 1.01
UintBigint32 x 100000 tree_deserialize 4.9519 ms/op 5.0010 ms/op 0.99
UintBigint32 x 100000 tree_serialize 1.2520 ms/op 1.2239 ms/op 1.02
UintBigint64 x 100000 tree_deserialize 5.1737 ms/op 5.2174 ms/op 0.99
UintBigint64 x 100000 tree_serialize 1.6115 ms/op 1.5732 ms/op 1.02
UintBigint8 x 100000 value_deserialize 433.55 us/op 521.80 us/op 0.83
UintBigint8 x 100000 value_serialize 669.98 us/op 666.63 us/op 1.01
UintBigint16 x 100000 value_deserialize 464.35 us/op 462.13 us/op 1.00
UintBigint16 x 100000 value_serialize 731.88 us/op 724.99 us/op 1.01
UintBigint32 x 100000 value_deserialize 434.93 us/op 433.07 us/op 1.00
UintBigint32 x 100000 value_serialize 706.12 us/op 698.01 us/op 1.01
UintBigint64 x 100000 value_deserialize 498.03 us/op 495.96 us/op 1.00
UintBigint64 x 100000 value_serialize 874.09 us/op 874.42 us/op 1.00
UintBigint8 x 100000 deserialize 3.1422 ms/op 2.9727 ms/op 1.06
UintBigint8 x 100000 serialize 1.5225 ms/op 1.6040 ms/op 0.95
UintBigint16 x 100000 deserialize 3.4170 ms/op 3.0156 ms/op 1.13
UintBigint16 x 100000 serialize 1.5606 ms/op 1.6335 ms/op 0.96
UintBigint32 x 100000 deserialize 3.4474 ms/op 3.0710 ms/op 1.12
UintBigint32 x 100000 serialize 2.8541 ms/op 2.7979 ms/op 1.02
UintBigint64 x 100000 deserialize 4.2550 ms/op 3.9455 ms/op 1.08
UintBigint64 x 100000 serialize 1.5662 ms/op 1.5557 ms/op 1.01
UintBigint128 x 100000 deserialize 5.8637 ms/op 5.4218 ms/op 1.08
UintBigint128 x 100000 serialize 15.366 ms/op 13.879 ms/op 1.11
UintBigint256 x 100000 deserialize 9.0728 ms/op 8.0535 ms/op 1.13
UintBigint256 x 100000 serialize 44.176 ms/op 41.441 ms/op 1.07
Slice from Uint8Array x25000 1.3394 ms/op 1.2979 ms/op 1.03
Slice from ArrayBuffer x25000 16.029 ms/op 15.006 ms/op 1.07
Slice from ArrayBuffer x25000 + new Uint8Array 17.131 ms/op 16.000 ms/op 1.07
Copy Uint8Array 100000 iterate 1.6516 ms/op 1.6460 ms/op 1.00
Copy Uint8Array 100000 slice 125.42 us/op 116.05 us/op 1.08
Copy Uint8Array 100000 Uint8Array.prototype.slice.call 122.81 us/op 117.91 us/op 1.04
Copy Buffer 100000 Uint8Array.prototype.slice.call 124.78 us/op 117.93 us/op 1.06
Copy Uint8Array 100000 slice + set 220.04 us/op 178.57 us/op 1.23
Copy Uint8Array 100000 subarray + set 126.59 us/op 117.59 us/op 1.08
Copy Uint8Array 100000 slice arrayBuffer 126.61 us/op 116.92 us/op 1.08
Uint64 deserialize 100000 - iterate Uint8Array 1.9243 ms/op 1.8024 ms/op 1.07
Uint64 deserialize 100000 - by Uint32A 1.9020 ms/op 1.7828 ms/op 1.07
Uint64 deserialize 100000 - by DataView.getUint32 x2 1.9197 ms/op 1.7841 ms/op 1.08
Uint64 deserialize 100000 - by DataView.getBigUint64 5.8380 ms/op 5.2553 ms/op 1.11
Uint64 deserialize 100000 - by byte 39.918 ms/op 39.228 ms/op 1.02

by benchmarkbot/action

@twoeths
Copy link
Contributor Author

twoeths commented Oct 21, 2024

tested this on feat1, see ChainSafe/lodestar#7171 (comment)
ready to review

@twoeths twoeths marked this pull request as ready for review October 21, 2024 03:18
@twoeths twoeths requested a review from a team as a code owner October 21, 2024 03:18
@twoeths twoeths mentioned this pull request Oct 21, 2024
Draft
@twoeths twoeths marked this pull request as draft October 31, 2024 03:39
@twoeths
Copy link
Contributor Author

twoeths commented Oct 31, 2024 •
edited
Loading

sha256 works in blocks, each is 64 bytes so perhaps it's more meaningful to reflect that for chunkBytesBuffer variable

also with holesky, there are 1.7M validators. For every 8 deposits we have to reallocate the whole 1.7M * 8 bytes = 13.6MB for BeaconState.balances which is not ideal. Need to instead allocate another 64 bytes in this case. This applies for all list types.

Update:

  • the hash of BeaconState.balances and everything inside BeaconState work through ViewDU so it's not revelant
  • it's more related to the hash of BeaconBlock, for example transaction data and lists like number of transactions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
as-sha256 ssz status-do-not-merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@twoeths