Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: archive state using BufferPool if provided #7042

Merged
merged 2 commits into from
Sep 5, 2024
Merged

feat: archive state using BufferPool if provided #7042

merged 2 commits into from
Sep 5, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Aug 21, 2024 •
edited
Loading

Motivation

  • When n-historical state flag is lived, we have a BufferPool. We should use it to archive state better
  • It's also a prerequisite for state-diff PR

Description

  • enhance BufferPool to have metric labels based on source
  • publish BufferPool on BeaconChain based on n-historical flag
  • refactor serializeState() api
  • bot state archiver and PersistentCheckpointsCache use serializeState() api
  • remove unused Grafana metrics

Follow up

  • Update Grafana panels
  • Move serializeState.ts and initState.ts to new chain/states if we want to

Closes #6935

cc @nazarhussain

@twoeths twoeths mentioned this pull request Aug 21, 2024
Open
@codecov Codecov
Copy link

codecov bot commented Aug 21, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 49.24%. Comparing base (6f470f8) to head (15434e0).
Report is 86 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7042      +/-   ##
============================================
- Coverage     49.25%   49.24%   -0.01%     
============================================
  Files           578      578              
  Lines         37443    37441       -2     
  Branches       2172     2171       -1     
============================================
- Hits          18441    18439       -2     
+ Misses        18962    18961       -1     
- Partials         40       41       +1

@github-actions GitHub Actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9286541 Previous: 4ea7edd Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9331 ms/op 2.6055 ms/op 0.74
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 49.538 us/op 80.209 us/op 0.62
BLS verify - blst 871.30 us/op 1.1234 ms/op 0.78
BLS verifyMultipleSignatures 3 - blst 1.5697 ms/op 1.7285 ms/op 0.91
BLS verifyMultipleSignatures 8 - blst 2.2197 ms/op 2.4733 ms/op 0.90
BLS verifyMultipleSignatures 32 - blst 5.9469 ms/op 7.4787 ms/op 0.80
BLS verifyMultipleSignatures 64 - blst 11.004 ms/op 12.507 ms/op 0.88
BLS verifyMultipleSignatures 128 - blst 18.549 ms/op 21.076 ms/op 0.88
BLS deserializing 10000 signatures 694.95 ms/op 792.81 ms/op 0.88
BLS deserializing 100000 signatures 7.8825 s/op 7.8729 s/op 1.00
BLS verifyMultipleSignatures - same message - 3 - blst 1.0874 ms/op 1.1819 ms/op 0.92
BLS verifyMultipleSignatures - same message - 8 - blst 1.3148 ms/op 1.3040 ms/op 1.01
BLS verifyMultipleSignatures - same message - 32 - blst 1.9220 ms/op 2.0659 ms/op 0.93
BLS verifyMultipleSignatures - same message - 64 - blst 2.8822 ms/op 3.1542 ms/op 0.91
BLS verifyMultipleSignatures - same message - 128 - blst 4.8415 ms/op 5.0196 ms/op 0.96
BLS aggregatePubkeys 32 - blst 21.858 us/op 22.129 us/op 0.99
BLS aggregatePubkeys 128 - blst 73.858 us/op 80.153 us/op 0.92
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 81.812 ms/op 102.57 ms/op 0.80
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 86.011 ms/op 75.357 ms/op 1.14
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 39.580 ms/op 47.173 ms/op 0.84
getSlashingsAndExits - default max 149.01 us/op 143.79 us/op 1.04
getSlashingsAndExits - 2k 830.58 us/op 382.02 us/op 2.17
proposeBlockBody type=full, size=empty 6.0956 ms/op 7.1536 ms/op 0.85
isKnown best case - 1 super set check 504.00 ns/op 652.00 ns/op 0.77
isKnown normal case - 2 super set checks 481.00 ns/op 672.00 ns/op 0.72
isKnown worse case - 16 super set checks 448.00 ns/op 660.00 ns/op 0.68
InMemoryCheckpointStateCache - add get delete 3.6860 us/op 3.9210 us/op 0.94
validate api signedAggregateAndProof - struct 1.6290 ms/op 1.9821 ms/op 0.82
validate gossip signedAggregateAndProof - struct 1.7032 ms/op 1.9275 ms/op 0.88
validate gossip attestation - vc 640000 1.1599 ms/op 1.2416 ms/op 0.93
batch validate gossip attestation - vc 640000 - chunk 32 156.74 us/op 161.92 us/op 0.97
batch validate gossip attestation - vc 640000 - chunk 64 132.68 us/op 143.08 us/op 0.93
batch validate gossip attestation - vc 640000 - chunk 128 130.11 us/op 145.04 us/op 0.90
batch validate gossip attestation - vc 640000 - chunk 256 120.55 us/op 151.30 us/op 0.80
pickEth1Vote - no votes 1.2151 ms/op 1.5792 ms/op 0.77
pickEth1Vote - max votes 8.3635 ms/op 12.616 ms/op 0.66
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.073 ms/op 19.847 ms/op 0.76
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.255 ms/op 32.458 ms/op 0.72
pickEth1Vote - Eth1Data fastSerialize value x2048 599.51 us/op 700.45 us/op 0.86
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.1894 ms/op 4.3902 ms/op 0.73
bytes32 toHexString 635.00 ns/op 909.00 ns/op 0.70
bytes32 Buffer.toString(hex) 261.00 ns/op 318.00 ns/op 0.82
bytes32 Buffer.toString(hex) from Uint8Array 470.00 ns/op 592.00 ns/op 0.79
bytes32 Buffer.toString(hex) + 0x 275.00 ns/op 336.00 ns/op 0.82
Object access 1 prop 0.19800 ns/op 0.23100 ns/op 0.86
Map access 1 prop 0.14300 ns/op 0.20200 ns/op 0.71
Object get x1000 6.2430 ns/op 8.0370 ns/op 0.78
Map get x1000 6.4710 ns/op 9.1740 ns/op 0.71
Object set x1000 50.052 ns/op 74.306 ns/op 0.67
Map set x1000 32.422 ns/op 52.721 ns/op 0.61
Return object 10000 times 0.30630 ns/op 0.36870 ns/op 0.83
Throw Error 10000 times 3.5091 us/op 4.3599 us/op 0.80
toHex 161.70 ns/op 224.30 ns/op 0.72
Buffer.from 150.84 ns/op 222.01 ns/op 0.68
shared Buffer 100.57 ns/op 121.41 ns/op 0.83
fastMsgIdFn sha256 / 200 bytes 2.4120 us/op 2.8420 us/op 0.85
fastMsgIdFn h32 xxhash / 200 bytes 288.00 ns/op 365.00 ns/op 0.79
fastMsgIdFn h64 xxhash / 200 bytes 279.00 ns/op 338.00 ns/op 0.83
fastMsgIdFn sha256 / 1000 bytes 7.5780 us/op 9.3670 us/op 0.81
fastMsgIdFn h32 xxhash / 1000 bytes 423.00 ns/op 561.00 ns/op 0.75
fastMsgIdFn h64 xxhash / 1000 bytes 358.00 ns/op 550.00 ns/op 0.65
fastMsgIdFn sha256 / 10000 bytes 65.839 us/op 93.243 us/op 0.71
fastMsgIdFn h32 xxhash / 10000 bytes 1.9660 us/op 3.0740 us/op 0.64
fastMsgIdFn h64 xxhash / 10000 bytes 1.2600 us/op 1.7500 us/op 0.72
send data - 1000 256B messages 14.716 ms/op 22.575 ms/op 0.65
send data - 1000 512B messages 20.109 ms/op 26.438 ms/op 0.76
send data - 1000 1024B messages 28.636 ms/op 36.652 ms/op 0.78
send data - 1000 1200B messages 29.286 ms/op 45.039 ms/op 0.65
send data - 1000 2048B messages 35.578 ms/op 54.344 ms/op 0.65
send data - 1000 4096B messages 35.822 ms/op 51.949 ms/op 0.69
send data - 1000 16384B messages 76.829 ms/op 137.14 ms/op 0.56
send data - 1000 65536B messages 233.89 ms/op 273.17 ms/op 0.86
enrSubnets - fastDeserialize 64 bits 1.4900 us/op 1.7130 us/op 0.87
enrSubnets - ssz BitVector 64 bits 474.00 ns/op 550.00 ns/op 0.86
enrSubnets - fastDeserialize 4 bits 203.00 ns/op 251.00 ns/op 0.81
enrSubnets - ssz BitVector 4 bits 474.00 ns/op 600.00 ns/op 0.79
prioritizePeers score -10:0 att 32-0.1 sync 2-0 226.85 us/op 230.61 us/op 0.98
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 193.11 us/op 262.09 us/op 0.74
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 363.20 us/op 658.98 us/op 0.55
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 537.81 us/op 693.32 us/op 0.78
prioritizePeers score 0:0 att 64-1 sync 4-1 1.0075 ms/op 1.2916 ms/op 0.78
array of 16000 items push then shift 1.7537 us/op 2.2201 us/op 0.79
LinkedList of 16000 items push then shift 9.2240 ns/op 14.804 ns/op 0.62
array of 16000 items push then pop 155.19 ns/op 222.07 ns/op 0.70
LinkedList of 16000 items push then pop 10.316 ns/op 12.847 ns/op 0.80
array of 24000 items push then shift 2.6239 us/op 3.5852 us/op 0.73
LinkedList of 24000 items push then shift 11.641 ns/op 14.791 ns/op 0.79
array of 24000 items push then pop 227.82 ns/op 298.84 ns/op 0.76
LinkedList of 24000 items push then pop 8.6010 ns/op 13.118 ns/op 0.66
intersect bitArray bitLen 8 7.2110 ns/op 10.865 ns/op 0.66
intersect array and set length 8 68.720 ns/op 110.78 ns/op 0.62
intersect bitArray bitLen 128 31.788 ns/op 42.524 ns/op 0.75
intersect array and set length 128 1.0270 us/op 1.4796 us/op 0.69
bitArray.getTrueBitIndexes() bitLen 128 2.7570 us/op 3.6980 us/op 0.75
bitArray.getTrueBitIndexes() bitLen 248 5.0610 us/op 7.2390 us/op 0.70
bitArray.getTrueBitIndexes() bitLen 512 9.6160 us/op 13.361 us/op 0.72
Buffer.concat 32 items 1.0410 us/op 1.6800 us/op 0.62
Uint8Array.set 32 items 2.1340 us/op 2.4630 us/op 0.87
Buffer.copy 2.1680 us/op 2.3540 us/op 0.92
Uint8Array.set - with subarray 3.5340 us/op 3.8280 us/op 0.92
Uint8Array.set - without subarray 2.2590 us/op 2.1610 us/op 1.05
getUint32 - dataview 330.00 ns/op 337.00 ns/op 0.98
getUint32 - manual 286.00 ns/op 298.00 ns/op 0.96
Set add up to 64 items then delete first 3.0379 us/op 3.4253 us/op 0.89
OrderedSet add up to 64 items then delete first 4.3640 us/op 4.9565 us/op 0.88
Set add up to 64 items then delete last 3.7722 us/op 3.6756 us/op 1.03
OrderedSet add up to 64 items then delete last 5.3082 us/op 6.1651 us/op 0.86
Set add up to 64 items then delete middle 3.3451 us/op 3.7004 us/op 0.90
OrderedSet add up to 64 items then delete middle 6.9926 us/op 7.2982 us/op 0.96
Set add up to 128 items then delete first 7.1103 us/op 6.4096 us/op 1.11
OrderedSet add up to 128 items then delete first 11.885 us/op 10.673 us/op 1.11
Set add up to 128 items then delete last 7.9144 us/op 7.0393 us/op 1.12
OrderedSet add up to 128 items then delete last 13.740 us/op 10.605 us/op 1.30
Set add up to 128 items then delete middle 6.9463 us/op 6.8816 us/op 1.01
OrderedSet add up to 128 items then delete middle 19.718 us/op 17.925 us/op 1.10
Set add up to 256 items then delete first 17.342 us/op 14.073 us/op 1.23
OrderedSet add up to 256 items then delete first 25.650 us/op 21.095 us/op 1.22
Set add up to 256 items then delete last 18.488 us/op 13.986 us/op 1.32
OrderedSet add up to 256 items then delete last 26.229 us/op 21.995 us/op 1.19
Set add up to 256 items then delete middle 14.403 us/op 13.388 us/op 1.08
OrderedSet add up to 256 items then delete middle 53.097 us/op 48.485 us/op 1.10
transfer serialized Status (84 B) 1.6690 us/op 1.5680 us/op 1.06
copy serialized Status (84 B) 1.4500 us/op 1.4160 us/op 1.02
transfer serialized SignedVoluntaryExit (112 B) 1.6940 us/op 1.6670 us/op 1.02
copy serialized SignedVoluntaryExit (112 B) 1.3870 us/op 1.4610 us/op 0.95
transfer serialized ProposerSlashing (416 B) 2.1900 us/op 2.6000 us/op 0.84
copy serialized ProposerSlashing (416 B) 2.5480 us/op 2.0190 us/op 1.26
transfer serialized Attestation (485 B) 2.9840 us/op 2.1290 us/op 1.40
copy serialized Attestation (485 B) 2.8020 us/op 1.9710 us/op 1.42
transfer serialized AttesterSlashing (33232 B) 3.4750 us/op 2.2040 us/op 1.58
copy serialized AttesterSlashing (33232 B) 12.019 us/op 7.9320 us/op 1.52
transfer serialized Small SignedBeaconBlock (128000 B) 5.5290 us/op 2.6260 us/op 2.11
copy serialized Small SignedBeaconBlock (128000 B) 30.829 us/op 27.324 us/op 1.13
transfer serialized Avg SignedBeaconBlock (200000 B) 5.3000 us/op 3.4160 us/op 1.55
copy serialized Avg SignedBeaconBlock (200000 B) 45.927 us/op 34.377 us/op 1.34
transfer serialized BlobsSidecar (524380 B) 5.1550 us/op 4.4150 us/op 1.17
copy serialized BlobsSidecar (524380 B) 129.23 us/op 101.64 us/op 1.27
transfer serialized Big SignedBeaconBlock (1000000 B) 5.7730 us/op 4.5380 us/op 1.27
copy serialized Big SignedBeaconBlock (1000000 B) 266.74 us/op 423.22 us/op 0.63
pass gossip attestations to forkchoice per slot 3.0629 ms/op 3.2007 ms/op 0.96
forkChoice updateHead vc 100000 bc 64 eq 0 646.55 us/op 568.20 us/op 1.14
forkChoice updateHead vc 600000 bc 64 eq 0 4.2503 ms/op 3.8148 ms/op 1.11
forkChoice updateHead vc 1000000 bc 64 eq 0 6.9146 ms/op 6.2877 ms/op 1.10
forkChoice updateHead vc 600000 bc 320 eq 0 3.8140 ms/op 3.4152 ms/op 1.12
forkChoice updateHead vc 600000 bc 1200 eq 0 3.8062 ms/op 3.3435 ms/op 1.14
forkChoice updateHead vc 600000 bc 7200 eq 0 4.6617 ms/op 4.4148 ms/op 1.06
forkChoice updateHead vc 600000 bc 64 eq 1000 11.693 ms/op 11.242 ms/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 10000 11.375 ms/op 11.358 ms/op 1.00
forkChoice updateHead vc 600000 bc 64 eq 300000 22.047 ms/op 18.408 ms/op 1.20
computeDeltas 500000 validators 300 proto nodes 4.3108 ms/op 3.9587 ms/op 1.09
computeDeltas 500000 validators 1200 proto nodes 4.1968 ms/op 3.7921 ms/op 1.11
computeDeltas 500000 validators 7200 proto nodes 3.6899 ms/op 3.7887 ms/op 0.97
computeDeltas 750000 validators 300 proto nodes 5.3509 ms/op 5.7003 ms/op 0.94
computeDeltas 750000 validators 1200 proto nodes 5.3703 ms/op 5.9040 ms/op 0.91
computeDeltas 750000 validators 7200 proto nodes 5.3910 ms/op 6.0759 ms/op 0.89
computeDeltas 1400000 validators 300 proto nodes 10.031 ms/op 11.369 ms/op 0.88
computeDeltas 1400000 validators 1200 proto nodes 10.277 ms/op 11.064 ms/op 0.93
computeDeltas 1400000 validators 7200 proto nodes 10.296 ms/op 10.665 ms/op 0.97
computeDeltas 2100000 validators 300 proto nodes 15.401 ms/op 16.616 ms/op 0.93
computeDeltas 2100000 validators 1200 proto nodes 15.225 ms/op 15.888 ms/op 0.96
computeDeltas 2100000 validators 7200 proto nodes 15.298 ms/op 15.760 ms/op 0.97
altair processAttestation - 250000 vs - 7PWei normalcase 2.2106 ms/op 1.7091 ms/op 1.29
altair processAttestation - 250000 vs - 7PWei worstcase 2.9847 ms/op 2.9195 ms/op 1.02
altair processAttestation - setStatus - 1/6 committees join 90.882 us/op 104.19 us/op 0.87
altair processAttestation - setStatus - 1/3 committees join 177.87 us/op 196.59 us/op 0.90
altair processAttestation - setStatus - 1/2 committees join 259.34 us/op 268.26 us/op 0.97
altair processAttestation - setStatus - 2/3 committees join 342.20 us/op 317.03 us/op 1.08
altair processAttestation - setStatus - 4/5 committees join 479.21 us/op 469.49 us/op 1.02
altair processAttestation - setStatus - 100% committees join 588.85 us/op 549.65 us/op 1.07
altair processBlock - 250000 vs - 7PWei normalcase 4.2838 ms/op 6.0797 ms/op 0.70
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.343 ms/op 26.221 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 40.731 ms/op 38.021 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase hashState 76.419 ms/op 77.832 ms/op 0.98
phase0 processBlock - 250000 vs - 7PWei normalcase 2.7852 ms/op 2.3454 ms/op 1.19
phase0 processBlock - 250000 vs - 7PWei worstcase 28.385 ms/op 24.904 ms/op 1.14
altair processEth1Data - 250000 vs - 7PWei normalcase 352.72 us/op 314.01 us/op 1.12
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.1070 us/op 6.1110 us/op 1.00
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 28.402 us/op 24.503 us/op 1.16
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 12.081 us/op 7.0420 us/op 1.72
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.8250 us/op 5.8360 us/op 1.00
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 125.30 us/op 101.92 us/op 1.23
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 907.54 us/op 1.0995 ms/op 0.83
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.0880 ms/op 1.7069 ms/op 0.64
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1327 ms/op 1.5301 ms/op 0.74
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.6271 ms/op 2.5898 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.9705 ms/op 1.8075 ms/op 1.09
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.0286 ms/op 3.8817 ms/op 1.04
Tree 40 250000 create 213.41 ms/op 229.83 ms/op 0.93
Tree 40 250000 get(125000) 144.39 ns/op 151.30 ns/op 0.95
Tree 40 250000 set(125000) 682.45 ns/op 733.96 ns/op 0.93
Tree 40 250000 toArray() 18.986 ms/op 18.387 ms/op 1.03
Tree 40 250000 iterate all - toArray() + loop 18.450 ms/op 23.077 ms/op 0.80
Tree 40 250000 iterate all - get(i) 55.408 ms/op 55.242 ms/op 1.00
MutableVector 250000 create 10.185 ms/op 11.259 ms/op 0.90
MutableVector 250000 get(125000) 6.2750 ns/op 6.3360 ns/op 0.99
MutableVector 250000 set(125000) 210.94 ns/op 193.34 ns/op 1.09
MutableVector 250000 toArray() 4.1007 ms/op 3.7833 ms/op 1.08
MutableVector 250000 iterate all - toArray() + loop 4.3214 ms/op 3.6151 ms/op 1.20
MutableVector 250000 iterate all - get(i) 1.6456 ms/op 1.6807 ms/op 0.98
Array 250000 create 3.5671 ms/op 3.4317 ms/op 1.04
Array 250000 clone - spread 1.4003 ms/op 1.4500 ms/op 0.97
Array 250000 get(125000) 0.41500 ns/op 0.42100 ns/op 0.99
Array 250000 set(125000) 0.43400 ns/op 0.44100 ns/op 0.98
Array 250000 iterate all - loop 100.61 us/op 93.124 us/op 1.08
effectiveBalanceIncrements clone Uint8Array 300000 30.558 us/op 31.535 us/op 0.97
effectiveBalanceIncrements clone MutableVector 300000 123.00 ns/op 128.00 ns/op 0.96
effectiveBalanceIncrements rw all Uint8Array 300000 200.19 us/op 198.29 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 61.987 ms/op 69.274 ms/op 0.89
phase0 afterProcessEpoch - 250000 vs - 7PWei 92.046 ms/op 89.871 ms/op 1.02
Array.fill - length 1000000 3.5053 ms/op 3.5923 ms/op 0.98
Array push - length 1000000 18.233 ms/op 17.200 ms/op 1.06
Array.get 0.26328 ns/op 0.29002 ns/op 0.91
Uint8Array.get 0.42867 ns/op 0.44924 ns/op 0.95
phase0 beforeProcessEpoch - 250000 vs - 7PWei 15.152 ms/op 17.872 ms/op 0.85
altair processEpoch - mainnet_e81889 363.30 ms/op 346.12 ms/op 1.05
mainnet_e81889 - altair beforeProcessEpoch 17.203 ms/op 17.186 ms/op 1.00
mainnet_e81889 - altair processJustificationAndFinalization 19.566 us/op 18.523 us/op 1.06
mainnet_e81889 - altair processInactivityUpdates 4.9334 ms/op 4.9391 ms/op 1.00
mainnet_e81889 - altair processRewardsAndPenalties 62.896 ms/op 60.481 ms/op 1.04
mainnet_e81889 - altair processRegistryUpdates 2.0700 us/op 2.7600 us/op 0.75
mainnet_e81889 - altair processSlashings 598.00 ns/op 563.00 ns/op 1.06
mainnet_e81889 - altair processEth1DataReset 467.00 ns/op 546.00 ns/op 0.86
mainnet_e81889 - altair processEffectiveBalanceUpdates 3.2204 ms/op 3.2122 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 4.8520 us/op 3.8050 us/op 1.28
mainnet_e81889 - altair processRandaoMixesReset 5.4980 us/op 9.0650 us/op 0.61
mainnet_e81889 - altair processHistoricalRootsUpdate 643.00 ns/op 808.00 ns/op 0.80
mainnet_e81889 - altair processParticipationFlagUpdates 3.0770 us/op 3.2050 us/op 0.96
mainnet_e81889 - altair processSyncCommitteeUpdates 745.00 ns/op 465.00 ns/op 1.60
mainnet_e81889 - altair afterProcessEpoch 91.936 ms/op 94.000 ms/op 0.98
capella processEpoch - mainnet_e217614 1.3881 s/op 1.2873 s/op 1.08
mainnet_e217614 - capella beforeProcessEpoch 76.223 ms/op 80.660 ms/op 0.94
mainnet_e217614 - capella processJustificationAndFinalization 23.285 us/op 16.000 us/op 1.46
mainnet_e217614 - capella processInactivityUpdates 16.979 ms/op 19.882 ms/op 0.85
mainnet_e217614 - capella processRewardsAndPenalties 264.29 ms/op 258.65 ms/op 1.02
mainnet_e217614 - capella processRegistryUpdates 21.065 us/op 17.680 us/op 1.19
mainnet_e217614 - capella processSlashings 637.00 ns/op 627.00 ns/op 1.02
mainnet_e217614 - capella processEth1DataReset 501.00 ns/op 329.00 ns/op 1.52
mainnet_e217614 - capella processEffectiveBalanceUpdates 10.631 ms/op 14.189 ms/op 0.75
mainnet_e217614 - capella processSlashingsReset 4.7700 us/op 3.0300 us/op 1.57
mainnet_e217614 - capella processRandaoMixesReset 6.5550 us/op 4.1400 us/op 1.58
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0970 us/op 965.00 ns/op 1.14
mainnet_e217614 - capella processParticipationFlagUpdates 2.7190 us/op 1.8390 us/op 1.48
mainnet_e217614 - capella afterProcessEpoch 261.53 ms/op 237.71 ms/op 1.10
phase0 processEpoch - mainnet_e58758 426.90 ms/op 450.52 ms/op 0.95
mainnet_e58758 - phase0 beforeProcessEpoch 78.069 ms/op 80.920 ms/op 0.96
mainnet_e58758 - phase0 processJustificationAndFinalization 12.942 us/op 21.346 us/op 0.61
mainnet_e58758 - phase0 processRewardsAndPenalties 29.100 ms/op 26.241 ms/op 1.11
mainnet_e58758 - phase0 processRegistryUpdates 8.1910 us/op 7.8360 us/op 1.05
mainnet_e58758 - phase0 processSlashings 408.00 ns/op 435.00 ns/op 0.94
mainnet_e58758 - phase0 processEth1DataReset 362.00 ns/op 358.00 ns/op 1.01
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6432 ms/op 1.1592 ms/op 1.42
mainnet_e58758 - phase0 processSlashingsReset 5.1020 us/op 4.6200 us/op 1.10
mainnet_e58758 - phase0 processRandaoMixesReset 5.9820 us/op 6.2170 us/op 0.96
mainnet_e58758 - phase0 processHistoricalRootsUpdate 461.00 ns/op 484.00 ns/op 0.95
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.8200 us/op 4.0770 us/op 0.69
mainnet_e58758 - phase0 afterProcessEpoch 79.916 ms/op 83.954 ms/op 0.95
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2173 ms/op 1.5922 ms/op 0.76
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.7026 ms/op 2.4097 ms/op 1.54
altair processInactivityUpdates - 250000 normalcase 19.308 ms/op 25.321 ms/op 0.76
altair processInactivityUpdates - 250000 worstcase 23.934 ms/op 25.423 ms/op 0.94
phase0 processRegistryUpdates - 250000 normalcase 9.9220 us/op 15.717 us/op 0.63
phase0 processRegistryUpdates - 250000 badcase_full_deposits 469.90 us/op 453.98 us/op 1.04
phase0 processRegistryUpdates - 250000 worstcase 0.5 129.92 ms/op 161.89 ms/op 0.80
altair processRewardsAndPenalties - 250000 normalcase 58.156 ms/op 55.740 ms/op 1.04
altair processRewardsAndPenalties - 250000 worstcase 47.789 ms/op 57.105 ms/op 0.84
phase0 getAttestationDeltas - 250000 normalcase 7.4346 ms/op 12.480 ms/op 0.60
phase0 getAttestationDeltas - 250000 worstcase 7.4474 ms/op 13.127 ms/op 0.57
phase0 processSlashings - 250000 worstcase 119.75 us/op 143.98 us/op 0.83
altair processSyncCommitteeUpdates - 250000 128.22 ms/op 165.80 ms/op 0.77
BeaconState.hashTreeRoot - No change 262.00 ns/op 558.00 ns/op 0.47
BeaconState.hashTreeRoot - 1 full validator 124.13 us/op 111.06 us/op 1.12
BeaconState.hashTreeRoot - 32 full validator 1.2518 ms/op 929.94 us/op 1.35
BeaconState.hashTreeRoot - 512 full validator 13.662 ms/op 9.7081 ms/op 1.41
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 149.71 us/op 119.61 us/op 1.25
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.1211 ms/op 1.9002 ms/op 1.12
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.937 ms/op 21.511 ms/op 1.35
BeaconState.hashTreeRoot - 1 balances 110.09 us/op 90.729 us/op 1.21
BeaconState.hashTreeRoot - 32 balances 1.2952 ms/op 1.3878 ms/op 0.93
BeaconState.hashTreeRoot - 512 balances 10.835 ms/op 11.625 ms/op 0.93
BeaconState.hashTreeRoot - 250000 balances 194.57 ms/op 220.70 ms/op 0.88
aggregationBits - 2048 els - zipIndexesInBitList 23.652 us/op 37.371 us/op 0.63
byteArrayEquals 32 54.092 ns/op 58.457 ns/op 0.93
Buffer.compare 32 17.269 ns/op 20.950 ns/op 0.82
byteArrayEquals 1024 1.6630 us/op 1.7507 us/op 0.95
Buffer.compare 1024 24.747 ns/op 27.108 ns/op 0.91
byteArrayEquals 16384 25.286 us/op 26.255 us/op 0.96
Buffer.compare 16384 190.81 ns/op 210.77 ns/op 0.91
byteArrayEquals 123687377 190.81 ms/op 200.82 ms/op 0.95
Buffer.compare 123687377 7.2569 ms/op 7.2944 ms/op 0.99
byteArrayEquals 32 - diff last byte 52.342 ns/op 54.439 ns/op 0.96
Buffer.compare 32 - diff last byte 16.855 ns/op 18.215 ns/op 0.93
byteArrayEquals 1024 - diff last byte 1.5368 us/op 1.6465 us/op 0.93
Buffer.compare 1024 - diff last byte 24.217 ns/op 27.384 ns/op 0.88
byteArrayEquals 16384 - diff last byte 24.417 us/op 26.013 us/op 0.94
Buffer.compare 16384 - diff last byte 201.12 ns/op 213.11 ns/op 0.94
byteArrayEquals 123687377 - diff last byte 189.00 ms/op 196.35 ms/op 0.96
Buffer.compare 123687377 - diff last byte 6.1245 ms/op 7.4064 ms/op 0.83
byteArrayEquals 32 - random bytes 4.9650 ns/op 5.4380 ns/op 0.91
Buffer.compare 32 - random bytes 17.005 ns/op 17.582 ns/op 0.97
byteArrayEquals 1024 - random bytes 4.9840 ns/op 5.5020 ns/op 0.91
Buffer.compare 1024 - random bytes 16.585 ns/op 17.607 ns/op 0.94
byteArrayEquals 16384 - random bytes 4.9540 ns/op 5.2210 ns/op 0.95
Buffer.compare 16384 - random bytes 16.625 ns/op 17.543 ns/op 0.95
byteArrayEquals 123687377 - random bytes 6.1900 ns/op 6.5300 ns/op 0.95
Buffer.compare 123687377 - random bytes 17.820 ns/op 18.930 ns/op 0.94
regular array get 100000 times 31.854 us/op 32.975 us/op 0.97
wrappedArray get 100000 times 31.819 us/op 32.756 us/op 0.97
arrayWithProxy get 100000 times 12.826 ms/op 13.784 ms/op 0.93
ssz.Root.equals 44.377 ns/op 45.952 ns/op 0.97
byteArrayEquals 43.776 ns/op 45.794 ns/op 0.96
Buffer.compare 10.025 ns/op 10.624 ns/op 0.94
shuffle list - 16384 els 6.0227 ms/op 6.6869 ms/op 0.90
shuffle list - 250000 els 89.141 ms/op 98.577 ms/op 0.90
processSlot - 1 slots 13.453 us/op 17.155 us/op 0.78
processSlot - 32 slots 2.7367 ms/op 3.0575 ms/op 0.90
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 39.171 ms/op 38.304 ms/op 1.02
getCommitteeAssignments - req 1 vs - 250000 vc 2.1636 ms/op 2.2258 ms/op 0.97
getCommitteeAssignments - req 100 vs - 250000 vc 4.1682 ms/op 4.3877 ms/op 0.95
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4497 ms/op 4.7652 ms/op 0.93
findModifiedValidators - 10000 modified validators 257.98 ms/op 303.24 ms/op 0.85
findModifiedValidators - 1000 modified validators 180.87 ms/op 228.13 ms/op 0.79
findModifiedValidators - 100 modified validators 162.36 ms/op 181.95 ms/op 0.89
findModifiedValidators - 10 modified validators 172.19 ms/op 184.47 ms/op 0.93
findModifiedValidators - 1 modified validators 163.75 ms/op 218.30 ms/op 0.75
findModifiedValidators - no difference 187.43 ms/op 247.89 ms/op 0.76
compare ViewDUs 3.2453 s/op 3.7347 s/op 0.87
compare each validator Uint8Array 1.6587 s/op 1.7799 s/op 0.93
compare ViewDU to Uint8Array 1.1197 s/op 1.5443 s/op 0.73
migrate state 1000000 validators, 24 modified, 0 new 610.98 ms/op 831.61 ms/op 0.73
migrate state 1000000 validators, 1700 modified, 1000 new 888.97 ms/op 1.0680 s/op 0.83
migrate state 1000000 validators, 3400 modified, 2000 new 1.0919 s/op 1.2938 s/op 0.84
migrate state 1500000 validators, 24 modified, 0 new 651.75 ms/op 699.75 ms/op 0.93
migrate state 1500000 validators, 1700 modified, 1000 new 823.03 ms/op 909.45 ms/op 0.90
migrate state 1500000 validators, 3400 modified, 2000 new 1.0767 s/op 1.1606 s/op 0.93
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3300 ns/op 5.2000 ns/op 0.83
state getBlockRootAtSlot - 250000 vs - 7PWei 695.90 ns/op 795.44 ns/op 0.87
computeProposers - vc 250000 8.2936 ms/op 11.623 ms/op 0.71
computeEpochShuffling - vc 250000 92.550 ms/op 107.51 ms/op 0.86
getNextSyncCommittee - vc 250000 134.00 ms/op 159.63 ms/op 0.84
computeSigningRoot for AttestationData 26.324 us/op 31.276 us/op 0.84
hash AttestationData serialized data then Buffer.toString(base64) 1.5532 us/op 1.8995 us/op 0.82
toHexString serialized data 888.32 ns/op 1.1483 us/op 0.77
Buffer.toString(base64) 170.67 ns/op 235.24 ns/op 0.73
block root to RootHex using toHex 150.45 ns/op 175.77 ns/op 0.86
block root to RootHex using toRootHex 102.16 ns/op 104.98 ns/op 0.97

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review August 21, 2024 09:53
@twoeths twoeths requested a review from a team as a code owner August 21, 2024 09:53
@philknows philknows added this to the v1.22.0 milestone Sep 3, 2024
@twoeths twoeths merged commit fe6c4ac into unstable Sep 5, 2024
20 checks passed
@twoeths twoeths deleted the te/improve_archive_state branch September 5, 2024 01:34
philknows pushed a commit that referenced this pull request Sep 11, 2024
@twoeths @philknows
feat: archive state using BufferPool if provided (#7042)
84e3a72 
* fix: archive state using BufferPool if provided

* chore: fix comment
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.22.0 🎉

@twoeths twoeths mentioned this pull request Sep 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.22.0
Development

Successfully merging this pull request may close these issues.

Archive state optimization
3 participants
@twoeths @wemeetagain @philknows