Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: consume new state cache apis #6250

Closed
wants to merge 15 commits into from
Closed

feat: consume new state cache apis #6250

wants to merge 15 commits into from

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 4, 2024 •
edited
Loading

Motivation

  • Right now we cache all check point states from the finalized checkpoint and 96-128 block states which is ok in good network condition but not sustainable
  • This work adds a feature flag which allow us to store and use only recent block/checkpoint states, any "missed" state will be reloaded from data store and regen.

Description

  • Consume state caches based on the new chain.nHistoricalStates flag
    • We can configure chain.maxBlockStates (default to 32) for the new block state and chain.maxCPStateEpochsInMemory (default to 2) for the persistent cp state
  • By default, do not turn on this flag. Need a lot of time to test this feature on mainnet
  • Regen: add "shouldReload" option to allow to reload state from data store, which is only true for block processing
  • Finalized state may not be available as a state anymore, we only have finalized state bytes instead
  • Add e2e for different reorg scenarios where checkpoint states are reloaded
  • New BufferPool to preallocate Buffer to serialize state at every epoch and for reload scenario, check serializeState.test.ts benchmark for more information
  • I've tested this work for a while, the node was able to survive with 0-historical state, i.e any reorg caused the node to reload state

Tradeoff of this approach:

  • at the last 1/3 slot of slot 0 of every epoch, we have to persist state to DB/data store

Some statistics I monitored when testing this work:

  • it takes ~1.2s to persist a holesky state
  • it takes <4s to reload a holesky state and <2.5s to reload s mainnet state (tested with 0-historical state config). In reality we should configure in a way that we almost don't have to reload any cp states
  • RSS is an issue when we store state per epoch. This is a known issue (see v1.9.0-rc.0: Memory leak #5549) so I'd investigate and resolve it in other PRs
  • less heap memory used
network validator count branch/config min (GB) max (GB)
holesky 1k. stable 4.8 5.8
holesky 1k. 0 hist states 3.2 4
mainnet N/A stable 3.8 4.8
mainnet N/A 0 hist states 2.9 3.8

this is a continuation of #6176

part of #5968

@twoeths twoeths mentioned this pull request Jan 4, 2024
Closed
9 tasks
@codecov Codecov
Copy link

codecov bot commented Jan 4, 2024 •
edited
Loading

Codecov Report

Merging #6250 (e70e4ab) into unstable (ea49409) will decrease coverage by 3.82%.
Report is 35 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6250      +/-   ##
============================================
- Coverage     80.38%   76.57%   -3.82%     
============================================
  Files           202      248      +46     
  Lines         19622    25985    +6363     
  Branches       1176     1453     +277     
============================================
+ Hits          15773    19897    +4124     
- Misses         3821     6058    +2237     
- Partials         28       30       +2

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 4, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 58a2800 Previous: 551fd8e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0971 ms/op 951.16 us/op 1.15
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 124.06 us/op 81.668 us/op 1.52
BLS verify - blst-native 1.3467 ms/op 1.2513 ms/op 1.08
BLS verifyMultipleSignatures 3 - blst-native 2.8450 ms/op 2.6265 ms/op 1.08
BLS verifyMultipleSignatures 8 - blst-native 6.2272 ms/op 5.7580 ms/op 1.08
BLS verifyMultipleSignatures 32 - blst-native 22.845 ms/op 21.079 ms/op 1.08
BLS verifyMultipleSignatures 64 - blst-native 45.519 ms/op 41.502 ms/op 1.10
BLS verifyMultipleSignatures 128 - blst-native 90.341 ms/op 82.325 ms/op 1.10
BLS deserializing 10000 signatures 976.61 ms/op 947.14 ms/op 1.03
BLS deserializing 100000 signatures 9.2504 s/op 9.5014 s/op 0.97
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3783 ms/op 1.3565 ms/op 1.02
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5673 ms/op 1.5331 ms/op 1.02
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4194 ms/op 2.3555 ms/op 1.03
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.5739 ms/op 4.9244 ms/op 0.73
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.7871 ms/op 5.6858 ms/op 1.02
BLS aggregatePubkeys 32 - blst-native 26.946 us/op 25.676 us/op 1.05
BLS aggregatePubkeys 128 - blst-native 103.82 us/op 100.32 us/op 1.03
getAttestationsForBlock 77.592 ms/op 54.397 ms/op 1.43
getSlashingsAndExits - default max 276.24 us/op 146.34 us/op 1.89
getSlashingsAndExits - 2k 667.12 us/op 638.08 us/op 1.05
proposeBlockBody type=full, size=empty 6.8639 ms/op 8.5420 ms/op 0.80
isKnown best case - 1 super set check 397.00 ns/op 775.00 ns/op 0.51
isKnown normal case - 2 super set checks 415.00 ns/op 844.00 ns/op 0.49
isKnown worse case - 16 super set checks 384.00 ns/op 827.00 ns/op 0.46
CheckpointStateCache - add get delete 5.8490 us/op 7.4820 us/op 0.78
validate api signedAggregateAndProof - struct 2.8498 ms/op 3.3250 ms/op 0.86
validate gossip signedAggregateAndProof - struct 2.8502 ms/op 3.2625 ms/op 0.87
validate gossip attestation - vc 640000 1.4051 ms/op 1.6625 ms/op 0.85
batch validate gossip attestation - vc 640000 - chunk 32 172.51 us/op 209.32 us/op 0.82
batch validate gossip attestation - vc 640000 - chunk 64 157.18 us/op 182.24 us/op 0.86
batch validate gossip attestation - vc 640000 - chunk 128 151.39 us/op 201.64 us/op 0.75
batch validate gossip attestation - vc 640000 - chunk 256 141.43 us/op 181.24 us/op 0.78
pickEth1Vote - no votes 1.3618 ms/op 1.7406 ms/op 0.78
pickEth1Vote - max votes 12.599 ms/op 18.305 ms/op 0.69
pickEth1Vote - Eth1Data hashTreeRoot value x2048 18.144 ms/op 27.802 ms/op 0.65
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 27.216 ms/op 47.619 ms/op 0.57
pickEth1Vote - Eth1Data fastSerialize value x2048 653.60 us/op 792.96 us/op 0.82
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.0198 ms/op 10.521 ms/op 0.48
bytes32 toHexString 528.00 ns/op 900.00 ns/op 0.59
bytes32 Buffer.toString(hex) 298.00 ns/op 395.00 ns/op 0.75
bytes32 Buffer.toString(hex) from Uint8Array 489.00 ns/op 652.00 ns/op 0.75
bytes32 Buffer.toString(hex) + 0x 322.00 ns/op 394.00 ns/op 0.82
Object access 1 prop 0.18600 ns/op 0.24600 ns/op 0.76
Map access 1 prop 0.15700 ns/op 0.19200 ns/op 0.82
Object get x1000 7.9270 ns/op 9.3790 ns/op 0.85
Map get x1000 0.82400 ns/op 0.99800 ns/op 0.83
Object set x1000 56.399 ns/op 85.440 ns/op 0.66
Map set x1000 46.041 ns/op 58.948 ns/op 0.78
Return object 10000 times 0.24950 ns/op 0.32340 ns/op 0.77
Throw Error 10000 times 3.8899 us/op 5.2775 us/op 0.74
fastMsgIdFn sha256 / 200 bytes 3.4010 us/op 4.6080 us/op 0.74
fastMsgIdFn h32 xxhash / 200 bytes 326.00 ns/op 440.00 ns/op 0.74
fastMsgIdFn h64 xxhash / 200 bytes 373.00 ns/op 504.00 ns/op 0.74
fastMsgIdFn sha256 / 1000 bytes 11.836 us/op 14.288 us/op 0.83
fastMsgIdFn h32 xxhash / 1000 bytes 455.00 ns/op 554.00 ns/op 0.82
fastMsgIdFn h64 xxhash / 1000 bytes 448.00 ns/op 555.00 ns/op 0.81
fastMsgIdFn sha256 / 10000 bytes 105.06 us/op 121.75 us/op 0.86
fastMsgIdFn h32 xxhash / 10000 bytes 2.0610 us/op 2.2580 us/op 0.91
fastMsgIdFn h64 xxhash / 10000 bytes 1.3990 us/op 1.6250 us/op 0.86
send data - 1000 256B messages 21.367 ms/op 27.652 ms/op 0.77
send data - 1000 512B messages 24.738 ms/op 28.764 ms/op 0.86
send data - 1000 1024B messages 44.716 ms/op 50.217 ms/op 0.89
send data - 1000 1200B messages 42.254 ms/op 52.743 ms/op 0.80
send data - 1000 2048B messages 50.777 ms/op 74.539 ms/op 0.68
send data - 1000 4096B messages 41.679 ms/op 65.035 ms/op 0.64
send data - 1000 16384B messages 117.54 ms/op 143.56 ms/op 0.82
send data - 1000 65536B messages 467.56 ms/op 672.90 ms/op 0.69
enrSubnets - fastDeserialize 64 bits 1.2620 us/op 2.0020 us/op 0.63
enrSubnets - ssz BitVector 64 bits 433.00 ns/op 1.0500 us/op 0.41
enrSubnets - fastDeserialize 4 bits 172.00 ns/op 341.00 ns/op 0.50
enrSubnets - ssz BitVector 4 bits 445.00 ns/op 664.00 ns/op 0.67
prioritizePeers score -10:0 att 32-0.1 sync 2-0 102.99 us/op 165.83 us/op 0.62
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 133.24 us/op 212.43 us/op 0.63
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 173.47 us/op 242.72 us/op 0.71
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 300.50 us/op 539.77 us/op 0.56
prioritizePeers score 0:0 att 64-1 sync 4-1 364.51 us/op 516.74 us/op 0.71
array of 16000 items push then shift 1.6546 us/op 1.8662 us/op 0.89
LinkedList of 16000 items push then shift 9.0690 ns/op 12.501 ns/op 0.73
array of 16000 items push then pop 87.376 ns/op 142.84 ns/op 0.61
LinkedList of 16000 items push then pop 8.8550 ns/op 13.170 ns/op 0.67
array of 24000 items push then shift 2.4641 us/op 2.9666 us/op 0.83
LinkedList of 24000 items push then shift 9.1180 ns/op 14.571 ns/op 0.63
array of 24000 items push then pop 112.39 ns/op 196.45 ns/op 0.57
LinkedList of 24000 items push then pop 8.8490 ns/op 10.785 ns/op 0.82
intersect bitArray bitLen 8 6.6200 ns/op 8.2100 ns/op 0.81
intersect array and set length 8 64.528 ns/op 107.15 ns/op 0.60
intersect bitArray bitLen 128 34.168 ns/op 43.174 ns/op 0.79
intersect array and set length 128 914.61 ns/op 1.3436 us/op 0.68
bitArray.getTrueBitIndexes() bitLen 128 1.4610 us/op 2.2690 us/op 0.64
bitArray.getTrueBitIndexes() bitLen 248 2.5200 us/op 3.4260 us/op 0.74
bitArray.getTrueBitIndexes() bitLen 512 4.9880 us/op 7.0540 us/op 0.71
Buffer.concat 32 items 937.00 ns/op 1.2540 us/op 0.75
Uint8Array.set 32 items 1.6410 us/op 2.0490 us/op 0.80
Set add up to 64 items then delete first 4.3818 us/op 4.9652 us/op 0.88
OrderedSet add up to 64 items then delete first 5.5071 us/op 6.0699 us/op 0.91
Set add up to 64 items then delete last 4.6764 us/op 5.0978 us/op 0.92
OrderedSet add up to 64 items then delete last 5.8618 us/op 6.7266 us/op 0.87
Set add up to 64 items then delete middle 4.6854 us/op 5.2232 us/op 0.90
OrderedSet add up to 64 items then delete middle 7.2177 us/op 7.7786 us/op 0.93
Set add up to 128 items then delete first 10.093 us/op 10.311 us/op 0.98
OrderedSet add up to 128 items then delete first 14.215 us/op 13.889 us/op 1.02
Set add up to 128 items then delete last 10.398 us/op 10.170 us/op 1.02
OrderedSet add up to 128 items then delete last 13.327 us/op 12.577 us/op 1.06
Set add up to 128 items then delete middle 10.349 us/op 10.373 us/op 1.00
OrderedSet add up to 128 items then delete middle 18.935 us/op 20.386 us/op 0.93
Set add up to 256 items then delete first 20.799 us/op 23.235 us/op 0.90
OrderedSet add up to 256 items then delete first 28.287 us/op 29.164 us/op 0.97
Set add up to 256 items then delete last 20.709 us/op 19.583 us/op 1.06
OrderedSet add up to 256 items then delete last 26.413 us/op 24.100 us/op 1.10
Set add up to 256 items then delete middle 20.722 us/op 19.048 us/op 1.09
OrderedSet add up to 256 items then delete middle 49.853 us/op 46.405 us/op 1.07
transfer serialized Status (84 B) 1.9470 us/op 1.7200 us/op 1.13
copy serialized Status (84 B) 1.4850 us/op 1.3730 us/op 1.08
transfer serialized SignedVoluntaryExit (112 B) 2.1070 us/op 2.1020 us/op 1.00
copy serialized SignedVoluntaryExit (112 B) 1.5850 us/op 1.5730 us/op 1.01
transfer serialized ProposerSlashing (416 B) 3.3640 us/op 2.2840 us/op 1.47
copy serialized ProposerSlashing (416 B) 3.0520 us/op 2.5660 us/op 1.19
transfer serialized Attestation (485 B) 3.4810 us/op 2.6570 us/op 1.31
copy serialized Attestation (485 B) 3.1120 us/op 3.2320 us/op 0.96
transfer serialized AttesterSlashing (33232 B) 3.0200 us/op 3.2680 us/op 0.92
copy serialized AttesterSlashing (33232 B) 6.4610 us/op 6.6050 us/op 0.98
transfer serialized Small SignedBeaconBlock (128000 B) 2.9680 us/op 3.3210 us/op 0.89
copy serialized Small SignedBeaconBlock (128000 B) 14.891 us/op 16.234 us/op 0.92
transfer serialized Avg SignedBeaconBlock (200000 B) 3.1320 us/op 3.5110 us/op 0.89
copy serialized Avg SignedBeaconBlock (200000 B) 21.603 us/op 21.740 us/op 0.99
transfer serialized BlobsSidecar (524380 B) 3.0620 us/op 3.9620 us/op 0.77
copy serialized BlobsSidecar (524380 B) 87.522 us/op 86.869 us/op 1.01
transfer serialized Big SignedBeaconBlock (1000000 B) 2.9680 us/op 4.0550 us/op 0.73
copy serialized Big SignedBeaconBlock (1000000 B) 167.21 us/op 171.62 us/op 0.97
pass gossip attestations to forkchoice per slot 3.8321 ms/op 4.6017 ms/op 0.83
forkChoice updateHead vc 100000 bc 64 eq 0 676.65 us/op 713.59 us/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 0 4.0638 ms/op 4.9243 ms/op 0.83
forkChoice updateHead vc 1000000 bc 64 eq 0 6.9375 ms/op 7.2484 ms/op 0.96
forkChoice updateHead vc 600000 bc 320 eq 0 4.1694 ms/op 4.3346 ms/op 0.96
forkChoice updateHead vc 600000 bc 1200 eq 0 4.9426 ms/op 8.3793 ms/op 0.59
forkChoice updateHead vc 600000 bc 7200 eq 0 5.2031 ms/op 5.5626 ms/op 0.94
forkChoice updateHead vc 600000 bc 64 eq 1000 11.088 ms/op 11.690 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 10000 11.882 ms/op 12.397 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 300000 15.684 ms/op 17.381 ms/op 0.90
computeDeltas 500000 validators 300 proto nodes 6.6494 ms/op 6.8089 ms/op 0.98
computeDeltas 500000 validators 1200 proto nodes 6.4557 ms/op 6.6765 ms/op 0.97
computeDeltas 500000 validators 7200 proto nodes 6.4876 ms/op 6.5224 ms/op 0.99
computeDeltas 750000 validators 300 proto nodes 9.7748 ms/op 9.9984 ms/op 0.98
computeDeltas 750000 validators 1200 proto nodes 10.169 ms/op 9.7919 ms/op 1.04
computeDeltas 750000 validators 7200 proto nodes 10.105 ms/op 10.248 ms/op 0.99
computeDeltas 1400000 validators 300 proto nodes 19.789 ms/op 19.536 ms/op 1.01
computeDeltas 1400000 validators 1200 proto nodes 18.940 ms/op 19.444 ms/op 0.97
computeDeltas 1400000 validators 7200 proto nodes 20.001 ms/op 19.336 ms/op 1.03
computeDeltas 2100000 validators 300 proto nodes 29.845 ms/op 29.164 ms/op 1.02
computeDeltas 2100000 validators 1200 proto nodes 28.722 ms/op 28.761 ms/op 1.00
computeDeltas 2100000 validators 7200 proto nodes 30.062 ms/op 29.558 ms/op 1.02
computeProposerBoostScoreFromBalances 500000 validators 3.8719 ms/op 3.8306 ms/op 1.01
computeProposerBoostScoreFromBalances 750000 validators 3.8482 ms/op 3.8309 ms/op 1.00
computeProposerBoostScoreFromBalances 1400000 validators 3.8982 ms/op 3.8591 ms/op 1.01
computeProposerBoostScoreFromBalances 2100000 validators 4.0577 ms/op 3.8272 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei normalcase 3.3552 ms/op 2.7349 ms/op 1.23
altair processAttestation - 250000 vs - 7PWei worstcase 4.3572 ms/op 3.6167 ms/op 1.20
altair processAttestation - setStatus - 1/6 committees join 161.15 us/op 191.15 us/op 0.84
altair processAttestation - setStatus - 1/3 committees join 321.18 us/op 362.10 us/op 0.89
altair processAttestation - setStatus - 1/2 committees join 437.10 us/op 483.22 us/op 0.90
altair processAttestation - setStatus - 2/3 committees join 529.17 us/op 608.96 us/op 0.87
altair processAttestation - setStatus - 4/5 committees join 825.29 us/op 824.26 us/op 1.00
altair processAttestation - setStatus - 100% committees join 867.67 us/op 971.29 us/op 0.89
altair processBlock - 250000 vs - 7PWei normalcase 14.273 ms/op 10.713 ms/op 1.33
altair processBlock - 250000 vs - 7PWei normalcase hashState 46.963 ms/op 34.804 ms/op 1.35
altair processBlock - 250000 vs - 7PWei worstcase 43.343 ms/op 39.740 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase hashState 104.69 ms/op 95.536 ms/op 1.10
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5744 ms/op 2.7965 ms/op 1.28
phase0 processBlock - 250000 vs - 7PWei worstcase 37.957 ms/op 36.128 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 834.55 us/op 543.07 us/op 1.54
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.333 us/op 13.902 us/op 1.46
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 65.444 us/op 46.429 us/op 1.41
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 28.208 us/op 22.087 us/op 1.28
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 17.631 us/op 11.437 us/op 1.54
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 214.53 us/op 205.78 us/op 1.04
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.6677 ms/op 1.5652 ms/op 1.07
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.3387 ms/op 2.0499 ms/op 1.14
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.2575 ms/op 2.4048 ms/op 0.94
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.9628 ms/op 4.1330 ms/op 1.20
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.9325 ms/op 3.0321 ms/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.8767 ms/op 5.9576 ms/op 1.15
Tree 40 250000 create 488.16 ms/op 401.82 ms/op 1.21
Tree 40 250000 get(125000) 226.51 ns/op 219.85 ns/op 1.03
Tree 40 250000 set(125000) 1.3137 us/op 1.0713 us/op 1.23
Tree 40 250000 toArray() 24.335 ms/op 19.581 ms/op 1.24
Tree 40 250000 iterate all - toArray() + loop 24.161 ms/op 20.008 ms/op 1.21
Tree 40 250000 iterate all - get(i) 81.772 ms/op 68.959 ms/op 1.19
MutableVector 250000 create 17.400 ms/op 18.261 ms/op 0.95
MutableVector 250000 get(125000) 6.8900 ns/op 6.5450 ns/op 1.05
MutableVector 250000 set(125000) 323.62 ns/op 279.95 ns/op 1.16
MutableVector 250000 toArray() 4.4857 ms/op 3.7704 ms/op 1.19
MutableVector 250000 iterate all - toArray() + loop 4.7016 ms/op 3.5544 ms/op 1.32
MutableVector 250000 iterate all - get(i) 1.5722 ms/op 1.5465 ms/op 1.02
Array 250000 create 3.8495 ms/op 2.9541 ms/op 1.30
Array 250000 clone - spread 1.2713 ms/op 1.3775 ms/op 0.92
Array 250000 get(125000) 1.1090 ns/op 1.1330 ns/op 0.98
Array 250000 set(125000) 5.1460 ns/op 4.2140 ns/op 1.22
Array 250000 iterate all - loop 175.77 us/op 167.26 us/op 1.05
effectiveBalanceIncrements clone Uint8Array 300000 38.797 us/op 29.811 us/op 1.30
effectiveBalanceIncrements clone MutableVector 300000 378.00 ns/op 431.00 ns/op 0.88
effectiveBalanceIncrements rw all Uint8Array 300000 207.40 us/op 202.20 us/op 1.03
effectiveBalanceIncrements rw all MutableVector 300000 92.324 ms/op 90.142 ms/op 1.02
phase0 afterProcessEpoch - 250000 vs - 7PWei 120.93 ms/op 117.62 ms/op 1.03
phase0 beforeProcessEpoch - 250000 vs - 7PWei 60.700 ms/op 53.608 ms/op 1.13
altair processEpoch - mainnet_e81889 522.84 ms/op 538.08 ms/op 0.97
mainnet_e81889 - altair beforeProcessEpoch 88.891 ms/op 90.609 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 14.883 us/op 16.318 us/op 0.91
mainnet_e81889 - altair processInactivityUpdates 6.2248 ms/op 6.2077 ms/op 1.00
mainnet_e81889 - altair processRewardsAndPenalties 73.888 ms/op 70.633 ms/op 1.05
mainnet_e81889 - altair processRegistryUpdates 2.5810 us/op 2.8780 us/op 0.90
mainnet_e81889 - altair processSlashings 454.00 ns/op 509.00 ns/op 0.89
mainnet_e81889 - altair processEth1DataReset 533.00 ns/op 483.00 ns/op 1.10
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7200 ms/op 1.6583 ms/op 1.64
mainnet_e81889 - altair processSlashingsReset 5.6790 us/op 3.7450 us/op 1.52
mainnet_e81889 - altair processRandaoMixesReset 7.3950 us/op 4.7830 us/op 1.55
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2240 us/op 649.00 ns/op 1.89
mainnet_e81889 - altair processParticipationFlagUpdates 3.0160 us/op 2.4420 us/op 1.24
mainnet_e81889 - altair processSyncCommitteeUpdates 1.2810 us/op 730.00 ns/op 1.75
mainnet_e81889 - altair afterProcessEpoch 121.28 ms/op 121.43 ms/op 1.00
capella processEpoch - mainnet_e217614 2.2480 s/op 2.4029 s/op 0.94
mainnet_e217614 - capella beforeProcessEpoch 510.82 ms/op 702.43 ms/op 0.73
mainnet_e217614 - capella processJustificationAndFinalization 30.535 us/op 27.901 us/op 1.09
mainnet_e217614 - capella processInactivityUpdates 18.615 ms/op 26.108 ms/op 0.71
mainnet_e217614 - capella processRewardsAndPenalties 411.57 ms/op 441.81 ms/op 0.93
mainnet_e217614 - capella processRegistryUpdates 21.523 us/op 31.507 us/op 0.68
mainnet_e217614 - capella processSlashings 921.00 ns/op 589.00 ns/op 1.56
mainnet_e217614 - capella processEth1DataReset 701.00 ns/op 499.00 ns/op 1.40
mainnet_e217614 - capella processEffectiveBalanceUpdates 5.5897 ms/op 4.8096 ms/op 1.16
mainnet_e217614 - capella processSlashingsReset 5.0170 us/op 4.1990 us/op 1.19
mainnet_e217614 - capella processRandaoMixesReset 7.5690 us/op 5.8380 us/op 1.30
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0320 us/op 819.00 ns/op 1.26
mainnet_e217614 - capella processParticipationFlagUpdates 2.2660 us/op 2.1980 us/op 1.03
mainnet_e217614 - capella afterProcessEpoch 303.84 ms/op 336.53 ms/op 0.90
phase0 processEpoch - mainnet_e58758 491.91 ms/op 497.32 ms/op 0.99
mainnet_e58758 - phase0 beforeProcessEpoch 157.52 ms/op 162.78 ms/op 0.97
mainnet_e58758 - phase0 processJustificationAndFinalization 16.931 us/op 18.720 us/op 0.90
mainnet_e58758 - phase0 processRewardsAndPenalties 55.115 ms/op 45.035 ms/op 1.22
mainnet_e58758 - phase0 processRegistryUpdates 14.929 us/op 13.891 us/op 1.07
mainnet_e58758 - phase0 processSlashings 539.00 ns/op 908.00 ns/op 0.59
mainnet_e58758 - phase0 processEth1DataReset 469.00 ns/op 423.00 ns/op 1.11
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.7351 ms/op 1.4007 ms/op 1.24
mainnet_e58758 - phase0 processSlashingsReset 4.2470 us/op 3.5900 us/op 1.18
mainnet_e58758 - phase0 processRandaoMixesReset 5.9440 us/op 6.0200 us/op 0.99
mainnet_e58758 - phase0 processHistoricalRootsUpdate 494.00 ns/op 411.00 ns/op 1.20
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.9700 us/op 4.3600 us/op 1.14
mainnet_e58758 - phase0 afterProcessEpoch 101.83 ms/op 101.20 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.6205 ms/op 1.6969 ms/op 0.95
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.7206 ms/op 1.4670 ms/op 1.17
altair processInactivityUpdates - 250000 normalcase 33.478 ms/op 25.456 ms/op 1.32
altair processInactivityUpdates - 250000 worstcase 25.899 ms/op 23.157 ms/op 1.12
phase0 processRegistryUpdates - 250000 normalcase 10.051 us/op 9.4140 us/op 1.07
phase0 processRegistryUpdates - 250000 badcase_full_deposits 562.43 us/op 396.98 us/op 1.42
phase0 processRegistryUpdates - 250000 worstcase 0.5 137.51 ms/op 133.63 ms/op 1.03
altair processRewardsAndPenalties - 250000 normalcase 65.343 ms/op 61.308 ms/op 1.07
altair processRewardsAndPenalties - 250000 worstcase 64.138 ms/op 62.195 ms/op 1.03
phase0 getAttestationDeltas - 250000 normalcase 9.4450 ms/op 12.109 ms/op 0.78
phase0 getAttestationDeltas - 250000 worstcase 10.428 ms/op 10.845 ms/op 0.96
phase0 processSlashings - 250000 worstcase 121.13 us/op 118.70 us/op 1.02
altair processSyncCommitteeUpdates - 250000 164.94 ms/op 180.32 ms/op 0.91
BeaconState.hashTreeRoot - No change 261.00 ns/op 258.00 ns/op 1.01
BeaconState.hashTreeRoot - 1 full validator 166.80 us/op 173.73 us/op 0.96
BeaconState.hashTreeRoot - 32 full validator 1.5000 ms/op 1.8981 ms/op 0.79
BeaconState.hashTreeRoot - 512 full validator 20.746 ms/op 18.777 ms/op 1.10
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 198.72 us/op 193.25 us/op 1.03
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.2808 ms/op 2.5373 ms/op 0.90
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 27.738 ms/op 36.360 ms/op 0.76
BeaconState.hashTreeRoot - 1 balances 133.53 us/op 164.26 us/op 0.81
BeaconState.hashTreeRoot - 32 balances 1.1999 ms/op 1.6532 ms/op 0.73
BeaconState.hashTreeRoot - 512 balances 11.080 ms/op 13.753 ms/op 0.81
BeaconState.hashTreeRoot - 250000 balances 213.23 ms/op 219.91 ms/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 20.092 us/op 20.800 us/op 0.97
byteArrayEquals 32 76.823 ns/op 77.910 ns/op 0.99
Buffer.compare 32 56.641 ns/op 57.028 ns/op 0.99
byteArrayEquals 1024 2.0955 us/op 2.0837 us/op 1.01
Buffer.compare 1024 71.671 ns/op 72.702 ns/op 0.99
byteArrayEquals 16384 33.361 us/op 33.538 us/op 0.99
Buffer.compare 16384 262.56 ns/op 278.87 ns/op 0.94
byteArrayEquals 123687377 250.69 ms/op 255.23 ms/op 0.98
Buffer.compare 123687377 6.4107 ms/op 8.0802 ms/op 0.79
byteArrayEquals 32 - diff last byte 77.781 ns/op 76.080 ns/op 1.02
Buffer.compare 32 - diff last byte 56.631 ns/op 58.289 ns/op 0.97
byteArrayEquals 1024 - diff last byte 2.0525 us/op 2.0724 us/op 0.99
Buffer.compare 1024 - diff last byte 71.909 ns/op 73.958 ns/op 0.97
byteArrayEquals 16384 - diff last byte 32.690 us/op 33.003 us/op 0.99
Buffer.compare 16384 - diff last byte 273.01 ns/op 344.80 ns/op 0.79
byteArrayEquals 123687377 - diff last byte 246.83 ms/op 254.61 ms/op 0.97
Buffer.compare 123687377 - diff last byte 6.0673 ms/op 6.8171 ms/op 0.89
byteArrayEquals 32 - random bytes 5.3210 ns/op 5.4290 ns/op 0.98
Buffer.compare 32 - random bytes 60.783 ns/op 63.055 ns/op 0.96
byteArrayEquals 1024 - random bytes 5.2260 ns/op 5.3640 ns/op 0.97
Buffer.compare 1024 - random bytes 60.412 ns/op 63.037 ns/op 0.96
byteArrayEquals 16384 - random bytes 5.2550 ns/op 5.3440 ns/op 0.98
Buffer.compare 16384 - random bytes 60.083 ns/op 60.823 ns/op 0.99
byteArrayEquals 123687377 - random bytes 8.7700 ns/op 8.5500 ns/op 1.03
Buffer.compare 123687377 - random bytes 63.360 ns/op 63.970 ns/op 0.99
regular array get 100000 times 44.606 us/op 44.905 us/op 0.99
wrappedArray get 100000 times 44.629 us/op 44.946 us/op 0.99
arrayWithProxy get 100000 times 14.833 ms/op 13.887 ms/op 1.07
ssz.Root.equals 53.911 ns/op 54.186 ns/op 0.99
byteArrayEquals 53.025 ns/op 53.098 ns/op 1.00
Buffer.compare 10.824 ns/op 10.935 ns/op 0.99
shuffle list - 16384 els 7.0168 ms/op 6.9966 ms/op 1.00
shuffle list - 250000 els 103.84 ms/op 103.19 ms/op 1.01
processSlot - 1 slots 16.019 us/op 15.511 us/op 1.03
processSlot - 32 slots 4.1626 ms/op 2.9902 ms/op 1.39
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 61.283 ms/op 62.986 ms/op 0.97
getCommitteeAssignments - req 1 vs - 250000 vc 2.5106 ms/op 2.5470 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 3.7052 ms/op 3.7672 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.0167 ms/op 4.1380 ms/op 0.97
findModifiedValidators - 10000 modified validators 565.46 ms/op 544.50 ms/op 1.04
findModifiedValidators - 1000 modified validators 470.57 ms/op 454.99 ms/op 1.03
findModifiedValidators - 100 modified validators 473.93 ms/op 451.89 ms/op 1.05
findModifiedValidators - 10 modified validators 441.22 ms/op 408.66 ms/op 1.08
findModifiedValidators - 1 modified validators 456.70 ms/op 400.31 ms/op 1.14
findModifiedValidators - no difference 435.26 ms/op 406.87 ms/op 1.07
compare ViewDUs 4.7834 s/op 4.7893 s/op 1.00
compare each validator Uint8Array 1.9416 s/op 1.8050 s/op 1.08
compare ViewDU to Uint8Array 1.1256 s/op 1.1644 s/op 0.97
migrate state 1000000 validators, 24 modified, 0 new 801.54 ms/op 867.24 ms/op 0.92
migrate state 1000000 validators, 1700 modified, 1000 new 1.0803 s/op 1.2260 s/op 0.88
migrate state 1000000 validators, 3400 modified, 2000 new 1.3899 s/op 1.3085 s/op 1.06
migrate state 1500000 validators, 24 modified, 0 new 880.59 ms/op 849.19 ms/op 1.04
migrate state 1500000 validators, 1700 modified, 1000 new 1.1468 s/op 1.1502 s/op 1.00
migrate state 1500000 validators, 3400 modified, 2000 new 1.3240 s/op 1.4314 s/op 0.92
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.1000 ns/op 4.9000 ns/op 1.04
state getBlockRootAtSlot - 250000 vs - 7PWei 669.64 ns/op 987.21 ns/op 0.68
computeProposers - vc 250000 9.6590 ms/op 9.8543 ms/op 0.98
computeEpochShuffling - vc 250000 109.67 ms/op 107.81 ms/op 1.02
getNextSyncCommittee - vc 250000 160.74 ms/op 163.62 ms/op 0.98
computeSigningRoot for AttestationData 26.425 us/op 27.870 us/op 0.95
hash AttestationData serialized data then Buffer.toString(base64) 2.4451 us/op 2.4045 us/op 1.02
toHexString serialized data 1.1640 us/op 1.0792 us/op 1.08
Buffer.toString(base64) 256.93 ns/op 231.62 ns/op 1.11

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review January 4, 2024 11:07
@twoeths twoeths requested a review from a team as a code owner January 4, 2024 11:07
@twoeths twoeths mentioned this pull request Jan 5, 2024
Merged
@twoeths twoeths marked this pull request as draft January 5, 2024 03:58
@twoeths twoeths mentioned this pull request Jan 8, 2024
Closed
6 tasks
@twoeths twoeths force-pushed the tuyen/n_historical_states_2 branch from 6caee55 to d1a3bd3 Compare January 9, 2024 07:11
@twoeths twoeths mentioned this pull request Jan 9, 2024
Merged
@twoeths twoeths closed this Jan 27, 2024
@twoeths twoeths deleted the tuyen/n_historical_states_2 branch January 27, 2024 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@twoeths @dapplion