Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add capella state transition perf tests #5807

Merged
merged 1 commit into from
Jul 27, 2023

Conversation

dapplion
Copy link
Contributor

Motivation

The mainnet beacon chain state has grown a lot, add performance tests for current mainnet test

Description

Add capella state transition perf tests

@dapplion dapplion requested a review from a team as a code owner July 26, 2023 16:22
@dapplion dapplion force-pushed the dapplion/capella-perf-tests branch from 9e4c7f6 to cb9d61a Compare July 26, 2023 17:12
@dapplion dapplion changed the title Add capella state transition perf tests chore: add capella state transition perf tests Jul 27, 2023
@dapplion dapplion force-pushed the dapplion/capella-perf-tests branch from cb9d61a to 21ae39e Compare July 27, 2023 08:26
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: cbcd78a Previous: 201dfc8 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 827.93 us/op 736.46 us/op 1.12
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 80.755 us/op 76.952 us/op 1.05
BLS verify - blst-native 1.2358 ms/op 1.2060 ms/op 1.02
BLS verifyMultipleSignatures 3 - blst-native 2.5212 ms/op 2.4125 ms/op 1.05
BLS verifyMultipleSignatures 8 - blst-native 5.3765 ms/op 5.2197 ms/op 1.03
BLS verifyMultipleSignatures 32 - blst-native 19.242 ms/op 18.944 ms/op 1.02
BLS aggregatePubkeys 32 - blst-native 25.286 us/op 24.972 us/op 1.01
BLS aggregatePubkeys 128 - blst-native 100.46 us/op 97.634 us/op 1.03
getAttestationsForBlock 60.718 ms/op 50.705 ms/op 1.20
isKnown best case - 1 super set check 288.00 ns/op 281.00 ns/op 1.02
isKnown normal case - 2 super set checks 306.00 ns/op 279.00 ns/op 1.10
isKnown worse case - 16 super set checks 288.00 ns/op 273.00 ns/op 1.05
CheckpointStateCache - add get delete 5.1550 us/op 4.8780 us/op 1.06
validate api signedAggregateAndProof - struct 2.8205 ms/op 2.7777 ms/op 1.02
validate gossip signedAggregateAndProof - struct 2.7855 ms/op 2.7793 ms/op 1.00
validate api attestation - struct 1.3952 ms/op 1.2721 ms/op 1.10
validate gossip attestation - struct 1.3626 ms/op 1.3023 ms/op 1.05
pickEth1Vote - no votes 1.3049 ms/op 1.1581 ms/op 1.13
pickEth1Vote - max votes 12.322 ms/op 9.2487 ms/op 1.33
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.276 ms/op 8.4711 ms/op 1.21
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.333 ms/op 13.962 ms/op 1.31
pickEth1Vote - Eth1Data fastSerialize value x2048 625.38 us/op 681.61 us/op 0.92
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.4807 ms/op 6.8071 ms/op 0.66
bytes32 toHexString 511.00 ns/op 474.00 ns/op 1.08
bytes32 Buffer.toString(hex) 296.00 ns/op 296.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 476.00 ns/op 420.00 ns/op 1.13
bytes32 Buffer.toString(hex) + 0x 293.00 ns/op 304.00 ns/op 0.96
Object access 1 prop 0.17600 ns/op 0.15700 ns/op 1.12
Map access 1 prop 0.15000 ns/op 0.15600 ns/op 0.96
Object get x1000 7.1600 ns/op 7.3120 ns/op 0.98
Map get x1000 0.69400 ns/op 0.59200 ns/op 1.17
Object set x1000 58.307 ns/op 46.762 ns/op 1.25
Map set x1000 48.081 ns/op 36.795 ns/op 1.31
Return object 10000 times 0.24540 ns/op 0.22370 ns/op 1.10
Throw Error 10000 times 4.1514 us/op 3.6596 us/op 1.13
fastMsgIdFn sha256 / 200 bytes 3.5020 us/op 3.1150 us/op 1.12
fastMsgIdFn h32 xxhash / 200 bytes 335.00 ns/op 267.00 ns/op 1.25
fastMsgIdFn h64 xxhash / 200 bytes 363.00 ns/op 334.00 ns/op 1.09
fastMsgIdFn sha256 / 1000 bytes 11.387 us/op 10.813 us/op 1.05
fastMsgIdFn h32 xxhash / 1000 bytes 425.00 ns/op 393.00 ns/op 1.08
fastMsgIdFn h64 xxhash / 1000 bytes 415.00 ns/op 408.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 103.86 us/op 99.007 us/op 1.05
fastMsgIdFn h32 xxhash / 10000 bytes 1.9340 us/op 1.7950 us/op 1.08
fastMsgIdFn h64 xxhash / 10000 bytes 1.3290 us/op 1.2430 us/op 1.07
enrSubnets - fastDeserialize 64 bits 1.3150 us/op 1.1700 us/op 1.12
enrSubnets - ssz BitVector 64 bits 430.00 ns/op 407.00 ns/op 1.06
enrSubnets - fastDeserialize 4 bits 191.00 ns/op 166.00 ns/op 1.15
enrSubnets - ssz BitVector 4 bits 445.00 ns/op 407.00 ns/op 1.09
prioritizePeers score -10:0 att 32-0.1 sync 2-0 133.29 us/op 98.573 us/op 1.35
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 127.81 us/op 124.61 us/op 1.03
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 168.40 us/op 154.11 us/op 1.09
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 314.57 us/op 274.80 us/op 1.14
prioritizePeers score 0:0 att 64-1 sync 4-1 366.73 us/op 328.72 us/op 1.12
array of 16000 items push then shift 1.6518 us/op 1.5440 us/op 1.07
LinkedList of 16000 items push then shift 9.1020 ns/op 8.7310 ns/op 1.04
array of 16000 items push then pop 66.549 ns/op 57.392 ns/op 1.16
LinkedList of 16000 items push then pop 8.8480 ns/op 8.5090 ns/op 1.04
array of 24000 items push then shift 2.4289 us/op 2.3634 us/op 1.03
LinkedList of 24000 items push then shift 9.0430 ns/op 8.8740 ns/op 1.02
array of 24000 items push then pop 115.55 ns/op 106.45 ns/op 1.09
LinkedList of 24000 items push then pop 8.7780 ns/op 8.6820 ns/op 1.01
intersect bitArray bitLen 8 6.8950 ns/op 6.8440 ns/op 1.01
intersect array and set length 8 54.456 ns/op 61.103 ns/op 0.89
intersect bitArray bitLen 128 32.138 ns/op 32.333 ns/op 0.99
intersect array and set length 128 761.07 ns/op 804.44 ns/op 0.95
bitArray.getTrueBitIndexes() bitLen 128 1.6450 us/op 1.6500 us/op 1.00
bitArray.getTrueBitIndexes() bitLen 248 2.5780 us/op 2.6290 us/op 0.98
bitArray.getTrueBitIndexes() bitLen 512 4.9050 us/op 5.0300 us/op 0.98
Buffer.concat 32 items 956.00 ns/op 977.00 ns/op 0.98
Uint8Array.set 32 items 1.7830 us/op 1.7740 us/op 1.01
transfer serialized Status (84 B) 1.8190 us/op 1.8300 us/op 0.99
copy serialized Status (84 B) 1.5220 us/op 1.5190 us/op 1.00
transfer serialized SignedVoluntaryExit (112 B) 1.8950 us/op 1.9490 us/op 0.97
copy serialized SignedVoluntaryExit (112 B) 1.5830 us/op 1.5780 us/op 1.00
transfer serialized ProposerSlashing (416 B) 2.0800 us/op 2.2210 us/op 0.94
copy serialized ProposerSlashing (416 B) 1.9800 us/op 1.7850 us/op 1.11
transfer serialized Attestation (485 B) 2.1590 us/op 1.9960 us/op 1.08
copy serialized Attestation (485 B) 2.1230 us/op 1.7470 us/op 1.22
transfer serialized AttesterSlashing (33232 B) 2.1580 us/op 2.0960 us/op 1.03
copy serialized AttesterSlashing (33232 B) 5.1430 us/op 4.8500 us/op 1.06
transfer serialized Small SignedBeaconBlock (128000 B) 2.4150 us/op 2.3500 us/op 1.03
copy serialized Small SignedBeaconBlock (128000 B) 14.181 us/op 13.614 us/op 1.04
transfer serialized Avg SignedBeaconBlock (200000 B) 2.7560 us/op 2.7240 us/op 1.01
copy serialized Avg SignedBeaconBlock (200000 B) 22.528 us/op 22.651 us/op 0.99
transfer serialized BlobsSidecar (524380 B) 2.8830 us/op 2.7100 us/op 1.06
copy serialized BlobsSidecar (524380 B) 79.173 us/op 123.40 us/op 0.64
transfer serialized Big SignedBeaconBlock (1000000 B) 3.0640 us/op 3.4040 us/op 0.90
copy serialized Big SignedBeaconBlock (1000000 B) 167.78 us/op 163.88 us/op 1.02
pass gossip attestations to forkchoice per slot 2.1309 ms/op 2.1666 ms/op 0.98
forkChoice updateHead vc 100000 bc 64 eq 0 2.4857 ms/op 2.1244 ms/op 1.17
forkChoice updateHead vc 600000 bc 64 eq 0 11.246 ms/op 14.343 ms/op 0.78
forkChoice updateHead vc 1000000 bc 64 eq 0 18.319 ms/op 23.352 ms/op 0.78
forkChoice updateHead vc 600000 bc 320 eq 0 16.409 ms/op 17.349 ms/op 0.95
forkChoice updateHead vc 600000 bc 1200 eq 0 86.165 ms/op 88.877 ms/op 0.97
forkChoice updateHead vc 600000 bc 64 eq 1000 19.203 ms/op 22.864 ms/op 0.84
forkChoice updateHead vc 600000 bc 64 eq 10000 21.065 ms/op 24.458 ms/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 300000 28.737 ms/op 30.592 ms/op 0.94
computeDeltas 3.9651 ms/op 3.2318 ms/op 1.23
computeProposerBoostScoreFromBalances 389.45 us/op 392.91 us/op 0.99
altair processAttestation - 250000 vs - 7PWei normalcase 2.2547 ms/op 2.6320 ms/op 0.86
altair processAttestation - 250000 vs - 7PWei worstcase 3.2458 ms/op 4.2532 ms/op 0.76
altair processAttestation - setStatus - 1/6 committees join 182.56 us/op 196.44 us/op 0.93
altair processAttestation - setStatus - 1/3 committees join 349.81 us/op 355.23 us/op 0.98
altair processAttestation - setStatus - 1/2 committees join 466.29 us/op 475.73 us/op 0.98
altair processAttestation - setStatus - 2/3 committees join 585.80 us/op 605.79 us/op 0.97
altair processAttestation - setStatus - 4/5 committees join 784.32 us/op 853.38 us/op 0.92
altair processAttestation - setStatus - 100% committees join 932.09 us/op 997.52 us/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase 10.340 ms/op 11.037 ms/op 0.94
altair processBlock - 250000 vs - 7PWei normalcase hashState 17.234 ms/op 18.173 ms/op 0.95
altair processBlock - 250000 vs - 7PWei worstcase 41.760 ms/op 39.992 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 64.155 ms/op 63.599 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 2.7961 ms/op 2.6827 ms/op 1.04
phase0 processBlock - 250000 vs - 7PWei worstcase 34.870 ms/op 31.686 ms/op 1.10
altair processEth1Data - 250000 vs - 7PWei normalcase 498.54 us/op 536.23 us/op 0.93
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 17.554 us/op 10.017 us/op 1.75
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 82.894 us/op 84.141 us/op 0.99
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 28.724 us/op 26.863 us/op 1.07
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 20.265 us/op 15.207 us/op 1.33
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 132.37 us/op 166.59 us/op 0.79
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 2.1294 ms/op 1.5911 ms/op 1.34
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.9914 ms/op 1.8075 ms/op 1.10
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 3.1095 ms/op 2.1619 ms/op 1.44
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.7323 ms/op 4.5320 ms/op 0.82
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.5667 ms/op 2.7610 ms/op 0.93
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.1677 ms/op 6.7336 ms/op 0.77
Tree 40 250000 create 318.59 ms/op 412.15 ms/op 0.77
Tree 40 250000 get(125000) 199.39 ns/op 214.60 ns/op 0.93
Tree 40 250000 set(125000) 887.51 ns/op 1.0545 us/op 0.84
Tree 40 250000 toArray() 22.349 ms/op 21.457 ms/op 1.04
Tree 40 250000 iterate all - toArray() + loop 23.269 ms/op 23.558 ms/op 0.99
Tree 40 250000 iterate all - get(i) 75.811 ms/op 74.195 ms/op 1.02
MutableVector 250000 create 16.263 ms/op 10.088 ms/op 1.61
MutableVector 250000 get(125000) 6.7640 ns/op 6.5350 ns/op 1.04
MutableVector 250000 set(125000) 266.28 ns/op 288.00 ns/op 0.92
MutableVector 250000 toArray() 3.2110 ms/op 3.9755 ms/op 0.81
MutableVector 250000 iterate all - toArray() + loop 3.6530 ms/op 4.1078 ms/op 0.89
MutableVector 250000 iterate all - get(i) 1.5994 ms/op 1.6852 ms/op 0.95
Array 250000 create 3.4581 ms/op 3.2554 ms/op 1.06
Array 250000 clone - spread 1.2647 ms/op 1.0297 ms/op 1.23
Array 250000 get(125000) 0.59700 ns/op 0.50300 ns/op 1.19
Array 250000 set(125000) 0.69800 ns/op 0.58500 ns/op 1.19
Array 250000 iterate all - loop 89.043 us/op 113.17 us/op 0.79
effectiveBalanceIncrements clone Uint8Array 300000 25.474 us/op 23.943 us/op 1.06
effectiveBalanceIncrements clone MutableVector 300000 256.00 ns/op 268.00 ns/op 0.96
effectiveBalanceIncrements rw all Uint8Array 300000 180.50 us/op 182.05 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 74.403 ms/op 76.216 ms/op 0.98
phase0 afterProcessEpoch - 250000 vs - 7PWei 114.27 ms/op 116.38 ms/op 0.98
phase0 beforeProcessEpoch - 250000 vs - 7PWei 33.057 ms/op 30.847 ms/op 1.07
altair processEpoch - mainnet_e81889 352.29 ms/op 313.97 ms/op 1.12
mainnet_e81889 - altair beforeProcessEpoch 49.973 ms/op 58.640 ms/op 0.85
mainnet_e81889 - altair processJustificationAndFinalization 21.830 us/op 15.578 us/op 1.40
mainnet_e81889 - altair processInactivityUpdates 6.3671 ms/op 5.0689 ms/op 1.26
mainnet_e81889 - altair processRewardsAndPenalties 56.415 ms/op 66.662 ms/op 0.85
mainnet_e81889 - altair processRegistryUpdates 3.8590 us/op 2.5930 us/op 1.49
mainnet_e81889 - altair processSlashings 964.00 ns/op 443.00 ns/op 2.18
mainnet_e81889 - altair processEth1DataReset 725.00 ns/op 539.00 ns/op 1.35
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3981 ms/op 1.2514 ms/op 1.92
mainnet_e81889 - altair processSlashingsReset 4.0770 us/op 2.6530 us/op 1.54
mainnet_e81889 - altair processRandaoMixesReset 6.1630 us/op 4.4260 us/op 1.39
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0760 us/op 710.00 ns/op 1.52
mainnet_e81889 - altair processParticipationFlagUpdates 2.6870 us/op 2.9580 us/op 0.91
mainnet_e81889 - altair processSyncCommitteeUpdates 899.00 ns/op 569.00 ns/op 1.58
mainnet_e81889 - altair afterProcessEpoch 130.10 ms/op 127.27 ms/op 1.02
capella processEpoch - mainnet_e217614 1.0950 s/op
mainnet_e217614 - capella beforeProcessEpoch 241.82 ms/op
mainnet_e217614 - capella processJustificationAndFinalization 20.449 us/op
mainnet_e217614 - capella processInactivityUpdates 15.270 ms/op
mainnet_e217614 - capella processRewardsAndPenalties 351.15 ms/op
mainnet_e217614 - capella processRegistryUpdates 29.756 us/op
mainnet_e217614 - capella processSlashings 860.00 ns/op
mainnet_e217614 - capella processEth1DataReset 856.00 ns/op
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.1143 ms/op
mainnet_e217614 - capella processSlashingsReset 3.1070 us/op
mainnet_e217614 - capella processRandaoMixesReset 6.6160 us/op
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0780 us/op
mainnet_e217614 - capella processParticipationFlagUpdates 3.2260 us/op
mainnet_e217614 - capella afterProcessEpoch 299.80 ms/op
phase0 processEpoch - mainnet_e58758 338.02 ms/op 359.90 ms/op 0.94
mainnet_e58758 - phase0 beforeProcessEpoch 118.30 ms/op 129.91 ms/op 0.91
mainnet_e58758 - phase0 processJustificationAndFinalization 20.298 us/op 13.882 us/op 1.46
mainnet_e58758 - phase0 processRewardsAndPenalties 64.585 ms/op 66.066 ms/op 0.98
mainnet_e58758 - phase0 processRegistryUpdates 13.247 us/op 11.076 us/op 1.20
mainnet_e58758 - phase0 processSlashings 510.00 ns/op 466.00 ns/op 1.09
mainnet_e58758 - phase0 processEth1DataReset 526.00 ns/op 458.00 ns/op 1.15
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.2616 ms/op 1.3583 ms/op 0.93
mainnet_e58758 - phase0 processSlashingsReset 2.2380 us/op 2.0530 us/op 1.09
mainnet_e58758 - phase0 processRandaoMixesReset 5.7390 us/op 4.6770 us/op 1.23
mainnet_e58758 - phase0 processHistoricalRootsUpdate 656.00 ns/op 539.00 ns/op 1.22
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.7780 us/op 2.9920 us/op 2.27
mainnet_e58758 - phase0 afterProcessEpoch 110.76 ms/op 97.915 ms/op 1.13
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3354 ms/op 1.2475 ms/op 1.07
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.4592 ms/op 1.4011 ms/op 1.76
altair processInactivityUpdates - 250000 normalcase 31.173 ms/op 24.568 ms/op 1.27
altair processInactivityUpdates - 250000 worstcase 25.842 ms/op 23.642 ms/op 1.09
phase0 processRegistryUpdates - 250000 normalcase 8.7660 us/op 10.271 us/op 0.85
phase0 processRegistryUpdates - 250000 badcase_full_deposits 436.44 us/op 380.47 us/op 1.15
phase0 processRegistryUpdates - 250000 worstcase 0.5 134.45 ms/op 133.40 ms/op 1.01
altair processRewardsAndPenalties - 250000 normalcase 72.756 ms/op 69.278 ms/op 1.05
altair processRewardsAndPenalties - 250000 worstcase 76.056 ms/op 71.947 ms/op 1.06
phase0 getAttestationDeltas - 250000 normalcase 8.4802 ms/op 8.2133 ms/op 1.03
phase0 getAttestationDeltas - 250000 worstcase 8.8686 ms/op 8.9916 ms/op 0.99
phase0 processSlashings - 250000 worstcase 2.4255 ms/op 2.4953 ms/op 0.97
altair processSyncCommitteeUpdates - 250000 163.15 ms/op 173.22 ms/op 0.94
BeaconState.hashTreeRoot - No change 269.00 ns/op 327.00 ns/op 0.82
BeaconState.hashTreeRoot - 1 full validator 57.111 us/op 54.734 us/op 1.04
BeaconState.hashTreeRoot - 32 full validator 554.10 us/op 617.30 us/op 0.90
BeaconState.hashTreeRoot - 512 full validator 5.5380 ms/op 5.6991 ms/op 0.97
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 66.741 us/op 68.706 us/op 0.97
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 956.39 us/op 1.0228 ms/op 0.94
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.577 ms/op 13.660 ms/op 0.92
BeaconState.hashTreeRoot - 1 balances 53.348 us/op 56.762 us/op 0.94
BeaconState.hashTreeRoot - 32 balances 472.56 us/op 473.04 us/op 1.00
BeaconState.hashTreeRoot - 512 balances 4.6792 ms/op 5.1815 ms/op 0.90
BeaconState.hashTreeRoot - 250000 balances 77.177 ms/op 80.260 ms/op 0.96
aggregationBits - 2048 els - zipIndexesInBitList 16.530 us/op 20.062 us/op 0.82
regular array get 100000 times 43.845 us/op 45.652 us/op 0.96
wrappedArray get 100000 times 34.067 us/op 34.819 us/op 0.98
arrayWithProxy get 100000 times 15.779 ms/op 15.525 ms/op 1.02
ssz.Root.equals 234.00 ns/op 255.00 ns/op 0.92
byteArrayEquals 228.00 ns/op 262.00 ns/op 0.87
shuffle list - 16384 els 7.1136 ms/op 7.3421 ms/op 0.97
shuffle list - 250000 els 105.14 ms/op 107.97 ms/op 0.97
processSlot - 1 slots 8.5790 us/op 9.5040 us/op 0.90
processSlot - 32 slots 1.3839 ms/op 1.4373 ms/op 0.96
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 57.905 ms/op 59.367 ms/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 2.5848 ms/op 2.5965 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 3.8193 ms/op 3.8568 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 4.1710 ms/op 4.2408 ms/op 0.98
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.2000 ns/op 5.2800 ns/op 0.98
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0083 us/op 636.85 ns/op 1.58
computeProposers - vc 250000 9.4209 ms/op 9.3891 ms/op 1.00
computeEpochShuffling - vc 250000 109.40 ms/op 107.68 ms/op 1.02
getNextSyncCommittee - vc 250000 151.56 ms/op 156.90 ms/op 0.97
computeSigningRoot for AttestationData 13.254 us/op 13.680 us/op 0.97
hash AttestationData serialized data then Buffer.toString(base64) 2.4134 us/op 2.4098 us/op 1.00
toHexString serialized data 1.1660 us/op 1.0786 us/op 1.08
Buffer.toString(base64) 251.59 ns/op 215.32 ns/op 1.17

by benchmarkbot/action

@dapplion
Copy link
Contributor Author

Results in CI

   capella processEpoch - mainnet_e217614
Downloading file https://github.com/dapplion/ethereum-consensus-test-data/releases/download/v0.1.0/state_mainnet_6963647.ssz
    ✔ capella processEpoch - mainnet_e217614                             0.9132258 ops/s    1.095019  s/op        -         23 runs   26.4 s
    capella processEpoch steps - mainnet_e217614
      ✔ mainnet_e217614 - capella beforeProcessEpoch                        4.135378 ops/s    241.8159 ms/op        -         82 runs   20.6 s
      ✔ mainnet_e217614 - capella processJustificationAndFinalization       48902.15 ops/s    20.44900 us/op        -       1104 runs   2.49 s
      ✔ mainnet_e217614 - capella processInactivityUpdates                  65.48910 ops/s    15.26972 ms/op        -         29 runs   1.01 s
      ✔ mainnet_e217614 - capella processRewardsAndPenalties                2.847790 ops/s    351.1494 ms/op        -         70 runs   25.4 s
      ✔ mainnet_e217614 - capella processRegistryUpdates                    33606.67 ops/s    29.75600 us/op        -       2186 runs   3.56 s
      ✔ mainnet_e217614 - capella processSlashings                           1162791 ops/s    860.0000 ns/op        -       1975 runs   3.19 s
      ✔ mainnet_e217614 - capella processEth1DataReset                       1168224 ops/s    856.0000 ns/op        -       1315 runs   2.49 s
      ✔ mainnet_e217614 - capella processEffectiveBalanceUpdates            243.0548 ops/s    4.114298 ms/op        -         61 runs  0.939 s
      ✔ mainnet_e217614 - capella processSlashingsReset                     321853.9 ops/s    3.107000 us/op        -       1505 runs   2.69 s
      ✔ mainnet_e217614 - capella processRandaoMixesReset                   151148.7 ops/s    6.616000 us/op        -       3743 runs   5.11 s
      ✔ mainnet_e217614 - capella processHistoricalRootsUpdate              927643.8 ops/s    1.078000 us/op        -       5208 runs   6.94 s
      ✔ mainnet_e217614 - capella processParticipationFlagUpdates           309981.4 ops/s    3.226000 us/op        -       1122 runs   2.45 s
      ✔ mainnet_e217614 - capella afterProcessEpoch                         3.335558 ops/s    299.7999 ms/op        -         12 runs   5.43 s

Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The statistics are not close to mainnet, I think we need to improve our perf test with more validators

@dapplion
Copy link
Contributor Author

The statistics are not close to mainnet, I think we need to improve our perf test with more validators

Note I used a real state from mainnet from yesterday so the number of validators is ok. What can deviate metrics in production is peripheral use making the machine overloaded, etc.

@wemeetagain wemeetagain merged commit 0c440c1 into unstable Jul 27, 2023
11 checks passed
@wemeetagain wemeetagain deleted the dapplion/capella-perf-tests branch July 27, 2023 18:29
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.10.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants