Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: getAttestationsForBlock performance issue #6367

Merged
merged 5 commits into from
Feb 1, 2024
Merged

fix: getAttestationsForBlock performance issue #6367

merged 5 commits into from
Feb 1, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 29, 2024

Motivation

Get aggregated attestations for block production may take up to >1s, this PR improve it

Description

  • Was able to reproduce the issue, it took ~1s in the benchmark, after all it takes ~100ms in the worse case
  • Some improvements:
    • Group attestations not just by attestation data but by slot & index so that we can get committee once and we run the participation function way less. This is possible thanks to the attestation validation against the state
    • The participation function was changed from seenAttestingIndices to notSeenAttestingIndices which makes the logic simpler
    • The attestation validation function is cached by beaconBlockRoot + targetEpoch
    • Ignore attestation data if the max possible score is even less than the lowest score for included attestations

Closes #6334

@twoeths twoeths changed the title Tuyen/get attestations for block fix: getAttestationsForBlock performance issue Jan 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 29, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: c824e40 Previous: ed43a98 Ratio
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.0660 us/op 18.346 us/op 0.33
Full benchmark results
Benchmark suite Current: c824e40 Previous: ed43a98 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 771.62 us/op 578.74 us/op 1.33
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 50.434 us/op 123.11 us/op 0.41
BLS verify - blst-native 1.0532 ms/op 1.3391 ms/op 0.79
BLS verifyMultipleSignatures 3 - blst-native 2.2466 ms/op 2.7858 ms/op 0.81
BLS verifyMultipleSignatures 8 - blst-native 4.9761 ms/op 6.1180 ms/op 0.81
BLS verifyMultipleSignatures 32 - blst-native 18.288 ms/op 22.196 ms/op 0.82
BLS verifyMultipleSignatures 64 - blst-native 36.020 ms/op 43.883 ms/op 0.82
BLS verifyMultipleSignatures 128 - blst-native 71.869 ms/op 87.836 ms/op 0.82
BLS deserializing 10000 signatures 782.96 ms/op 951.16 ms/op 0.82
BLS deserializing 100000 signatures 8.0194 s/op 9.2077 s/op 0.87
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1691 ms/op 1.3800 ms/op 0.85
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2142 ms/op 1.5968 ms/op 0.76
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.3529 ms/op 2.3782 ms/op 0.99
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.8720 ms/op 3.5077 ms/op 1.10
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.6211 ms/op 5.7801 ms/op 1.15
BLS aggregatePubkeys 32 - blst-native 22.131 us/op 26.442 us/op 0.84
BLS aggregatePubkeys 128 - blst-native 86.914 us/op 102.77 us/op 0.85
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 42.176 ms/op
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 37.772 ms/op
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 23.143 ms/op
getSlashingsAndExits - default max 183.71 us/op 150.22 us/op 1.22
getSlashingsAndExits - 2k 336.17 us/op 537.38 us/op 0.63
proposeBlockBody type=full, size=empty 3.9994 ms/op 5.3949 ms/op 0.74
isKnown best case - 1 super set check 321.00 ns/op 324.00 ns/op 0.99
isKnown normal case - 2 super set checks 314.00 ns/op 324.00 ns/op 0.97
isKnown worse case - 16 super set checks 313.00 ns/op 339.00 ns/op 0.92
CheckpointStateCache - add get delete 3.3620 us/op 6.0090 us/op 0.56
validate api signedAggregateAndProof - struct 2.2617 ms/op 2.8199 ms/op 0.80
validate gossip signedAggregateAndProof - struct 2.2719 ms/op 2.8096 ms/op 0.81
validate gossip attestation - vc 640000 1.0973 ms/op 1.3771 ms/op 0.80
batch validate gossip attestation - vc 640000 - chunk 32 130.97 us/op 167.40 us/op 0.78
batch validate gossip attestation - vc 640000 - chunk 64 119.39 us/op 146.44 us/op 0.82
batch validate gossip attestation - vc 640000 - chunk 128 112.83 us/op 133.74 us/op 0.84
batch validate gossip attestation - vc 640000 - chunk 256 111.52 us/op 130.03 us/op 0.86
pickEth1Vote - no votes 960.17 us/op 1.2819 ms/op 0.75
pickEth1Vote - max votes 8.9056 ms/op 7.8639 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.548 ms/op 17.378 ms/op 0.78
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 28.861 ms/op 33.090 ms/op 0.87
pickEth1Vote - Eth1Data fastSerialize value x2048 417.43 us/op 614.39 us/op 0.68
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.2580 ms/op 4.5916 ms/op 1.58
bytes32 toHexString 452.00 ns/op 537.00 ns/op 0.84
bytes32 Buffer.toString(hex) 356.00 ns/op 298.00 ns/op 1.19
bytes32 Buffer.toString(hex) from Uint8Array 478.00 ns/op 444.00 ns/op 1.08
bytes32 Buffer.toString(hex) + 0x 341.00 ns/op 296.00 ns/op 1.15
Object access 1 prop 0.20900 ns/op 0.15900 ns/op 1.31
Map access 1 prop 0.18700 ns/op 0.15500 ns/op 1.21
Object get x1000 5.6060 ns/op 7.7870 ns/op 0.72
Map get x1000 0.72100 ns/op 0.87200 ns/op 0.83
Object set x1000 24.227 ns/op 59.082 ns/op 0.41
Map set x1000 16.941 ns/op 45.448 ns/op 0.37
Return object 10000 times 0.22600 ns/op 0.24760 ns/op 0.91
Throw Error 10000 times 2.6791 us/op 3.8709 us/op 0.69
fastMsgIdFn sha256 / 200 bytes 1.9140 us/op 3.3990 us/op 0.56
fastMsgIdFn h32 xxhash / 200 bytes 287.00 ns/op 293.00 ns/op 0.98
fastMsgIdFn h64 xxhash / 200 bytes 336.00 ns/op 346.00 ns/op 0.97
fastMsgIdFn sha256 / 1000 bytes 6.0870 us/op 11.627 us/op 0.52
fastMsgIdFn h32 xxhash / 1000 bytes 408.00 ns/op 447.00 ns/op 0.91
fastMsgIdFn h64 xxhash / 1000 bytes 400.00 ns/op 444.00 ns/op 0.90
fastMsgIdFn sha256 / 10000 bytes 52.840 us/op 103.00 us/op 0.51
fastMsgIdFn h32 xxhash / 10000 bytes 1.7880 us/op 2.0000 us/op 0.89
fastMsgIdFn h64 xxhash / 10000 bytes 1.2250 us/op 1.3540 us/op 0.90
send data - 1000 256B messages 12.059 ms/op 19.166 ms/op 0.63
send data - 1000 512B messages 16.506 ms/op 27.931 ms/op 0.59
send data - 1000 1024B messages 30.264 ms/op 40.623 ms/op 0.75
send data - 1000 1200B messages 28.540 ms/op 38.023 ms/op 0.75
send data - 1000 2048B messages 38.984 ms/op 47.976 ms/op 0.81
send data - 1000 4096B messages 33.214 ms/op 43.221 ms/op 0.77
send data - 1000 16384B messages 91.420 ms/op 121.25 ms/op 0.75
send data - 1000 65536B messages 420.50 ms/op 450.15 ms/op 0.93
enrSubnets - fastDeserialize 64 bits 941.00 ns/op 1.3180 us/op 0.71
enrSubnets - ssz BitVector 64 bits 448.00 ns/op 435.00 ns/op 1.03
enrSubnets - fastDeserialize 4 bits 209.00 ns/op 170.00 ns/op 1.23
enrSubnets - ssz BitVector 4 bits 425.00 ns/op 476.00 ns/op 0.89
prioritizePeers score -10:0 att 32-0.1 sync 2-0 68.595 us/op 106.48 us/op 0.64
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 80.991 us/op 134.85 us/op 0.60
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 115.50 us/op 190.10 us/op 0.61
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 200.50 us/op 325.39 us/op 0.62
prioritizePeers score 0:0 att 64-1 sync 4-1 220.03 us/op 366.94 us/op 0.60
array of 16000 items push then shift 1.2540 us/op 1.6773 us/op 0.75
LinkedList of 16000 items push then shift 6.7430 ns/op 9.1230 ns/op 0.74
array of 16000 items push then pop 56.561 ns/op 101.46 ns/op 0.56
LinkedList of 16000 items push then pop 5.6490 ns/op 8.8190 ns/op 0.64
array of 24000 items push then shift 1.7876 us/op 2.5267 us/op 0.71
LinkedList of 24000 items push then shift 6.3700 ns/op 9.2090 ns/op 0.69
array of 24000 items push then pop 146.19 ns/op 112.84 ns/op 1.30
LinkedList of 24000 items push then pop 5.6320 ns/op 9.2360 ns/op 0.61
intersect bitArray bitLen 8 4.5530 ns/op 6.5410 ns/op 0.70
intersect array and set length 8 49.947 ns/op 65.140 ns/op 0.77
intersect bitArray bitLen 128 28.342 ns/op 34.419 ns/op 0.82
intersect array and set length 128 699.81 ns/op 895.84 ns/op 0.78
bitArray.getTrueBitIndexes() bitLen 128 1.4830 us/op 1.6030 us/op 0.93
bitArray.getTrueBitIndexes() bitLen 248 2.5470 us/op 2.4420 us/op 1.04
bitArray.getTrueBitIndexes() bitLen 512 4.8440 us/op 5.0640 us/op 0.96
Buffer.concat 32 items 851.00 ns/op 1.0940 us/op 0.78
Uint8Array.set 32 items 1.7270 us/op 1.9700 us/op 0.88
Set add up to 64 items then delete first 1.7669 us/op 4.9905 us/op 0.35
OrderedSet add up to 64 items then delete first 2.7446 us/op 6.1921 us/op 0.44
Set add up to 64 items then delete last 1.9770 us/op 5.0000 us/op 0.40
OrderedSet add up to 64 items then delete last 3.1434 us/op 6.1881 us/op 0.51
Set add up to 64 items then delete middle 1.9476 us/op 4.6460 us/op 0.42
OrderedSet add up to 64 items then delete middle 4.2832 us/op 7.2685 us/op 0.59
Set add up to 128 items then delete first 3.9817 us/op 9.3606 us/op 0.43
OrderedSet add up to 128 items then delete first 6.1486 us/op 12.577 us/op 0.49
Set add up to 128 items then delete last 3.7845 us/op 9.2721 us/op 0.41
OrderedSet add up to 128 items then delete last 5.7057 us/op 11.756 us/op 0.49
Set add up to 128 items then delete middle 3.8080 us/op 9.2847 us/op 0.41
OrderedSet add up to 128 items then delete middle 10.706 us/op 17.072 us/op 0.63
Set add up to 256 items then delete first 7.6500 us/op 19.531 us/op 0.39
OrderedSet add up to 256 items then delete first 12.181 us/op 25.885 us/op 0.47
Set add up to 256 items then delete last 7.4147 us/op 18.663 us/op 0.40
OrderedSet add up to 256 items then delete last 11.323 us/op 23.459 us/op 0.48
Set add up to 256 items then delete middle 7.3893 us/op 18.434 us/op 0.40
OrderedSet add up to 256 items then delete middle 30.456 us/op 46.073 us/op 0.66
transfer serialized Status (84 B) 1.3350 us/op 1.7540 us/op 0.76
copy serialized Status (84 B) 1.0740 us/op 1.2460 us/op 0.86
transfer serialized SignedVoluntaryExit (112 B) 1.4110 us/op 1.8540 us/op 0.76
copy serialized SignedVoluntaryExit (112 B) 1.0900 us/op 1.3280 us/op 0.82
transfer serialized ProposerSlashing (416 B) 1.6500 us/op 2.0740 us/op 0.80
copy serialized ProposerSlashing (416 B) 1.3440 us/op 1.7710 us/op 0.76
transfer serialized Attestation (485 B) 1.7630 us/op 2.1280 us/op 0.83
copy serialized Attestation (485 B) 1.2740 us/op 1.9510 us/op 0.65
transfer serialized AttesterSlashing (33232 B) 1.6530 us/op 2.0740 us/op 0.80
copy serialized AttesterSlashing (33232 B) 3.3430 us/op 6.2360 us/op 0.54
transfer serialized Small SignedBeaconBlock (128000 B) 1.9660 us/op 3.3620 us/op 0.58
copy serialized Small SignedBeaconBlock (128000 B) 8.4130 us/op 19.557 us/op 0.43
transfer serialized Avg SignedBeaconBlock (200000 B) 1.9550 us/op 3.7370 us/op 0.52
copy serialized Avg SignedBeaconBlock (200000 B) 12.289 us/op 27.306 us/op 0.45
transfer serialized BlobsSidecar (524380 B) 2.4790 us/op 2.9740 us/op 0.83
copy serialized BlobsSidecar (524380 B) 78.884 us/op 106.39 us/op 0.74
transfer serialized Big SignedBeaconBlock (1000000 B) 2.6800 us/op 3.7080 us/op 0.72
copy serialized Big SignedBeaconBlock (1000000 B) 146.12 us/op 270.58 us/op 0.54
pass gossip attestations to forkchoice per slot 2.7850 ms/op 4.5160 ms/op 0.62
forkChoice updateHead vc 100000 bc 64 eq 0 500.75 us/op 692.33 us/op 0.72
forkChoice updateHead vc 600000 bc 64 eq 0 2.6977 ms/op 4.5256 ms/op 0.60
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3795 ms/op 7.3672 ms/op 0.59
forkChoice updateHead vc 600000 bc 320 eq 0 2.6580 ms/op 4.3301 ms/op 0.61
forkChoice updateHead vc 600000 bc 1200 eq 0 2.7137 ms/op 4.4835 ms/op 0.61
forkChoice updateHead vc 600000 bc 7200 eq 0 3.5050 ms/op 5.6148 ms/op 0.62
forkChoice updateHead vc 600000 bc 64 eq 1000 9.3722 ms/op 12.063 ms/op 0.78
forkChoice updateHead vc 600000 bc 64 eq 10000 9.1487 ms/op 12.205 ms/op 0.75
forkChoice updateHead vc 600000 bc 64 eq 300000 11.574 ms/op 16.509 ms/op 0.70
computeDeltas 500000 validators 300 proto nodes 3.0006 ms/op 7.2644 ms/op 0.41
computeDeltas 500000 validators 1200 proto nodes 3.0369 ms/op 7.0790 ms/op 0.43
computeDeltas 500000 validators 7200 proto nodes 2.9046 ms/op 6.7170 ms/op 0.43
computeDeltas 750000 validators 300 proto nodes 4.5678 ms/op 10.255 ms/op 0.45
computeDeltas 750000 validators 1200 proto nodes 4.7103 ms/op 9.8883 ms/op 0.48
computeDeltas 750000 validators 7200 proto nodes 4.6605 ms/op 10.175 ms/op 0.46
computeDeltas 1400000 validators 300 proto nodes 9.0341 ms/op 19.385 ms/op 0.47
computeDeltas 1400000 validators 1200 proto nodes 9.0052 ms/op 19.990 ms/op 0.45
computeDeltas 1400000 validators 7200 proto nodes 9.1560 ms/op 20.817 ms/op 0.44
computeDeltas 2100000 validators 300 proto nodes 13.111 ms/op 30.793 ms/op 0.43
computeDeltas 2100000 validators 1200 proto nodes 13.354 ms/op 28.865 ms/op 0.46
computeDeltas 2100000 validators 7200 proto nodes 13.563 ms/op 29.728 ms/op 0.46
altair processAttestation - 250000 vs - 7PWei normalcase 1.4644 ms/op 3.5391 ms/op 0.41
altair processAttestation - 250000 vs - 7PWei worstcase 2.4221 ms/op 5.0751 ms/op 0.48
altair processAttestation - setStatus - 1/6 committees join 77.744 us/op 161.70 us/op 0.48
altair processAttestation - setStatus - 1/3 committees join 147.90 us/op 282.36 us/op 0.52
altair processAttestation - setStatus - 1/2 committees join 199.78 us/op 392.89 us/op 0.51
altair processAttestation - setStatus - 2/3 committees join 262.15 us/op 475.54 us/op 0.55
altair processAttestation - setStatus - 4/5 committees join 386.41 us/op 721.47 us/op 0.54
altair processAttestation - setStatus - 100% committees join 461.26 us/op 794.52 us/op 0.58
altair processBlock - 250000 vs - 7PWei normalcase 7.0406 ms/op 11.194 ms/op 0.63
altair processBlock - 250000 vs - 7PWei normalcase hashState 30.347 ms/op 44.829 ms/op 0.68
altair processBlock - 250000 vs - 7PWei worstcase 30.763 ms/op 42.933 ms/op 0.72
altair processBlock - 250000 vs - 7PWei worstcase hashState 72.207 ms/op 114.30 ms/op 0.63
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1261 ms/op 3.2319 ms/op 0.66
phase0 processBlock - 250000 vs - 7PWei worstcase 24.850 ms/op 35.322 ms/op 0.70
altair processEth1Data - 250000 vs - 7PWei normalcase 312.93 us/op 705.00 us/op 0.44
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.4250 us/op 18.556 us/op 0.35
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 54.206 us/op 59.278 us/op 0.91
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 19.048 us/op 26.068 us/op 0.73
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.0660 us/op 18.346 us/op 0.33
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 132.48 us/op 223.85 us/op 0.59
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.1146 ms/op 1.6376 ms/op 0.68
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.9752 ms/op 2.0279 ms/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 932.41 us/op 1.9046 ms/op 0.49
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.3962 ms/op 4.2016 ms/op 0.57
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.7765 ms/op 3.0264 ms/op 0.59
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.4167 ms/op 6.4870 ms/op 0.68
Tree 40 250000 create 274.79 ms/op 478.30 ms/op 0.57
Tree 40 250000 get(125000) 113.62 ns/op 224.69 ns/op 0.51
Tree 40 250000 set(125000) 805.00 ns/op 1.3882 us/op 0.58
Tree 40 250000 toArray() 18.709 ms/op 26.413 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 17.313 ms/op 26.006 ms/op 0.67
Tree 40 250000 iterate all - get(i) 42.598 ms/op 81.027 ms/op 0.53
MutableVector 250000 create 10.361 ms/op 15.339 ms/op 0.68
MutableVector 250000 get(125000) 6.0580 ns/op 6.8570 ns/op 0.88
MutableVector 250000 set(125000) 208.92 ns/op 555.39 ns/op 0.38
MutableVector 250000 toArray() 2.1718 ms/op 4.9517 ms/op 0.44
MutableVector 250000 iterate all - toArray() + loop 2.2740 ms/op 5.0667 ms/op 0.45
MutableVector 250000 iterate all - get(i) 1.3469 ms/op 1.5956 ms/op 0.84
Array 250000 create 2.0505 ms/op 4.6289 ms/op 0.44
Array 250000 clone - spread 1.1970 ms/op 1.7595 ms/op 0.68
Array 250000 get(125000) 1.0690 ns/op 1.9020 ns/op 0.56
Array 250000 set(125000) 1.2800 ns/op 5.6420 ns/op 0.23
Array 250000 iterate all - loop 155.06 us/op 175.16 us/op 0.89
effectiveBalanceIncrements clone Uint8Array 300000 14.773 us/op 70.213 us/op 0.21
effectiveBalanceIncrements clone MutableVector 300000 444.00 ns/op 508.00 ns/op 0.87
effectiveBalanceIncrements rw all Uint8Array 300000 187.41 us/op 204.58 us/op 0.92
effectiveBalanceIncrements rw all MutableVector 300000 76.427 ms/op 171.28 ms/op 0.45
phase0 afterProcessEpoch - 250000 vs - 7PWei 73.965 ms/op 132.30 ms/op 0.56
phase0 beforeProcessEpoch - 250000 vs - 7PWei 39.244 ms/op 47.557 ms/op 0.83
altair processEpoch - mainnet_e81889 452.22 ms/op 563.96 ms/op 0.80
mainnet_e81889 - altair beforeProcessEpoch 87.858 ms/op 94.742 ms/op 0.93
mainnet_e81889 - altair processJustificationAndFinalization 12.820 us/op 17.674 us/op 0.73
mainnet_e81889 - altair processInactivityUpdates 7.6872 ms/op 5.9791 ms/op 1.29
mainnet_e81889 - altair processRewardsAndPenalties 67.038 ms/op 43.043 ms/op 1.56
mainnet_e81889 - altair processRegistryUpdates 2.3460 us/op 2.6080 us/op 0.90
mainnet_e81889 - altair processSlashings 635.00 ns/op 530.00 ns/op 1.20
mainnet_e81889 - altair processEth1DataReset 722.00 ns/op 522.00 ns/op 1.38
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6178 ms/op 1.4473 ms/op 1.12
mainnet_e81889 - altair processSlashingsReset 2.8690 us/op 3.4870 us/op 0.82
mainnet_e81889 - altair processRandaoMixesReset 7.9380 us/op 7.2750 us/op 1.09
mainnet_e81889 - altair processHistoricalRootsUpdate 776.00 ns/op 1.0420 us/op 0.74
mainnet_e81889 - altair processParticipationFlagUpdates 1.9380 us/op 2.1650 us/op 0.90
mainnet_e81889 - altair processSyncCommitteeUpdates 766.00 ns/op 1.3480 us/op 0.57
mainnet_e81889 - altair afterProcessEpoch 82.198 ms/op 123.33 ms/op 0.67
capella processEpoch - mainnet_e217614 1.9322 s/op 2.5325 s/op 0.76
mainnet_e217614 - capella beforeProcessEpoch 407.69 ms/op 617.58 ms/op 0.66
mainnet_e217614 - capella processJustificationAndFinalization 11.388 us/op 31.018 us/op 0.37
mainnet_e217614 - capella processInactivityUpdates 16.677 ms/op 31.802 ms/op 0.52
mainnet_e217614 - capella processRewardsAndPenalties 372.24 ms/op 447.85 ms/op 0.83
mainnet_e217614 - capella processRegistryUpdates 19.776 us/op 32.475 us/op 0.61
mainnet_e217614 - capella processSlashings 718.00 ns/op 1.0660 us/op 0.67
mainnet_e217614 - capella processEth1DataReset 576.00 ns/op 1.1550 us/op 0.50
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.6204 ms/op 6.4566 ms/op 0.56
mainnet_e217614 - capella processSlashingsReset 1.5630 us/op 5.3090 us/op 0.29
mainnet_e217614 - capella processRandaoMixesReset 3.5900 us/op 8.6010 us/op 0.42
mainnet_e217614 - capella processHistoricalRootsUpdate 592.00 ns/op 1.4180 us/op 0.42
mainnet_e217614 - capella processParticipationFlagUpdates 1.0840 us/op 2.4530 us/op 0.44
mainnet_e217614 - capella afterProcessEpoch 216.92 ms/op 322.09 ms/op 0.67
phase0 processEpoch - mainnet_e58758 376.44 ms/op 534.80 ms/op 0.70
mainnet_e58758 - phase0 beforeProcessEpoch 118.55 ms/op 163.00 ms/op 0.73
mainnet_e58758 - phase0 processJustificationAndFinalization 11.304 us/op 24.980 us/op 0.45
mainnet_e58758 - phase0 processRewardsAndPenalties 52.130 ms/op 60.148 ms/op 0.87
mainnet_e58758 - phase0 processRegistryUpdates 10.155 us/op 14.460 us/op 0.70
mainnet_e58758 - phase0 processSlashings 633.00 ns/op 786.00 ns/op 0.81
mainnet_e58758 - phase0 processEth1DataReset 499.00 ns/op 676.00 ns/op 0.74
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 831.20 us/op 1.2370 ms/op 0.67
mainnet_e58758 - phase0 processSlashingsReset 1.0880 us/op 3.7970 us/op 0.29
mainnet_e58758 - phase0 processRandaoMixesReset 3.1330 us/op 8.8600 us/op 0.35
mainnet_e58758 - phase0 processHistoricalRootsUpdate 594.00 ns/op 700.00 ns/op 0.85
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.2490 us/op 4.1050 us/op 0.79
mainnet_e58758 - phase0 afterProcessEpoch 64.137 ms/op 96.744 ms/op 0.66
phase0 processEffectiveBalanceUpdates - 250000 normalcase 962.18 us/op 1.4392 ms/op 0.67
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.0752 ms/op 1.6247 ms/op 0.66
altair processInactivityUpdates - 250000 normalcase 26.161 ms/op 30.786 ms/op 0.85
altair processInactivityUpdates - 250000 worstcase 20.164 ms/op 27.200 ms/op 0.74
phase0 processRegistryUpdates - 250000 normalcase 7.7330 us/op 11.395 us/op 0.68
phase0 processRegistryUpdates - 250000 badcase_full_deposits 375.18 us/op 434.61 us/op 0.86
phase0 processRegistryUpdates - 250000 worstcase 0.5 113.83 ms/op 159.28 ms/op 0.71
altair processRewardsAndPenalties - 250000 normalcase 56.260 ms/op 52.897 ms/op 1.06
altair processRewardsAndPenalties - 250000 worstcase 57.114 ms/op 62.662 ms/op 0.91
phase0 getAttestationDeltas - 250000 normalcase 5.6678 ms/op 11.167 ms/op 0.51
phase0 getAttestationDeltas - 250000 worstcase 5.8167 ms/op 12.481 ms/op 0.47
phase0 processSlashings - 250000 worstcase 81.971 us/op 128.00 us/op 0.64
altair processSyncCommitteeUpdates - 250000 105.98 ms/op 173.97 ms/op 0.61
BeaconState.hashTreeRoot - No change 328.00 ns/op 736.00 ns/op 0.45
BeaconState.hashTreeRoot - 1 full validator 149.78 us/op 151.26 us/op 0.99
BeaconState.hashTreeRoot - 32 full validator 1.6434 ms/op 1.5728 ms/op 1.04
BeaconState.hashTreeRoot - 512 full validator 14.796 ms/op 22.116 ms/op 0.67
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 154.09 us/op 195.42 us/op 0.79
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.4202 ms/op 2.3697 ms/op 1.02
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 27.020 ms/op 33.906 ms/op 0.80
BeaconState.hashTreeRoot - 1 balances 132.49 us/op 128.58 us/op 1.03
BeaconState.hashTreeRoot - 32 balances 1.1788 ms/op 1.3087 ms/op 0.90
BeaconState.hashTreeRoot - 512 balances 11.006 ms/op 16.927 ms/op 0.65
BeaconState.hashTreeRoot - 250000 balances 191.19 ms/op 218.23 ms/op 0.88
aggregationBits - 2048 els - zipIndexesInBitList 10.223 us/op 21.281 us/op 0.48
byteArrayEquals 32 64.554 ns/op 77.793 ns/op 0.83
Buffer.compare 32 38.889 ns/op 58.565 ns/op 0.66
byteArrayEquals 1024 1.7601 us/op 2.1320 us/op 0.83
Buffer.compare 1024 53.959 ns/op 73.747 ns/op 0.73
byteArrayEquals 16384 28.044 us/op 33.912 us/op 0.83
Buffer.compare 16384 229.62 ns/op 285.68 ns/op 0.80
byteArrayEquals 123687377 208.85 ms/op 251.40 ms/op 0.83
Buffer.compare 123687377 3.8412 ms/op 6.2332 ms/op 0.62
byteArrayEquals 32 - diff last byte 60.995 ns/op 73.700 ns/op 0.83
Buffer.compare 32 - diff last byte 38.097 ns/op 56.404 ns/op 0.68
byteArrayEquals 1024 - diff last byte 1.7354 us/op 2.0849 us/op 0.83
Buffer.compare 1024 - diff last byte 44.661 ns/op 72.042 ns/op 0.62
byteArrayEquals 16384 - diff last byte 27.299 us/op 32.407 us/op 0.84
Buffer.compare 16384 - diff last byte 230.06 ns/op 276.79 ns/op 0.83
byteArrayEquals 123687377 - diff last byte 194.64 ms/op 249.76 ms/op 0.78
Buffer.compare 123687377 - diff last byte 4.0038 ms/op 7.9721 ms/op 0.50
byteArrayEquals 32 - random bytes 4.4400 ns/op 5.9150 ns/op 0.75
Buffer.compare 32 - random bytes 38.644 ns/op 62.913 ns/op 0.61
byteArrayEquals 1024 - random bytes 4.3980 ns/op 5.6050 ns/op 0.78
Buffer.compare 1024 - random bytes 36.811 ns/op 62.140 ns/op 0.59
byteArrayEquals 16384 - random bytes 4.4130 ns/op 5.4970 ns/op 0.80
Buffer.compare 16384 - random bytes 35.067 ns/op 62.345 ns/op 0.56
byteArrayEquals 123687377 - random bytes 7.5200 ns/op 9.4500 ns/op 0.80
Buffer.compare 123687377 - random bytes 40.090 ns/op 69.530 ns/op 0.58
regular array get 100000 times 40.561 us/op 46.253 us/op 0.88
wrappedArray get 100000 times 40.476 us/op 46.262 us/op 0.87
arrayWithProxy get 100000 times 9.7489 ms/op 14.345 ms/op 0.68
ssz.Root.equals 54.397 ns/op 55.168 ns/op 0.99
byteArrayEquals 52.648 ns/op 54.205 ns/op 0.97
Buffer.compare 9.7350 ns/op 12.276 ns/op 0.79
shuffle list - 16384 els 4.1846 ms/op 7.1604 ms/op 0.58
shuffle list - 250000 els 61.554 ms/op 105.01 ms/op 0.59
processSlot - 1 slots 16.754 us/op 17.471 us/op 0.96
processSlot - 32 slots 3.0808 ms/op 3.5126 ms/op 0.88
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 58.694 ms/op 60.058 ms/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 2.2223 ms/op 2.5045 ms/op 0.89
getCommitteeAssignments - req 100 vs - 250000 vc 3.3268 ms/op 3.7498 ms/op 0.89
getCommitteeAssignments - req 1000 vs - 250000 vc 3.5940 ms/op 4.0164 ms/op 0.89
findModifiedValidators - 10000 modified validators 418.53 ms/op 546.18 ms/op 0.77
findModifiedValidators - 1000 modified validators 322.51 ms/op 436.85 ms/op 0.74
findModifiedValidators - 100 modified validators 317.25 ms/op 404.82 ms/op 0.78
findModifiedValidators - 10 modified validators 275.99 ms/op 464.56 ms/op 0.59
findModifiedValidators - 1 modified validators 291.48 ms/op 430.69 ms/op 0.68
findModifiedValidators - no difference 282.35 ms/op 454.20 ms/op 0.62
compare ViewDUs 3.8511 s/op 4.7274 s/op 0.81
compare each validator Uint8Array 1.4306 s/op 1.8256 s/op 0.78
compare ViewDU to Uint8Array 788.34 ms/op 1.3148 s/op 0.60
migrate state 1000000 validators, 24 modified, 0 new 751.56 ms/op 827.61 ms/op 0.91
migrate state 1000000 validators, 1700 modified, 1000 new 1.0269 s/op 1.1480 s/op 0.89
migrate state 1000000 validators, 3400 modified, 2000 new 1.2398 s/op 1.3755 s/op 0.90
migrate state 1500000 validators, 24 modified, 0 new 759.70 ms/op 839.32 ms/op 0.91
migrate state 1500000 validators, 1700 modified, 1000 new 1.0215 s/op 1.1903 s/op 0.86
migrate state 1500000 validators, 3400 modified, 2000 new 1.1975 s/op 1.4908 s/op 0.80
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3200 ns/op 5.3500 ns/op 0.81
state getBlockRootAtSlot - 250000 vs - 7PWei 542.01 ns/op 689.17 ns/op 0.79
computeProposers - vc 250000 6.1983 ms/op 10.183 ms/op 0.61
computeEpochShuffling - vc 250000 67.480 ms/op 108.06 ms/op 0.62
getNextSyncCommittee - vc 250000 107.72 ms/op 174.60 ms/op 0.62
computeSigningRoot for AttestationData 24.790 us/op 30.940 us/op 0.80
hash AttestationData serialized data then Buffer.toString(base64) 1.2091 us/op 2.4678 us/op 0.49
toHexString serialized data 866.42 ns/op 1.4613 us/op 0.59
Buffer.toString(base64) 163.07 ns/op 285.94 ns/op 0.57

by benchmarkbot/action

@twoeths twoeths force-pushed the tuyen/get_attestations_for_block branch from 095e68e to 1d6887c Compare January 30, 2024 04:16
@codecov Codecov
Copy link

codecov bot commented Jan 30, 2024

Codecov Report

Merging #6367 (1d6887c) into unstable (ed43a98) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #6367   +/-   ##
=========================================
  Coverage     60.15%   60.15%           
=========================================
  Files           407      407           
  Lines         46511    46511           
  Branches       1550     1550           
=========================================
  Hits          27980    27980           
  Misses        18499    18499           
  Partials         32       32

@twoeths
Copy link
Contributor Author

twoeths commented Jan 30, 2024

bringing back finalizedSync.test.ts from #4759 to make sure vc publishing good block with qualified attestations and the chain can finalize, without it I'm not confident to get this work in considering our failed sim test.

Even when sim tests is green, I think it's still good to have this e2e test:

  • it's basic functionality to make sure validators publish good block and the chain can finalize
  • it's different to sim test where it contains lodestar nodes only
  • it's easier to debug when there is an issue
  • in the past we made the timeout so big (10 minutes) that's the the reason we moved to sim test, however now it takes only <1.5 minute

cc @nazarhussain @wemeetagain

@twoeths
Copy link
Contributor Author

twoeths commented Jan 30, 2024

holesky_produced_block_884610.json.zip

produced this block in holesky using curl http://localhost:9596/eth/v3/validator/blocks/884610?randao_reveal=0xa84817d056d083544cb187e6fedf8865dda4a6a477b32b1e2ee7c3caaa67f2a141a92121cc75ee4d93efac5a2d20d76a137168115dbc01d48f5fc97355f9c541d09016c5ff80a27d9e073493a2a604c044c78722b852843c1a286f8d7925dee2 -o block.json

it looks good with 128 attestations, most of them are for previous block slot

@twoeths twoeths marked this pull request as ready for review January 30, 2024 06:49
@twoeths twoeths requested a review from a team as a code owner January 30, 2024 06:49
g11tech
g11tech approved these changes Feb 1, 2024
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure how significant the boost will be if from the starting itself we only insert and maintain in a sorted array by score each time we evaluate and add attestation by score (and we can easily get the score cutoff by inspecting MAX_ATTESTIONS's index in the array for each iteration, so probably we can cut off more attestations with lower score in average case

Overall compute time will not change because there is sorting still involved later which would not need to be done as the array is already sorted by score.

However the current approach already optimizes it quite a bit (1sec => 100ms), so may be it would be worth to look at the above approach only if there we are looking to extract further juice.

For now looks all good 👍

@wemeetagain wemeetagain merged commit 863ee5f into unstable Feb 1, 2024
14 of 15 checks passed
@wemeetagain wemeetagain deleted the tuyen/get_attestations_for_block branch February 1, 2024 19:51
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.16.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

@g11tech g11tech g11tech approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Performance issue getting aggregated attestations for block productions
3 participants
@twoeths @wemeetagain @g11tech