Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: sanitize URL to prevent leaking user credentials in logs #6175

Merged
merged 4 commits into from
Dec 12, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Dec 9, 2023

Motivation

Recently added printing out URLs of connected services (#6099). Those might contain user credentials, e.g. if validator is connected to rescue node. We should make sure to sanitize URLs before printing those in logs to prevent potentially leaking those credentials.

Description

Sanitize URL to prevent leaking user credentials in logs

@nflaig nflaig requested a review from a team as a code owner December 9, 2023 12:33
Copy link

codecov bot commented Dec 9, 2023

Codecov Report

Merging #6175 (71fef20) into unstable (1a63b07) will increase coverage by 10.33%.
Report is 1 commits behind head on unstable.
The diff coverage is n/a.

❗ Current head 71fef20 differs from pull request most recent head d64cdca. Consider uploading reports for the commit d64cdca to get more accurate results

Additional details and impacted files
@@              Coverage Diff              @@
##           unstable    #6175       +/-   ##
=============================================
+ Coverage     80.02%   90.35%   +10.33%     
=============================================
  Files            19       78       +59     
  Lines          1717     8089     +6372     
  Branches        155      490      +335     
=============================================
+ Hits           1374     7309     +5935     
- Misses          341      772      +431     
- Partials          2        8        +6     

jeluard
jeluard previously approved these changes Dec 9, 2023
* Sanitize URL to prevent leaking user credentials in logs
*/
export function toSafePrintableUrl(urlStr: string): string {
return new URL(urlStr).origin;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will strip out searchpart of URL, assuming this is fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is desired in case someone provides secrets in query params. I don't know of anybody that does this but we can't prevent users from doing that and passing credentials directly as part of the URL is common because it works with all clients.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, somehow I assumed credentials could be passed via URL username/password properties

Copy link
Member Author

@nflaig nflaig Dec 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of rescue node, they are passing the user credentials in the userinfo part of the URL. We move those into the Authorization header (#5884) before the request is sent but at the time it's logged at startup, the URL still contains credentials.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that calling URL with an invalid URL with throw an error. Is urlStr sure to be a valid URL in the normal Beacon Node init flow? Else users may get funny errors with the wrong URL.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Completely forgot about that, I added a note to the function now and moved all URL logs after they are already validated by http / jsonrpc client.

Also looked into sanitizing invalid URLs (in catch block) but that got too messy.

Copy link
Contributor

github-actions bot commented Dec 9, 2023

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: 72b1fd7 Previous: 959a8af Ratio
getSlashingsAndExits - default max 87.621 us/op 443.58 us/op 0.20
Object set x1000 24.971 ns/op 80.456 ns/op 0.31
Map set x1000 17.024 ns/op 60.552 ns/op 0.28
Full benchmark results
Benchmark suite Current: 72b1fd7 Previous: 959a8af Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 383.38 us/op 911.68 us/op 0.42
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 50.658 us/op 143.52 us/op 0.35
BLS verify - blst-native 1.0985 ms/op 1.4383 ms/op 0.76
BLS verifyMultipleSignatures 3 - blst-native 2.3457 ms/op 2.9960 ms/op 0.78
BLS verifyMultipleSignatures 8 - blst-native 5.1935 ms/op 6.9110 ms/op 0.75
BLS verifyMultipleSignatures 32 - blst-native 19.085 ms/op 25.059 ms/op 0.76
BLS verifyMultipleSignatures 64 - blst-native 37.602 ms/op 46.851 ms/op 0.80
BLS verifyMultipleSignatures 128 - blst-native 74.670 ms/op 96.479 ms/op 0.77
BLS deserializing 10000 signatures 797.80 ms/op 974.85 ms/op 0.82
BLS deserializing 100000 signatures 8.2301 s/op 9.4597 s/op 0.87
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1682 ms/op 1.4177 ms/op 0.82
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2847 ms/op 1.6431 ms/op 0.78
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.5963 ms/op 2.4896 ms/op 1.04
BLS verifyMultipleSignatures - same message - 64 - blst-native 2.9532 ms/op 3.8370 ms/op 0.77
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.5427 ms/op 6.5170 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 23.407 us/op 29.000 us/op 0.81
BLS aggregatePubkeys 128 - blst-native 86.861 us/op 105.46 us/op 0.82
getAttestationsForBlock 38.028 ms/op 70.130 ms/op 0.54
getSlashingsAndExits - default max 87.621 us/op 443.58 us/op 0.20
getSlashingsAndExits - 2k 384.01 us/op 686.14 us/op 0.56
proposeBlockBody type=full, size=empty 4.7434 ms/op 7.0502 ms/op 0.67
isKnown best case - 1 super set check 359.00 ns/op 708.00 ns/op 0.51
isKnown normal case - 2 super set checks 342.00 ns/op 769.00 ns/op 0.44
isKnown worse case - 16 super set checks 342.00 ns/op 768.00 ns/op 0.45
CheckpointStateCache - add get delete 3.8770 us/op 6.9440 us/op 0.56
validate api signedAggregateAndProof - struct 2.4369 ms/op 3.2607 ms/op 0.75
validate gossip signedAggregateAndProof - struct 2.4212 ms/op 3.1945 ms/op 0.76
validate gossip attestation - vc 640000 1.1676 ms/op 1.4856 ms/op 0.79
batch validate gossip attestation - vc 640000 - chunk 32 140.59 us/op 190.39 us/op 0.74
batch validate gossip attestation - vc 640000 - chunk 64 126.24 us/op 169.64 us/op 0.74
batch validate gossip attestation - vc 640000 - chunk 128 117.84 us/op 167.40 us/op 0.70
batch validate gossip attestation - vc 640000 - chunk 256 117.71 us/op 148.47 us/op 0.79
pickEth1Vote - no votes 898.57 us/op 1.4465 ms/op 0.62
pickEth1Vote - max votes 6.0716 ms/op 13.807 ms/op 0.44
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.424 ms/op 22.297 ms/op 0.51
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.890 ms/op 32.967 ms/op 0.57
pickEth1Vote - Eth1Data fastSerialize value x2048 436.01 us/op 784.87 us/op 0.56
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.1131 ms/op 6.1774 ms/op 1.48
bytes32 toHexString 771.00 ns/op 882.00 ns/op 0.87
bytes32 Buffer.toString(hex) 374.00 ns/op 311.00 ns/op 1.20
bytes32 Buffer.toString(hex) from Uint8Array 647.00 ns/op 615.00 ns/op 1.05
bytes32 Buffer.toString(hex) + 0x 416.00 ns/op 315.00 ns/op 1.32
Object access 1 prop 0.23500 ns/op 0.25800 ns/op 0.91
Map access 1 prop 0.18800 ns/op 0.16400 ns/op 1.15
Object get x1000 5.6880 ns/op 10.074 ns/op 0.56
Map get x1000 0.79600 ns/op 1.0590 ns/op 0.75
Object set x1000 24.971 ns/op 80.456 ns/op 0.31
Map set x1000 17.024 ns/op 60.552 ns/op 0.28
Return object 10000 times 0.23250 ns/op 0.28160 ns/op 0.83
Throw Error 10000 times 3.1695 us/op 4.2248 us/op 0.75
fastMsgIdFn sha256 / 200 bytes 2.1260 us/op 3.5570 us/op 0.60
fastMsgIdFn h32 xxhash / 200 bytes 303.00 ns/op 375.00 ns/op 0.81
fastMsgIdFn h64 xxhash / 200 bytes 369.00 ns/op 408.00 ns/op 0.90
fastMsgIdFn sha256 / 1000 bytes 6.5540 us/op 12.030 us/op 0.54
fastMsgIdFn h32 xxhash / 1000 bytes 464.00 ns/op 520.00 ns/op 0.89
fastMsgIdFn h64 xxhash / 1000 bytes 446.00 ns/op 526.00 ns/op 0.85
fastMsgIdFn sha256 / 10000 bytes 53.553 us/op 113.42 us/op 0.47
fastMsgIdFn h32 xxhash / 10000 bytes 2.0430 us/op 2.3660 us/op 0.86
fastMsgIdFn h64 xxhash / 10000 bytes 1.3350 us/op 1.6670 us/op 0.80
send data - 1000 256B messages 12.014 ms/op 26.837 ms/op 0.45
send data - 1000 512B messages 17.093 ms/op 30.437 ms/op 0.56
send data - 1000 1024B messages 25.410 ms/op 46.990 ms/op 0.54
send data - 1000 1200B messages 31.113 ms/op 57.095 ms/op 0.54
send data - 1000 2048B messages 39.564 ms/op 60.554 ms/op 0.65
send data - 1000 4096B messages 36.300 ms/op 51.938 ms/op 0.70
send data - 1000 16384B messages 94.813 ms/op 120.91 ms/op 0.78
send data - 1000 65536B messages 452.19 ms/op 491.40 ms/op 0.92
enrSubnets - fastDeserialize 64 bits 1.2740 us/op 1.4350 us/op 0.89
enrSubnets - ssz BitVector 64 bits 493.00 ns/op 483.00 ns/op 1.02
enrSubnets - fastDeserialize 4 bits 216.00 ns/op 209.00 ns/op 1.03
enrSubnets - ssz BitVector 4 bits 496.00 ns/op 592.00 ns/op 0.84
prioritizePeers score -10:0 att 32-0.1 sync 2-0 68.779 us/op 112.96 us/op 0.61
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 87.441 us/op 145.84 us/op 0.60
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 127.43 us/op 196.42 us/op 0.65
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 210.35 us/op 333.78 us/op 0.63
prioritizePeers score 0:0 att 64-1 sync 4-1 246.35 us/op 356.96 us/op 0.69
array of 16000 items push then shift 1.3817 us/op 1.7004 us/op 0.81
LinkedList of 16000 items push then shift 6.6820 ns/op 9.1810 ns/op 0.73
array of 16000 items push then pop 110.04 ns/op 105.65 ns/op 1.04
LinkedList of 16000 items push then pop 6.8030 ns/op 9.8020 ns/op 0.69
array of 24000 items push then shift 2.0870 us/op 2.5838 us/op 0.81
LinkedList of 24000 items push then shift 7.4360 ns/op 9.4740 ns/op 0.78
array of 24000 items push then pop 132.31 ns/op 145.46 ns/op 0.91
LinkedList of 24000 items push then pop 5.9010 ns/op 9.1240 ns/op 0.65
intersect bitArray bitLen 8 5.2230 ns/op 6.7060 ns/op 0.78
intersect array and set length 8 50.009 ns/op 70.184 ns/op 0.71
intersect bitArray bitLen 128 28.524 ns/op 35.055 ns/op 0.81
intersect array and set length 128 723.72 ns/op 989.22 ns/op 0.73
bitArray.getTrueBitIndexes() bitLen 128 1.2680 us/op 1.6370 us/op 0.77
bitArray.getTrueBitIndexes() bitLen 248 2.3570 us/op 2.6430 us/op 0.89
bitArray.getTrueBitIndexes() bitLen 512 4.8130 us/op 5.9570 us/op 0.81
Buffer.concat 32 items 880.00 ns/op 1.2460 us/op 0.71
Uint8Array.set 32 items 1.7600 us/op 2.1320 us/op 0.83
Set add up to 64 items then delete first 1.7880 us/op 5.0827 us/op 0.35
OrderedSet add up to 64 items then delete first 2.7150 us/op 5.9899 us/op 0.45
Set add up to 64 items then delete last 2.0123 us/op 5.0841 us/op 0.40
OrderedSet add up to 64 items then delete last 3.0445 us/op 6.0187 us/op 0.51
Set add up to 64 items then delete middle 2.0269 us/op 4.8251 us/op 0.42
OrderedSet add up to 64 items then delete middle 4.5070 us/op 7.1754 us/op 0.63
Set add up to 128 items then delete first 4.0082 us/op 9.4938 us/op 0.42
OrderedSet add up to 128 items then delete first 6.2730 us/op 12.918 us/op 0.49
Set add up to 128 items then delete last 3.8786 us/op 9.4159 us/op 0.41
OrderedSet add up to 128 items then delete last 6.1367 us/op 11.833 us/op 0.52
Set add up to 128 items then delete middle 3.8601 us/op 10.407 us/op 0.37
OrderedSet add up to 128 items then delete middle 11.210 us/op 20.301 us/op 0.55
Set add up to 256 items then delete first 7.9067 us/op 21.594 us/op 0.37
OrderedSet add up to 256 items then delete first 12.754 us/op 30.389 us/op 0.42
Set add up to 256 items then delete last 7.8266 us/op 22.062 us/op 0.35
OrderedSet add up to 256 items then delete last 12.655 us/op 29.063 us/op 0.44
Set add up to 256 items then delete middle 8.5222 us/op 20.098 us/op 0.42
OrderedSet add up to 256 items then delete middle 39.195 us/op 49.767 us/op 0.79
transfer serialized Status (84 B) 1.6370 us/op 1.9810 us/op 0.83
copy serialized Status (84 B) 1.6150 us/op 1.6270 us/op 0.99
transfer serialized SignedVoluntaryExit (112 B) 1.5910 us/op 1.8610 us/op 0.85
copy serialized SignedVoluntaryExit (112 B) 1.7540 us/op 1.6450 us/op 1.07
transfer serialized ProposerSlashing (416 B) 2.1380 us/op 2.2020 us/op 0.97
copy serialized ProposerSlashing (416 B) 2.3510 us/op 2.1100 us/op 1.11
transfer serialized Attestation (485 B) 3.1570 us/op 2.5920 us/op 1.22
copy serialized Attestation (485 B) 3.4210 us/op 2.7260 us/op 1.25
transfer serialized AttesterSlashing (33232 B) 3.0660 us/op 3.0050 us/op 1.02
copy serialized AttesterSlashing (33232 B) 5.1150 us/op 6.4330 us/op 0.80
transfer serialized Small SignedBeaconBlock (128000 B) 2.0240 us/op 2.7790 us/op 0.73
copy serialized Small SignedBeaconBlock (128000 B) 10.225 us/op 16.316 us/op 0.63
transfer serialized Avg SignedBeaconBlock (200000 B) 2.0750 us/op 3.1660 us/op 0.66
copy serialized Avg SignedBeaconBlock (200000 B) 14.375 us/op 21.072 us/op 0.68
transfer serialized BlobsSidecar (524380 B) 2.8400 us/op 3.4510 us/op 0.82
copy serialized BlobsSidecar (524380 B) 147.79 us/op 132.64 us/op 1.11
transfer serialized Big SignedBeaconBlock (1000000 B) 2.7100 us/op 3.9930 us/op 0.68
copy serialized Big SignedBeaconBlock (1000000 B) 145.42 us/op 308.69 us/op 0.47
pass gossip attestations to forkchoice per slot 2.9063 ms/op 4.4865 ms/op 0.65
forkChoice updateHead vc 100000 bc 64 eq 0 513.90 us/op 711.90 us/op 0.72
forkChoice updateHead vc 600000 bc 64 eq 0 2.6916 ms/op 4.3284 ms/op 0.62
forkChoice updateHead vc 1000000 bc 64 eq 0 4.5738 ms/op 7.0640 ms/op 0.65
forkChoice updateHead vc 600000 bc 320 eq 0 2.6692 ms/op 4.2164 ms/op 0.63
forkChoice updateHead vc 600000 bc 1200 eq 0 2.8170 ms/op 4.3704 ms/op 0.64
forkChoice updateHead vc 600000 bc 7200 eq 0 3.5607 ms/op 5.3400 ms/op 0.67
forkChoice updateHead vc 600000 bc 64 eq 1000 10.261 ms/op 11.408 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 10000 10.519 ms/op 12.199 ms/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 300000 12.265 ms/op 20.223 ms/op 0.61
computeDeltas 500000 validators 300 proto nodes 3.8116 ms/op 7.0513 ms/op 0.54
computeDeltas 500000 validators 1200 proto nodes 3.4371 ms/op 6.6754 ms/op 0.51
computeDeltas 500000 validators 7200 proto nodes 3.3778 ms/op 6.5907 ms/op 0.51
computeDeltas 750000 validators 300 proto nodes 5.1856 ms/op 9.7794 ms/op 0.53
computeDeltas 750000 validators 1200 proto nodes 4.9967 ms/op 9.5770 ms/op 0.52
computeDeltas 750000 validators 7200 proto nodes 4.7711 ms/op 9.4889 ms/op 0.50
computeDeltas 1400000 validators 300 proto nodes 8.9787 ms/op 19.036 ms/op 0.47
computeDeltas 1400000 validators 1200 proto nodes 8.8553 ms/op 19.429 ms/op 0.46
computeDeltas 1400000 validators 7200 proto nodes 8.7106 ms/op 19.117 ms/op 0.46
computeDeltas 2100000 validators 300 proto nodes 13.234 ms/op 28.997 ms/op 0.46
computeDeltas 2100000 validators 1200 proto nodes 13.316 ms/op 29.171 ms/op 0.46
computeDeltas 2100000 validators 7200 proto nodes 13.931 ms/op 29.902 ms/op 0.47
computeProposerBoostScoreFromBalances 500000 validators 3.3031 ms/op 3.8310 ms/op 0.86
computeProposerBoostScoreFromBalances 750000 validators 3.3150 ms/op 3.8424 ms/op 0.86
computeProposerBoostScoreFromBalances 1400000 validators 3.2448 ms/op 3.8969 ms/op 0.83
computeProposerBoostScoreFromBalances 2100000 validators 3.3002 ms/op 4.0735 ms/op 0.81
altair processAttestation - 250000 vs - 7PWei normalcase 2.4417 ms/op 2.5444 ms/op 0.96
altair processAttestation - 250000 vs - 7PWei worstcase 2.3632 ms/op 4.2966 ms/op 0.55
altair processAttestation - setStatus - 1/6 committees join 74.110 us/op 203.62 us/op 0.36
altair processAttestation - setStatus - 1/3 committees join 173.71 us/op 378.28 us/op 0.46
altair processAttestation - setStatus - 1/2 committees join 222.17 us/op 510.80 us/op 0.43
altair processAttestation - setStatus - 2/3 committees join 271.67 us/op 646.91 us/op 0.42
altair processAttestation - setStatus - 4/5 committees join 394.00 us/op 853.04 us/op 0.46
altair processAttestation - setStatus - 100% committees join 482.55 us/op 999.95 us/op 0.48
altair processBlock - 250000 vs - 7PWei normalcase 7.6817 ms/op 11.817 ms/op 0.65
altair processBlock - 250000 vs - 7PWei normalcase hashState 31.081 ms/op 41.779 ms/op 0.74
altair processBlock - 250000 vs - 7PWei worstcase 30.227 ms/op 40.790 ms/op 0.74
altair processBlock - 250000 vs - 7PWei worstcase hashState 94.714 ms/op 109.81 ms/op 0.86
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1342 ms/op 3.1684 ms/op 0.67
phase0 processBlock - 250000 vs - 7PWei worstcase 27.586 ms/op 33.832 ms/op 0.82
altair processEth1Data - 250000 vs - 7PWei normalcase 425.15 us/op 791.81 us/op 0.54
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 10.313 us/op 11.257 us/op 0.92
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 45.731 us/op 99.270 us/op 0.46
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 23.259 us/op 32.196 us/op 0.72
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.7550 us/op 19.570 us/op 0.35
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 188.28 us/op 187.58 us/op 1.00
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0281 ms/op 1.3045 ms/op 0.79
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.2384 ms/op 2.0607 ms/op 0.60
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.2439 ms/op 2.1386 ms/op 0.58
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.4562 ms/op 4.4531 ms/op 0.55
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.9966 ms/op 2.8024 ms/op 0.71
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.7267 ms/op 6.8683 ms/op 0.54
Tree 40 250000 create 265.21 ms/op 420.33 ms/op 0.63
Tree 40 250000 get(125000) 113.68 ns/op 213.91 ns/op 0.53
Tree 40 250000 set(125000) 816.40 ns/op 1.0496 us/op 0.78
Tree 40 250000 toArray() 21.577 ms/op 23.169 ms/op 0.93
Tree 40 250000 iterate all - toArray() + loop 21.529 ms/op 24.215 ms/op 0.89
Tree 40 250000 iterate all - get(i) 51.231 ms/op 77.775 ms/op 0.66
MutableVector 250000 create 12.777 ms/op 17.677 ms/op 0.72
MutableVector 250000 get(125000) 5.7400 ns/op 6.7180 ns/op 0.85
MutableVector 250000 set(125000) 204.90 ns/op 308.13 ns/op 0.66
MutableVector 250000 toArray() 2.4011 ms/op 5.6349 ms/op 0.43
MutableVector 250000 iterate all - toArray() + loop 2.5166 ms/op 4.5221 ms/op 0.56
MutableVector 250000 iterate all - get(i) 1.3524 ms/op 1.5794 ms/op 0.86
Array 250000 create 2.3085 ms/op 3.3311 ms/op 0.69
Array 250000 clone - spread 1.1798 ms/op 1.2541 ms/op 0.94
Array 250000 get(125000) 1.0640 ns/op 1.0820 ns/op 0.98
Array 250000 set(125000) 1.2510 ns/op 4.2110 ns/op 0.30
Array 250000 iterate all - loop 152.17 us/op 170.24 us/op 0.89
effectiveBalanceIncrements clone Uint8Array 300000 13.606 us/op 33.802 us/op 0.40
effectiveBalanceIncrements clone MutableVector 300000 422.00 ns/op 375.00 ns/op 1.13
effectiveBalanceIncrements rw all Uint8Array 300000 182.51 us/op 212.86 us/op 0.86
effectiveBalanceIncrements rw all MutableVector 300000 66.658 ms/op 97.650 ms/op 0.68
phase0 afterProcessEpoch - 250000 vs - 7PWei 82.513 ms/op 120.04 ms/op 0.69
phase0 beforeProcessEpoch - 250000 vs - 7PWei 35.657 ms/op 60.270 ms/op 0.59
altair processEpoch - mainnet_e81889 456.11 ms/op 473.92 ms/op 0.96
mainnet_e81889 - altair beforeProcessEpoch 85.278 ms/op 92.684 ms/op 0.92
mainnet_e81889 - altair processJustificationAndFinalization 21.252 us/op 19.786 us/op 1.07
mainnet_e81889 - altair processInactivityUpdates 5.3366 ms/op 7.3070 ms/op 0.73
mainnet_e81889 - altair processRewardsAndPenalties 66.652 ms/op 82.105 ms/op 0.81
mainnet_e81889 - altair processRegistryUpdates 3.4810 us/op 3.0280 us/op 1.15
mainnet_e81889 - altair processSlashings 895.00 ns/op 877.00 ns/op 1.02
mainnet_e81889 - altair processEth1DataReset 967.00 ns/op 1.4020 us/op 0.69
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.1223 ms/op 2.5082 ms/op 0.45
mainnet_e81889 - altair processSlashingsReset 5.4800 us/op 7.9100 us/op 0.69
mainnet_e81889 - altair processRandaoMixesReset 6.3490 us/op 12.058 us/op 0.53
mainnet_e81889 - altair processHistoricalRootsUpdate 1.5320 us/op 1.0910 us/op 1.40
mainnet_e81889 - altair processParticipationFlagUpdates 3.4600 us/op 3.6550 us/op 0.95
mainnet_e81889 - altair processSyncCommitteeUpdates 1.3080 us/op 1.0870 us/op 1.20
mainnet_e81889 - altair afterProcessEpoch 86.746 ms/op 128.59 ms/op 0.67
capella processEpoch - mainnet_e217614 1.8595 s/op 2.4033 s/op 0.77
mainnet_e217614 - capella beforeProcessEpoch 414.80 ms/op 522.97 ms/op 0.79
mainnet_e217614 - capella processJustificationAndFinalization 14.123 us/op 19.899 us/op 0.71
mainnet_e217614 - capella processInactivityUpdates 17.281 ms/op 25.090 ms/op 0.69
mainnet_e217614 - capella processRewardsAndPenalties 346.35 ms/op 468.37 ms/op 0.74
mainnet_e217614 - capella processRegistryUpdates 17.977 us/op 30.148 us/op 0.60
mainnet_e217614 - capella processSlashings 573.00 ns/op 1.0210 us/op 0.56
mainnet_e217614 - capella processEth1DataReset 501.00 ns/op 694.00 ns/op 0.72
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.9937 ms/op 7.4805 ms/op 0.53
mainnet_e217614 - capella processSlashingsReset 2.6600 us/op 6.7600 us/op 0.39
mainnet_e217614 - capella processRandaoMixesReset 3.3430 us/op 8.7710 us/op 0.38
mainnet_e217614 - capella processHistoricalRootsUpdate 551.00 ns/op 877.00 ns/op 0.63
mainnet_e217614 - capella processParticipationFlagUpdates 1.6840 us/op 4.3210 us/op 0.39
mainnet_e217614 - capella afterProcessEpoch 199.82 ms/op 333.26 ms/op 0.60
phase0 processEpoch - mainnet_e58758 357.80 ms/op 557.77 ms/op 0.64
mainnet_e58758 - phase0 beforeProcessEpoch 121.73 ms/op 152.40 ms/op 0.80
mainnet_e58758 - phase0 processJustificationAndFinalization 12.650 us/op 20.057 us/op 0.63
mainnet_e58758 - phase0 processRewardsAndPenalties 41.069 ms/op 63.551 ms/op 0.65
mainnet_e58758 - phase0 processRegistryUpdates 9.7450 us/op 14.985 us/op 0.65
mainnet_e58758 - phase0 processSlashings 679.00 ns/op 1.1170 us/op 0.61
mainnet_e58758 - phase0 processEth1DataReset 627.00 ns/op 455.00 ns/op 1.38
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0992 ms/op 1.2157 ms/op 0.90
mainnet_e58758 - phase0 processSlashingsReset 2.4130 us/op 3.2290 us/op 0.75
mainnet_e58758 - phase0 processRandaoMixesReset 3.4270 us/op 5.1990 us/op 0.66
mainnet_e58758 - phase0 processHistoricalRootsUpdate 533.00 ns/op 475.00 ns/op 1.12
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.2920 us/op 12.744 us/op 0.26
mainnet_e58758 - phase0 afterProcessEpoch 64.599 ms/op 107.58 ms/op 0.60
phase0 processEffectiveBalanceUpdates - 250000 normalcase 971.86 us/op 1.5921 ms/op 0.61
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.1008 ms/op 1.5340 ms/op 0.72
altair processInactivityUpdates - 250000 normalcase 25.197 ms/op 25.137 ms/op 1.00
altair processInactivityUpdates - 250000 worstcase 24.599 ms/op 24.518 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 6.7090 us/op 14.164 us/op 0.47
phase0 processRegistryUpdates - 250000 badcase_full_deposits 302.71 us/op 403.33 us/op 0.75
phase0 processRegistryUpdates - 250000 worstcase 0.5 95.813 ms/op 154.05 ms/op 0.62
altair processRewardsAndPenalties - 250000 normalcase 47.953 ms/op 63.865 ms/op 0.75
altair processRewardsAndPenalties - 250000 worstcase 49.429 ms/op 61.609 ms/op 0.80
phase0 getAttestationDeltas - 250000 normalcase 5.7898 ms/op 12.050 ms/op 0.48
phase0 getAttestationDeltas - 250000 worstcase 5.2654 ms/op 13.109 ms/op 0.40
phase0 processSlashings - 250000 worstcase 82.994 us/op 130.59 us/op 0.64
altair processSyncCommitteeUpdates - 250000 103.39 ms/op 181.12 ms/op 0.57
BeaconState.hashTreeRoot - No change 307.00 ns/op 300.00 ns/op 1.02
BeaconState.hashTreeRoot - 1 full validator 159.68 us/op 161.70 us/op 0.99
BeaconState.hashTreeRoot - 32 full validator 1.4013 ms/op 1.6978 ms/op 0.83
BeaconState.hashTreeRoot - 512 full validator 13.377 ms/op 15.561 ms/op 0.86
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 166.54 us/op 177.36 us/op 0.94
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.0943 ms/op 2.8349 ms/op 0.74
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 25.287 ms/op 33.988 ms/op 0.74
BeaconState.hashTreeRoot - 1 balances 132.04 us/op 156.09 us/op 0.85
BeaconState.hashTreeRoot - 32 balances 1.2379 ms/op 1.4767 ms/op 0.84
BeaconState.hashTreeRoot - 512 balances 11.141 ms/op 14.718 ms/op 0.76
BeaconState.hashTreeRoot - 250000 balances 201.65 ms/op 235.47 ms/op 0.86
aggregationBits - 2048 els - zipIndexesInBitList 14.685 us/op 31.765 us/op 0.46
byteArrayEquals 32 63.194 ns/op 80.944 ns/op 0.78
Buffer.compare 32 36.155 ns/op 58.475 ns/op 0.62
byteArrayEquals 1024 1.6990 us/op 2.1796 us/op 0.78
Buffer.compare 1024 43.499 ns/op 81.957 ns/op 0.53
byteArrayEquals 16384 26.765 us/op 35.250 us/op 0.76
Buffer.compare 16384 190.29 ns/op 296.20 ns/op 0.64
byteArrayEquals 123687377 214.48 ms/op 272.76 ms/op 0.79
Buffer.compare 123687377 5.9775 ms/op 11.968 ms/op 0.50
byteArrayEquals 32 - diff last byte 61.963 ns/op 91.374 ns/op 0.68
Buffer.compare 32 - diff last byte 38.826 ns/op 61.239 ns/op 0.63
byteArrayEquals 1024 - diff last byte 1.7743 us/op 2.1731 us/op 0.82
Buffer.compare 1024 - diff last byte 48.387 ns/op 82.834 ns/op 0.58
byteArrayEquals 16384 - diff last byte 27.596 us/op 33.839 us/op 0.82
Buffer.compare 16384 - diff last byte 224.62 ns/op 316.11 ns/op 0.71
byteArrayEquals 123687377 - diff last byte 211.43 ms/op 269.74 ms/op 0.78
Buffer.compare 123687377 - diff last byte 5.6607 ms/op 11.755 ms/op 0.48
byteArrayEquals 32 - random bytes 4.6920 ns/op 8.7450 ns/op 0.54
Buffer.compare 32 - random bytes 38.748 ns/op 70.871 ns/op 0.55
byteArrayEquals 1024 - random bytes 5.0550 ns/op 7.1690 ns/op 0.71
Buffer.compare 1024 - random bytes 37.476 ns/op 68.386 ns/op 0.55
byteArrayEquals 16384 - random bytes 5.0110 ns/op 7.3550 ns/op 0.68
Buffer.compare 16384 - random bytes 35.522 ns/op 67.346 ns/op 0.53
byteArrayEquals 123687377 - random bytes 8.0600 ns/op 22.840 ns/op 0.35
Buffer.compare 123687377 - random bytes 42.810 ns/op 81.400 ns/op 0.53
regular array get 100000 times 42.164 us/op 55.798 us/op 0.76
wrappedArray get 100000 times 41.510 us/op 47.781 us/op 0.87
arrayWithProxy get 100000 times 10.543 ms/op 15.966 ms/op 0.66
ssz.Root.equals 56.992 ns/op 57.645 ns/op 0.99
byteArrayEquals 54.712 ns/op 57.018 ns/op 0.96
Buffer.compare 9.9890 ns/op 13.342 ns/op 0.75
shuffle list - 16384 els 4.5756 ms/op 7.5900 ms/op 0.60
shuffle list - 250000 els 67.694 ms/op 117.16 ms/op 0.58
processSlot - 1 slots 15.252 us/op 20.849 us/op 0.73
processSlot - 32 slots 2.6004 ms/op 4.3583 ms/op 0.60
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 50.495 ms/op 81.681 ms/op 0.62
getCommitteeAssignments - req 1 vs - 250000 vc 2.2461 ms/op 3.0244 ms/op 0.74
getCommitteeAssignments - req 100 vs - 250000 vc 3.3547 ms/op 4.4168 ms/op 0.76
getCommitteeAssignments - req 1000 vs - 250000 vc 3.6700 ms/op 5.1320 ms/op 0.72
findModifiedValidators - 10000 modified validators 409.65 ms/op 801.55 ms/op 0.51
findModifiedValidators - 1000 modified validators 331.78 ms/op 624.77 ms/op 0.53
findModifiedValidators - 100 modified validators 311.79 ms/op 656.57 ms/op 0.47
findModifiedValidators - 10 modified validators 306.11 ms/op 602.26 ms/op 0.51
findModifiedValidators - 1 modified validators 302.05 ms/op 608.15 ms/op 0.50
findModifiedValidators - no difference 322.22 ms/op 550.35 ms/op 0.59
compare ViewDUs 3.9698 s/op 5.2614 s/op 0.75
compare each validator Uint8Array 1.2896 s/op 2.1780 s/op 0.59
compare ViewDU to Uint8Array 827.49 ms/op 1.4644 s/op 0.57
migrate state 1000000 validators, 24 modified, 0 new 552.77 ms/op 858.28 ms/op 0.64
migrate state 1000000 validators, 1700 modified, 1000 new 866.32 ms/op 1.4165 s/op 0.61
migrate state 1000000 validators, 3400 modified, 2000 new 1.0401 s/op 1.8393 s/op 0.57
migrate state 1500000 validators, 24 modified, 0 new 670.00 ms/op 1.3266 s/op 0.51
migrate state 1500000 validators, 1700 modified, 1000 new 867.98 ms/op 1.5583 s/op 0.56
migrate state 1500000 validators, 3400 modified, 2000 new 1.0850 s/op 1.7538 s/op 0.62
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6100 ns/op 6.2300 ns/op 0.74
state getBlockRootAtSlot - 250000 vs - 7PWei 541.16 ns/op 869.22 ns/op 0.62
computeProposers - vc 250000 7.8069 ms/op 11.451 ms/op 0.68
computeEpochShuffling - vc 250000 68.430 ms/op 122.61 ms/op 0.56
getNextSyncCommittee - vc 250000 126.57 ms/op 200.71 ms/op 0.63
computeSigningRoot for AttestationData 24.317 us/op 40.465 us/op 0.60
hash AttestationData serialized data then Buffer.toString(base64) 1.2404 us/op 2.7089 us/op 0.46
toHexString serialized data 837.57 ns/op 2.0970 us/op 0.40
Buffer.toString(base64) 151.12 ns/op 321.92 ns/op 0.47

by benchmarkbot/action

@nflaig nflaig marked this pull request as draft December 11, 2023 17:37
packages/utils/src/url.ts Fixed Show fixed Hide fixed
@@ -59,6 +58,7 @@ export class ExecutionBuilderHttp implements IExecutionBuilder {
},
{config, metrics: metrics?.builderHttpClient}
);
logger?.info("External builder", {url: toSafePrintableUrl(baseUrl)});
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builder is a bit of an outlier right now, we allow to set multiple URLs via --builder.urls but only use the first URL (opts.urls[0]). I opted to only log the URL which is actually used.

@g11tech Is it intentional that we don't allow a fallback for the builder? I guess mev-boost is already doing the multiplexing and builder flow is stateful which makes fallback logic a bit problematic.

Maybe we want to reflect that in the CLI as well as currently it looks like it supports multiple urls

image

@nflaig nflaig marked this pull request as ready for review December 11, 2023 21:20
@nflaig nflaig requested a review from dapplion December 11, 2023 21:21
@philknows philknows added this to the v1.13.0 milestone Dec 12, 2023
@wemeetagain wemeetagain merged commit d7357fd into unstable Dec 12, 2023
15 checks passed
@wemeetagain wemeetagain deleted the nflaig/sanitize-url-before-logging branch December 12, 2023 16:15
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.13.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants