Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spurious no bid for this getPayload payload found #636

Open
michaelsproul opened this issue Mar 20, 2024 · 5 comments
Open

Spurious no bid for this getPayload payload found #636

michaelsproul opened this issue Mar 20, 2024 · 5 comments

Comments

@michaelsproul
Copy link

I have logs from a Lighthouse user where mev-boost logged:

level=error msg="no bid for this getPayload payload found, was getHeader called before?" blockHash=X method=getPayload parentHash=Y slot=N slotUID=M ua=Lighthouse/v5.1.0-10a38a8 version=1.7

this is despite a bid for the same blockHash appearing 1.5s earlier:

level=debug msg="new best bid" blockHash=X blockNumber=B method=getHeader parentHash=Y pubkey=? slot=N slotUID=M txRoot=? ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/N/Y/XXX" value=0.036... version=1.7

Is this a known issue? What is the expected impact when mev-boost isn't aware of the bid? This block ended up being published ~5s after mev-boost attempted to unblind it and I'm trying to establish whether that was the relay's fault
@avalonche
Copy link
Collaborator

this seems to be an issue on mev-boost losing information about the bids. Is it possible that mev-boost restarted after the header call but before the get payload call?

@SimonSMH1015
Copy link

SimonSMH1015 commented Mar 21, 2024 •
edited
Loading

Thank you @michaelsproul for helping me with my questions, I was going to post one myself. lol

The mev-boost didnot restart at that time. @avalonche

I post the mev debug logs at that slot below. If you need more, just contact me here.

ime="2024-03-16T01:13:50.075+08:00" level=info msg="http: GET / 200" duration=0.000040 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:00.032+08:00" level=info msg="http: GET / 200" duration=0.000035 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:10.459+08:00" level=info msg="http: GET / 200" duration=0.000045 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:11.674+08:00" level=debug msg=getHeader method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 ua=Lighthouse/v5.1.0-10a38a8 version=1.7

time="2024-03-16T01:14:11.679+08:00" level=info msg="getHeader request start - 679 milliseconds into slot 8641569" genesisTime=1606824023 method=getHeader msIntoSlot=679 parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotTimeSec=12 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7

time="2024-03-16T01:14:11.874+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:11.883+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:11.913+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:11.996+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:12.191+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:12.349+08:00" level=debug msg="bid received" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0xb509880ed4640b6c080b9ee4cdaef72f42d815a50481652f1c5af29b24697e66 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461354838835401 version=1.7

time="2024-03-16T01:14:12.349+08:00" level=debug msg="new best bid" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0xb509880ed4640b6c080b9ee4cdaef72f42d815a50481652f1c5af29b24697e66 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461354838835401 version=1.7

time="2024-03-16T01:14:12.349+08:00" level=debug msg="bid received" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461374838835401 version=1.7

time="2024-03-16T01:14:12.35+08:00" level=debug msg="new best bid" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461374838835401 version=1.7

time="2024-03-16T01:14:12.625+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:12.634+08:00" level=warning msg="error making request to relay" error="Get \"https://relay.wenmerge.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7

time="2024-03-16T01:14:12.634+08:00" level=info msg="best bid" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 relays="https://0x8b5d2e73e2a3a55c6c87b8b6eb92e0149a125c852751db1422fa951e42a09b82c142c3ea98d0d9930b056a3bc9896b8f@bloxroute.max-profit.blxrbdn.com" slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 value=0.036461374838835401 version=1.7

time="2024-03-16T01:14:12.635+08:00" level=info msg="http: GET /eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 200" duration=0.961974 method=GET path=/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 status=200 version=1.7

time="2024-03-16T01:14:13.387+08:00" level=debug msg="getPayload request starts" method=getPayload version=1.7

time="2024-03-16T01:14:13.398+08:00" level=info msg="submitBlindedBlock request start - 2398 milliseconds into slot 8641569" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a genesisTime=1606824023 method=getPayload msIntoSlot=2398 parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotTimeSec=12 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7

time="2024-03-16T01:14:13.398+08:00" level=error msg="no bid for this getPayload payload found, was getHeader called before?" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:13.857+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.117+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.303+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.312+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.381+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.487+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"failed to publish block\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.547+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.552+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"failed to publish block\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.655+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.9+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:14.936+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.04+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.115+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request, block was never seen by this relay or bid was accepted but payload was lost, if you got this bid from us, please contact the relay\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.146+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.196+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.22+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.448+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.562+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.634+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.674+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.681+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.694+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.734+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="max retries exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:15.892+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.027+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.189+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.293+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.34+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.459+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.57+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.624+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.808+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.843+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.863+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.896+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request, block was never seen by this relay or bid was accepted but payload was lost, if you got this bid from us, please contact the relay\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.909+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="max retries exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:16.912+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.013+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="max retries exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.239+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.243+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.34+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="max retries exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.344+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="max retries exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.7+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.802+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="request context error after 5 attempts: context deadline exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.831+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:17.932+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="request context error after 5 attempts: context deadline exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:18.045+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"timestamp too late\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:18.146+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="request context error after 6 attempts: context deadline exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:18.37+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request, block was never seen by this relay or bid was accepted but payload was lost, if you got this bid from us, please contact the relay\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:18.47+08:00" level=error msg="error making request to relay" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="request context error after 4 attempts: context deadline exceeded" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7

time="2024-03-16T01:14:18.47+08:00" level=error msg="no payload received from relay!" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe relaysWithBid= slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7

time="2024-03-16T01:14:18.47+08:00" level=info msg="http: POST /eth/v1/builder/blinded_blocks 502" duration=5.083896 method=POST path=/eth/v1/builder/blinded_blocks status=502 version=1.7

time="2024-03-16T01:14:20.034+08:00" level=info msg="http: GET / 200" duration=0.000039 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:30.084+08:00" level=info msg="http: GET / 200" duration=0.000045 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:40.068+08:00" level=info msg="http: GET / 200" duration=0.000039 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:14:50.38+08:00" level=info msg="http: GET / 200" duration=0.000031 method=GET path=/ status=200 version=1.7

time="2024-03-16T01:15:00.03+08:00" level=info msg="http: GET / 200" duration=0.000042 method=GET path=/ status=200 version=1.7

@michaelsproul michaelsproul mentioned this issue Mar 21, 2024
Closed
@metachris
Copy link
Collaborator

The error log you mentioned is more of a warning, but doesn't impact the request flow (code is here).

The reason for the missed slot here was that the bloXroute relay didn't return the payload:

Let's take a closer look at the logs for slot 8641569, mostly around block hash 0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a which is used for getPayload:

time="2024-03-16T01:14:12.349+08:00" level=debug msg="bid received" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0xb509880ed4640b6c080b9ee4cdaef72f42d815a50481652f1c5af29b24697e66 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461354838835401 version=1.7
time="2024-03-16T01:14:12.349+08:00" level=debug msg="new best bid" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0xb509880ed4640b6c080b9ee4cdaef72f42d815a50481652f1c5af29b24697e66 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461354838835401 version=1.7
time="2024-03-16T01:14:12.349+08:00" level=debug msg="bid received" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461374838835401 version=1.7
time="2024-03-16T01:14:12.35+08:00" level=debug msg="new best bid" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461374838835401 version=1.7
time="2024-03-16T01:14:12.625+08:00" level=debug msg="no-content response" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7
time="2024-03-16T01:14:12.634+08:00" level=warning msg="error making request to relay" error="Get \"https://relay.wenmerge.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" version=1.7
time="2024-03-16T01:14:12.634+08:00" level=info msg="best bid" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 relays="https://0x8b5d2e73e2a3a55c6c87b8b6eb92e0149a125c852751db1422fa951e42a09b82c142c3ea98d0d9930b056a3bc9896b8f@bloxroute.max-profit.blxrbdn.com" slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 value=0.036461374838835401 version=1.7
time="2024-03-16T01:14:12.635+08:00" level=info msg="http: GET /eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 200" duration=0.961974 method=GET path=/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 status=200 version=1.7
time="2024-03-16T01:14:13.387+08:00" level=debug msg="getPayload request starts" method=getPayload version=1.7
time="2024-03-16T01:14:13.398+08:00" level=info msg="submitBlindedBlock request start - 2398 milliseconds into slot 8641569" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a genesisTime=1606824023 method=getPayload msIntoSlot=2398 parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotTimeSec=12 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7
time="2024-03-16T01:14:13.398+08:00" level=error msg=", was getHeader called before?" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://aestus.live/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.ultrasound.money/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.398+08:00" level=debug msg="calling getPayload" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:13.857+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.117+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.303+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://agnostic-relay.net/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.312+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.381+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://boost-relay.flashbots.net/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.487+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"failed to publish block\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.max-profit.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.547+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://mainnet-relay.securerpc.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.552+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"failed to publish block\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.655+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"not the expected proposer index\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.edennetwork.io/eth/v1/builder/blinded_blocks" version=1.7
time="2024-03-16T01:14:14.9+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"no execution payload for this request - block was never seen by this relay\"}\n{\"code\":400,\"message\":\"no execution payload for this request\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://relay.wenmerge.com/eth/v1/builder/blinded_blocks" version=1.7
  1. The bid is returned by bloXroute relay (bloxroute.regulated.blxrbdn.com):
time="2024-03-16T01:14:12.349+08:00" level=debug msg="new best bid" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0xb509880ed4640b6c080b9ee4cdaef72f42d815a50481652f1c5af29b24697e66 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/header/8641569/0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe/0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746" value=0.036461354838835401 version=1.7
  1. When calling getPayload, the bloxroute relay does not return the payload:
time="2024-03-16T01:14:14.552+08:00" level=warning msg="error making request to relay, retrying" blockHash=0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a error="HTTP error response: 400 / {\"code\":400,\"message\":\"failed to publish block\"}\n" method=getPayload parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 ua=Lighthouse/v5.1.0-10a38a8 url="https://bloxroute.regulated.blxrbdn.com/eth/v1/builder/blinded_blocks" version=1.7

@metachris metachris mentioned this issue Mar 22, 2024
Open
@michaelsproul
Copy link
Author

Thanks @metachris. I'm trying to find out from bloxroute why this payload didn't get revealed in a timely way

@avalonche
Copy link
Collaborator

avalonche commented Mar 22, 2024 •
edited
Loading

Looking at the logs the best bid was block with block hash 0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd.

time="2024-03-16T01:14:12.634+08:00" level=info msg="best bid" blockHash=0x168f6a50367e3a44edc8303d95a46eaa1dda7043753dc1d944c889e61d6b21dd blockNumber=19441763 method=getHeader parentHash=0x2e76661b14aebdad9c56a2c1cd24f2b5fb803da77b0d830e984ddaf675c01dbe pubkey=0xa6af3fdbf1369e7836ffb219f187306ba262caca38b981c42b05ed3f45c24ec811782436a821f1572ab3ca522ddd4746 relays="https://0x8b5d2e73e2a3a55c6c87b8b6eb92e0149a125c852751db1422fa951e42a09b82c142c3ea98d0d9930b056a3bc9896b8f@bloxroute.max-profit.blxrbdn.com" slot=8641569 slotUID=434ea013-56e7-4ead-ba6b-d71cc8d10850 txRoot=0x5a42e1b2b848527ca8b45deb2811aaa5c0793a37c71b802b77dc69448eea2324 ua=Lighthouse/v5.1.0-10a38a8 value=0.036461374838835401 version=1.7

even though mev-boost saw 0x655cbb05b29223cabe49fbcb001d0704aad928c73e4f611d9bb9561adcb3036a it was not the best bid and so did not save it. However that was the block the proposer requested but mev-boost assumes the proposer to request the same bid in getHeader. It doesn't affect the request flow, just a warning.

From the logs the error from bloxroute is that the block publishing step failed

failed to publish block

while other relays responded with

not the expected proposer index

So seems like the view of the consensus state was different between the relays and the proposer, resulting in the block publishing and validation step to fail. Will need to confirm with bloxroute though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@metachris @michaelsproul @avalonche @SimonSMH1015