network/libp2p: DHT Bootstrap error detected on Kusama Validator

[lexnv]

Libp2p has been deployed on one of our Kusama Validators from this debug PR:

• [DO NOT MERGE] network/libp2p: Investigate requests that don't timeout #7154.

The PR is based on origin/master from commit 023763da204.

A very high number of warnings are detected as DHT timeouts and replication failures coming from the Kademlia component:

2025-01-15 12:57:21.471  WARN tokio-runtime-worker sub-libp2p: Libp2p => DHT bootstrap error: Timeout { peer: PeerId("1AWmbDZr2zafuegydwsdp68JFvC24FfREM6H9YqLkHnVMx"), num_remaining: Some(7) }

2025-01-15 13:18:35.986  WARN tokio-runtime-worker libp2p_kad::behaviour: Locating closest peers for replication failed: Err(Timeout { key: Key(b"'/\xbd\x91\xf7R\x0b\xf1}R\xe9kqB\xd4\xa3\xbf\xd3\t\xbe<\x80\xac\xdc<8\x9dB\x96\xb2\xe0\\"), success: [], quorum: 20 })

The libp2p version used: 0.54.1
Version bumped in:

• Upgrade libp2p from 0.52.4 to 0.54.1 #6248

Grafana link: https://grafana.teleport.parity.io/goto/0wyllsvHg?orgId=1

cc @paritytech/networking