v24.11rc4: Calling connect fails with Unable to connect, no address known for peer

[JssDWt]

Issue and Steps to Reproduce

• node l1 and l2 are connected via a channel
• node l2 restarts
• node l2 calls connect to l1 with explicit host and port

Connect fails with Unable to connect, no address known for peer, while the connection details were in the request.

getinfo output

v24.11rc4

[JssDWt]

I can fix my integration tests by omitting the connect call after restart. The error seems wrong to me though. Possible change that caused it: 23dc10c.

[rustyrussell]

OK, this is fascinating. Here's what's happening:

1. On restart, l2's gossipd aggressively tries to connect to l1 (it's the only node it knows about, and it wants gossip). It doesn't have an address though, so as a last-ditch effort, it tries to do a DNS lookup.
2. You tell it to connect, with an address. It's already connecting, so it simply appends to the addresses.
3. The DNS lookup doesn't work (those servers don't answer, and anyway we're not on mainnet, so we shouldn't even try)
4. Because that was our "last ditch" effort, we don't re-check to see if any addresses have been added.

Of course, l1 knows how to connect to l2, and eventually it does.

You can simply ask l1 to connect to l2 not vice-versa and it will work. But I shall queue a few fixes for this case next release.

[grubles]

I think this might be preventing channels from being opened? The connection at some point drops, then when manually trying to reconnect withlightning-cli connect <pubkey> there is some kind of error reconnect loop that occurs. This is with v24.11 release BTW, not a release candidate.

Here is a log snippet of this happening:

2024-12-09T18:50:04.442Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Connected out, starting crypto                                                                       
2024-12-09T18:50:04.462Z DEBUG   036e680823a129ac9a2eeb8c43514a5b6cb09ae28e3e698a0e34aaa8865d6cc344-hsmd: Got WIRE_HSMD_ECDH_REQ                                                                                   
2024-12-09T18:50:04.462Z DEBUG   hsmd: Client: Received message 1 from client                                                                                                                                      
2024-12-09T18:50:04.462Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Connect OUT                                                                                          
2024-12-09T18:50:04.462Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_out WIRE_INIT                                                                                   
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_in WIRE_INIT                                                                                    
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Handed peer, entering loop                                                                           
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_out WIRE_GOSSIP_TIMESTAMP_FILTER                                                                
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-gossipd: seeker: chosen as startup peer                                                                        
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-gossipd: seeker: starting gossip (EVERYTHING)                                                                  
2024-12-09T18:50:04.463Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-chan#3: Peer has reconnected, state CHANNELD_AWAITING_LOCKIN: connecting subd                                  
2024-12-09T18:50:04.465Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-channeld-chan#3: pid 171612, msgfd 85                                                                          
2024-12-09T18:50:04.465Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-chan#3: Already have funding locked in                                                                         
2024-12-09T18:50:04.465Z UNUSUAL 035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-chan#3: Ignoring fee limits!                                                                                   
2024-12-09T18:50:04.465Z DEBUG   hsmd: new_client: 3                                                                                                                                                               
2024-12-09T18:50:04.465Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-channeld-chan#3: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1    
2024-12-09T18:50:04.465Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-channeld-chan#3: init LOCAL: remote_per_commit = 028ac29a7a2256e00be60fe343b28c0cd15eff7b129c9b16edd7d298da03b3
a3f6, old_remote_per_commit = 028ac29a7a2256e00be60fe343b28c0cd15eff7b129c9b16edd7d298da03b3a3f6 next_idx_local = 1 next_idx_remote = 1 revocations_received = 0 feerates { SENT_ADD_ACK_REVOCATION:1250 } range 1-
4294967295 blockheights { SENT_ADD_ACK_REVOCATION:0 }, our current 225659                                                                                                                                          
2024-12-09T18:50:04.466Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT                                                                   
2024-12-09T18:50:04.466Z DEBUG   hsmd: Client: Received message 18 from client                                                                                                                                     
2024-12-09T18:50:04.466Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT                                                                   
2024-12-09T18:50:04.466Z DEBUG   hsmd: Client: Received message 18 from client                                                                                                                                     
2024-12-09T18:50:04.466Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-channeld-chan#3: peer_out WIRE_CHANNEL_REESTABLISH                                                             
2024-12-09T18:50:04.466Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-channeld-chan#3: billboard: Sent reestablish, waiting for theirs                                               
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Activating for message WIRE_CHANNEL_REESTABLISH                                                      
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-lightningd: Telling connectd to send error 00111c675c099a12ff9f34256bcfd6d4030939b17889c52940437f9b7a069314b8e3
00226368616e6e656c20696e20737461746520434c4f53494e47445f434f4d504c455445                                                                                                                                           
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_out WIRE_CHANNEL_REESTABLISH                                                                    
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_out WIRE_ERROR                                                                                  
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: disconnect_peer                                                                                      
2024-12-09T18:50:04.468Z DEBUG   connectd: drain_peer                                                                                                                                                              
2024-12-09T18:50:04.468Z DEBUG   connectd: drain_peer draining subd!                                                                                                                                               
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-lightningd: peer_disconnect_done                                                                               
2024-12-09T18:50:04.468Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Will try reconnect in 300 seconds                                                                    
2024-12-09T18:50:04.468Z DEBUG   plugin-funder: Cleaning up inflights for peer id 035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923                                                               
2024-12-09T18:50:08.494Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Connected out, starting crypto                                                                       
2024-12-09T18:50:08.504Z DEBUG   gossipd: seeker: no peers, waiting                                                                                                                                                
2024-12-09T18:50:08.504Z DEBUG   gossipd: seeker: need more peers for gossip (have 0)                                                                                                                              
2024-12-09T18:50:08.504Z DEBUG   lightningd: attempting connection to 035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923 for additional gossip                                                     
2024-12-09T18:50:08.505Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Adding 0 addresses to important peer                                                                 
2024-12-09T18:50:08.514Z DEBUG   036e680823a129ac9a2eeb8c43514a5b6cb09ae28e3e698a0e34aaa8865d6cc344-hsmd: Got WIRE_HSMD_ECDH_REQ                                                                                   
2024-12-09T18:50:08.514Z DEBUG   hsmd: Client: Received message 1 from client                                                                                                                                      
2024-12-09T18:50:08.515Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: Connect OUT                                                                                          
2024-12-09T18:50:08.515Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_out WIRE_INIT                                                                                   
2024-12-09T18:50:08.621Z DEBUG   035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-connectd: peer_in WIRE_INIT                                                                                    
2024-12-09T18:50:08.621Z INFO    035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923-chan#3: Peer transient failure in CHANNELD_AWAITING_LOCKIN: Disconnected                                       

This is a brand new signet channel with an LND v0.18.3-beta node and seems stuck at CHANNELD_AWAITING_LOCKIN.

      {
         "peer_id": "035ca2fe4793a5e789ce846062eb4834f573c060d9200ce77544a29b48a0aa5923",
         "connected": false,
         "state": "CHANNELD_AWAITING_LOCKIN",
         "channel_id": "bc7b355dbb04076c370e64e92ecc5d5e9c4f3eebbd67e3a396526439bdc55fdb",
         "short_channel_id": "225649x270x0",
         "our_amount_msat": 1000000000000,
         "amount_msat": 1000000000000,
         "funding_txid": "db5fc5bd39645296a3e367bdeb3e4f9c5e5dcc2ee9640e376c0704bb5d357bbc",
         "funding_output": 0
      }