[Bug] Client syncing issue with invalid next block, missing transaction ID in db

[WietzeSlagman]

🐛 Bug Report

During syncing on canary net a client halted syncing due to a database mismatch, the client receives a next block that contains a transaction that is not in its ledger and therefore could not be added to its DB. This lead to the client halting its syncing process aas that was the only block it would receive.

Steps to Reproduce

Currently unclear on how to get to this corrupted(?) database state, however the client seems to be unable to recoup from it. logs are below:

2024-04-15T07:00:24.062872Z TRACE snarkos_node_sync::block_sync: Updating is_block_synced: greatest_peer_height = 34087, canon_height = 34008
2024-04-15T07:00:24.063273Z TRACE snarkos_node_sync::block_sync: Prepared 1 block requests
2024-04-15T07:00:25.856542Z  WARN snarkos_node_sync::block_sync: The next block (34009) is invalid - Transaction ID 'at12pue9rvlmc08g699t0v5e8u0mt9y226gn2lachtn4kj8rt2nfs8sa46tyr' does not exist in the ledger

Expected Behavior

The node should have the correct transactions in its DB to be able to sync up to the latest tip.

Your Environment

• Running snarkOS commit: [ec943b3](https://github.com/AleoNet/snarkOS/commit/ec943b39968215e17f43b5f53fd73acba66e3a5f)
• rustc 1.77.2 (25ef9e3d8 2024-04-09)
• Ubuntu 22.04.4 LTS

[vicsn]

This might resolve the issue: https://github.com/AleoHQ/snarkOS/pull/3217
It is currently being tested