Keep data in fails cases in sync service

[AurelienFT]

Linked Issues/PRs

Closes #2357

Description

This pull request introduces a caching mechanism to the sync service to avoid redundant data fetching from the network. The most important changes include adding a cache module, modifying the Import struct to include a cache, and updating related methods to utilize this cache.

Caching Mechanism:

• crates/services/sync/src/import.rs: Added a new cache module and integrated it into the Import struct. Updated methods to use the cache for fetching and storing headers and blocks.
• Cache mechanism allow use to retrieve a stream of batches of either cached headers, cached full blocks, or range to fetch data.

Test Updates:

• Update the P2P port in mocks to use async to simulate more complex tests needed for this feature.

This PR contains 50% of changes in the tests and addition of tests in the cache.

Checklist

• Breaking changes are clearly marked as such in the PR description and changelog
• New behavior is reflected in tests
• The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

• I have reviewed the code myself
• I have created follow-up issues caused by this PR and linked them here

[netrome]

I don't understand the import task well enough to approve right now. I need clarification on the following points:

1. How do we ensure this cache doesn't grow forever? Is the Import task short-lived? While the import task launches short-lived streams, it seems like a long-living task to me.
2. How can we be sure we'll query exactly the same ranges as we have cached? Where is that invariant maintained.

Let me know if you want to jump on a call to chat about this, or just write if I'm missing something obvious here.

[AurelienFT]

@netrome Thanks for taking the time to review this Regarding your interrogations :
1 - Yes for me it will leave a long time but all asked data should be ok at some point and so be cleared otherwise we will only have batch_size as number of element in the cache. But I'm not very sure about this that's why I placed a comment about this in "Interrogation" in the PR. Maybe we need a pruning management.
2 - I was thinking that we re-ask all the same ranges because the batch_size doesn't change but the starting point can change to the last synced block and so ranges can change. I think you are right then the ranges can change I will ask few questions to @xgreenx

[rafal-ch]

So far looks good, I need to have a deeper look at the tests though.

[AurelienFT]

Convert to draft because of big refacto.

Copilot reviewed 6 out of 10 changed files in this pull request and generated no suggestions.

Files not reviewed (4)

• crates/services/sync/src/import/test_helpers/pressure_peer_to_peer.rs: Evaluated as low risk
• crates/services/sync/src/import/tests.rs: Evaluated as low risk
• crates/services/sync/src/ports.rs: Evaluated as low risk
• CHANGELOG.md: Evaluated as low risk

[xgreenx]

The change looks really good=)

[xgreenx]

While it is true, let's return an error and print a log that this place shouldn't be reachable

[AurelienFT]

I have added a log and I returned a malformed batch which is used as error in this whole process. I don't want to change the whole architecture of the module for this error. (the other solution is to panic like it's done here :

fuel-core/crates/services/sync/src/import.rs

Line 475 in 99135e3

.expect("We checked headers are not empty above"),

)

[AurelienFT]

@xgreenx Thanks for the kind comment and I have addressed all of your concerns some may still need some answers :)

[netrome]

Nice stuff! Some minor questions and comments from me, but overall looks good.

[netrome]

Missed this previously, but saw the clippy error in CI regarding the arithmetic operation and want to ensure we avoid the array slicing as well since that can also panic.

[netrome]

Thanks for the update 🙏

[AurelienFT]

@xgreenx Why you prefer to make the check inside the loop instead of the split at the end to simplify the logic in the loop ?

[xgreenx]

Why you prefer to make the check inside the loop instead of the split at the end to simplify the logic in the loop ?

It is much easier to read. It is more performant, because the old implementation was doing into_iter, chunks, collect each batch_size blocks.