[TON]: Eliminate nondeterminism in TON BoC serialization

[10gic]

Fix issue #4004

The test case SignMessageToTransferAndDeployWalletV5R1 occasionally fails.

Root Cause

A HashMap is unordered, so iterating over it multiple times may yield different results.

wallet-core/rust/frameworks/tw_ton_sdk/src/boc/boc_to_raw_boc.rs

Line 85 in af67cae

for index_cell in cells_by_hash.values() {

Resolution

Change the HashMap to an ordered data structure, such as a BTreeMap.

[satoshiotomakan]

Good catch! I believe you spent a lot of time on trying to find it

[satoshiotomakan]

This still will be non-deterministic. That's because HashMap is randomly seeded:
https://doc.rust-lang.org/stable/std/collections/struct.HashMap.html
Look like we need to replace HashMap<H256, IndexedCellRef> with BTreeMap<H256, IndexedCellRef> everywhere in this file, and even add a type alias like type CellsByHash = BTreeMap<H256, IndexedCellRef>.

[10gic]

You are right. A HashMap is unordered; however, the subsequent code does not iterate over it directly; instead, it converts it to a Vec and then sorts it by the index of the cell (not the HashMap key).

wallet-core/rust/frameworks/tw_ton_sdk/src/boc/boc_to_raw_boc.rs

Line 30 in af67cae

index_slice.sort_unstable_by(|a, b| a.borrow().index.cmp(&b.borrow().index));

So, I think it's deterministic.

Even if I use a BTreeMap, I won't remove the sorting process because it needs to be sorted by the index of the cell.

[satoshiotomakan]

It's also passed into

wallet-core/rust/frameworks/tw_ton_sdk/src/boc/boc_to_raw_boc.rs

Lines 41 to 42 in af67cae

	let raw_cells = raw_cells_from_cells(cells_iter, &cells_by_hash)?;
	let root_indices = root_indices(&boc.roots, &cells_by_hash)?;

[10gic]

It's also passed into

wallet-core/rust/frameworks/tw_ton_sdk/src/boc/boc_to_raw_boc.rs

Lines 41 to 42 in af67cae

	let raw_cells = raw_cells_from_cells(cells_iter, &cells_by_hash)?;
	let root_indices = root_indices(&boc.roots, &cells_by_hash)?;

The scenarios you mentioned do not iterate over the HashMap.

[satoshiotomakan]

Ok I see, you're right, I didn't notice that.
We still have two options with pros and cons:

1. Use BTreeMap within build_and_verify_index only, and then convert it into HashMap.
   Cons: Double memory allocation + iterate over all items within BTreeMap to insert them into HashMap
2. Use BTreeMap within build_and_verify_index and other functions too.
   Cons: Searching for a Cell hash within BTreeMap has O(logn) complexity, while searching within HashMap has O(1).

I'd follow the second way tbh as it's more clear, and less overhead from my point of view

[10gic]

I've made the requested adjustments to the code. Could you please review it again? The code has been formatted using cargo fmt.

[satoshiotomakan]

Thank you for the adjustments! LGTM