fix(tpu-v2): fix tpu-v2 wait for payment spend and extract secret

[laruh]

• In TakerCoinSwapOpsV2 trait wait_for_taker_payment_spend function was renamed to find_taker_payment_spend_tx (to avoid misunderstanding with wait_for_confirmation logic)

  In EthCoin function find_taker_payment_spend_tx_impl was provided for trait find_taker_payment_spend_tx method

• Added new extract_secret_v2 method in TakerCoinSwapOpsV2 trait. implemented for UTXO and EthCoin.
  Did it as EthCoin legacy extract_secret doesnt fit TPU V2, so its better to require extract_secret_v2 for coins.

• in lp_connect_start_bob and in lp_connected_alice provided fallback to legacy swap protocol.

  Legacy fallback happens when:
  user sets ctx.use_trading_proto_v2() as false, or other trading side uses legacy swap protocol, then even if user set ctx.use_trading_proto_v2() as true, they have to use legacy swap as well.
  Other reason of starting legacy swap, even if ctx.use_trading_proto_v2() is true, is the trading coin which doesnt support TPU V2

- provide swap protocol version #2112 (review) reverted

• handle require confirmations before changing taker/maker swap state as Completed
  Note: you can find the implementation above this line in taker_swap_v2.rs and in maker_swap_v2.rs

Self::change_state(Completed::new(), state_machine).await

• moved SecretHashAlgo to crypto lib and provided detect_secret_hash_algo_v2 function for TPU (starting from this commit c1a5063)
  ETH coin should use SHA256 in TPU.

[laruh]

ps: will resolve conflicts or cherrypick commits in new up to date branch after eth-maker-tpu-v2 merge #2211

[mariocynicys]

thanks, first iteration

[mariocynicys]

shouldn't we advance from_block if the event wasn't found?

[laruh]

shouldn't we advance from_block if the event wasn't found?

Shouldn't change blocks range, It introduces risks to skip necessary block.

Receiving empty events list does not necessarily indicate that there are no events, network latency can cause delays in the propagation and indexing of event logs even after a transaction is mined.
After a transaction is mined, the logs related to it need to be extracted and made available for querying. This process is not instantaneous.

Also we dont know all the nuances and differences of all blockchains. It is much safer to keep block range starting from swap start block.

[laruh]

UPD: l suggest to add if events.is_empty check to continue loop without moving forward
UPD2: added is empty check 8d5ed46

[mariocynicys]

will this ever be None? will this be a recoverable state then? otherwise we can terminate early.

[laruh]

will this ever be None?

are you talking about event.transaction_hash? This type is from a dependency, we should handle it as it is. The transaction_hash could be None if the log is emitted by a transaction in a pending state. Once the transaction is included in a mined block, the value should be Some.

will this be a recoverable state then?

For TPU V2 we aim to have automatic recover process, if find_taker_payment_spend_tx return error then refund process will be started

komodo-defi-framework/mm2src/mm2_main/src/lp_swap/taker_swap_v2.rs

Lines 1698 to 1716 in c700b59

	let taker_payment_spend = match state_machine
	.taker_coin
	.find_taker_payment_spend_tx(
	&self.taker_payment,
	self.taker_coin_start_block,
	state_machine.taker_payment_locktime(),
	)
	.await
	{
	Ok(tx) => tx,
	Err(e) => {
	let next_state = TakerPaymentRefundRequired {
	taker_payment: self.taker_payment,
	negotiation_data: self.negotiation_data,
	reason: TakerPaymentRefundReason::MakerDidNotSpendInTime(format!("{}", e)),
	};
	return Self::change_state(next_state, state_machine).await;
	},
	};

otherwise we can terminate early

Could clarify what do you mean by termination? You want to return error and break loop?
We should try to find transaction in the loop until time is out

[mariocynicys]

The transaction_hash could be None if the log is emitted by a transaction in a pending state. Once the transaction is included in a mined block, the value should be Some.

aren't we getting events till the current mined block? so this tx shouldn't be pending?

For TPU V2 we aim to have automatic recover process

I just meant we can not do nothing about the fact that tx hash is none.
nothing to do with swap recovery.

how I was thinking (which might be wrong) is that some event types don't have a tx hash which means we supplied a bad event id from the beginning meaning that we can't proceed further. this might not be the case though, gotta read more about this, excuse my eth illiteracy 😂

[laruh]

aren't we getting events till the current mined block? so this tx shouldn't be pending?

yeah, you are right as we are using from and to block filters for eth_getLogs API, tx should be confirmed

    async fn events_from_block(
        &self,
        swap_contract_address: Address,
        event_name: &str,
        from_block: u64,
        to_block: u64,
    ) -> MmResult<Vec<Log>, FindPaymentSpendError> {
        let contract_event = TAKER_SWAP_V2.event(event_name)?;
        let filter = FilterBuilder::default()
            .topics(Some(vec![contract_event.signature()]), None, None, None)
            .from_block(BlockNumber::Number(from_block.into()))
            .to_block(BlockNumber::Number(to_block.into()))
            .address(vec![swap_contract_address])
            .build();
        let events_logs = self
            .logs(filter)
            .await
            .map_err(|e| FindPaymentSpendError::Transport(e.to_string()))?;
        Ok(events_logs)
    }

some event types don't have a tx hash

I think you are confused a bit. Events/logs themselves don't necessarily contain the transaction hash. Instead, the transaction hash is associated with the transaction that emitted the event. So Log type from web3 just contains info about tx which emitted this log.

Note about event and log words. event in Solidity is a way for a contract to emit logs during the execution of a function.
So they are close words, just events refer to the Solidity construct used in the smart contract to emit logs, while logs refer to the actual data that is recorded in the blockchain when events are emitted.

So using from to block range we are looking for Log which was emitted by spend taker payment transaction.

As for empty tx hash I would like to refer to previous comment #2261 (comment) empty event list or none transaction_hash are not 100% that there is no tx which we are looking for, it could be just blockchain indexation delays or other reasons.

[mariocynicys]

i will read further regarding the truncated (not necessarily empty? right?) event list issue, any resourced regarding the indexaction delay issues and such would be so helpful!

regarding an event not having a transaction_hash thought, how would we successfully get the event which has None for the transaction_hash and then try again and all of a sudden we get a Some transaction_hash! is that possible? are these event logs mutable?

[laruh]

i will read further regarding the truncated (not necessarily empty? right?) event list issue, any resourced regarding the indexaction delay issues and such would be so helpful!

I would say that chain reorganization, network latency, node synchronization lag issues could cause missing information issues. These problems usually temporarily, as I understand. You should wait for more confirmed blocks also try to fetch the info from different nodes.

regarding an event not having a transaction_hash thought, how would we successfully get the event which has None for the transaction_hash and then try again and all of a sudden we get a Some transaction_hash! is that possible? are these event logs mutable?

Logs are not mutable. Also they are tied to transactions. When a transaction calls a smart contract function that emits an event, this event generates log, which is permanently recorded in the blockchain.

But there’s a nuance. Lest check doc. According to the documentation https://www.chainnodes.org/docs/ethereum/eth_getLogs, a log from a pending transaction might lack a transaction hash, but when the transaction is confirmed, the log should include it.

Therefore, ideally, when we request a list of logs using the events_from_block function with from_block and to_block filters, it should return only logs from confirmed blocks, which means confirmed transactions. In this case, event.transaction_hash should ideally always be Some.

komodo-defi-framework/mm2src/coins/eth/eth_swap_v2/eth_taker_swap_v2.rs

Lines 530 to 531 in 8d5ed46

	if let Some(event) = found_event {
	if let Some(tx_hash) = event.transaction_hash {

We dont need to change from_block. However, if Log has transaction_hash:None, it doesn't mean Log doesn't have transaction (actually its not correct by itself, as logs are not owners of txs), it means smth went wrong as eth_getLogs API with fromBlock and toBlock filters will use confirmed blocks.

[laruh]

Some additional info https://docs.alchemy.com/docs/deep-dive-into-eth_getlogs#what-are-logs-or-events

Logs and events are used synonymously—smart contracts generate logs by firing off events, so logs provide insights into events that occur within the smart contract. Logs can be found on transaction receipts.

Anytime a transaction is mined, we can see event logs for that transaction by making a request to eth_getLogs and then take actions based off those results. For example, if a purchase is being made using crypto payments, we can use eth_getLogs to see if the sender successfully made the payment before providing the item purchased.

[mariocynicys]

However, if Log has transaction_hash:None, it doesn't mean Log doesn't have transaction, it means smth went wrong as eth_getLogs API with fromBlock and toBlock filters will use confirmed blocks.

im missing u between the lines here so let me repeat that to you and see if i got it correctly. transaction_hash MUST always be Some eventually, if it was None this is a temporary reporting/mining/etc... thing.

[laruh]

However, if Log has transaction_hash:None, it doesn't mean Log doesn't have transaction, it means smth went wrong as eth_getLogs API with fromBlock and toBlock filters will use confirmed blocks.

im missing u between the lines here so let me repeat that to you and see if i got it correctly. transaction_hash MUST always be Some eventually, if it was None this is a temporary reporting/mining/etc... thing.

Yes, in the context of not using "pending" tag in "eth_getLogs" API, we expect transaction_hash always be Some.
The best we can do is to repeat loop cycle until time is out, if None occurred.

[onur-ozkan]

Could you add some PR description to describe the work and help us on review?

[laruh]

Could you add some PR description to describe the work and help us on review?

Updated PR description

[laruh]

@shamardy @mariocynicys @dimxy I reverted swap version f8ba975 to not to block this PR

[mariocynicys]

as per our conversation thread here: #2261 (comment)

i think this flag doesn't make sense anymore

[laruh]

i think this flag doesn't make sense anymore

Discussion you're referencing is about making wait for confirmation time equal one lock time after adding refund or spend functionality to TPU

[borngraced]

Great work! last note from me

[borngraced]

can we have a type definition for [u8; 32] and use everywhere instead ?

type PaymentSecret = [u8; 32];

[laruh]

I dont like that we will have to create new values instead of using references

[borngraced]

we can do &'a PaymentSecret I guess or is your concern different ?

[laruh]

we can do &'a PaymentSecret I guess or is your concern different ?

I mean when from this &'a [u8] we make this &'a [u8; 32] no new memory is allocated, both pointers reference the same memory
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0af3327b4e03e3d51d14855b582a1ecc

use std::convert::TryInto;

fn main() {
    let slice: &[u8] = &[0; 32];
    let array: &[u8; 32] = slice.try_into().expect("Not 32 bytes long");

    let slice_raw: *const u8 = slice.as_ptr();
    println!("{:?}", slice_raw);
    let array_raw: *const u8 = array.as_ptr();
    println!("{:?}", array_raw);

    assert_eq!(slice_raw, array_raw); 
}

This type PaymentSecret = [u8; 32]; will force us to allocate new memory to create fixed size array from slice

[dimxy]

I'd suggest to refactor such objects as the new type pattern, like struct PaymentSecret([u8; 32]),
but I guess this is better to be done as a dedicated PR.

[laruh]

I'd suggest to refactor such objects as the new type pattern, like struct PaymentSecret([u8; 32]), but I guess this is better to be done as a dedicated PR.

same as #2261 (comment), it means we have to allocate new mem from slice to create owned fixed-size array and put it into tuple struct

[dimxy]

I mean we always deal with objects themselves, no need to extract their internal content and copy it.
So you may just store refs to objects

[laruh]

I mean we always deal with objects themselves, no need to extract their internal content and copy it. So you may just store refs to objects

Ah, I forgot that rust is the best lang ever and we can create this

struct PaymentSecret<'a>(&'a [u8; 32]);

Yes, we can do some refactoring, added todo #1895 (comment)

[mariocynicys]

I think this thread still holds (trying to refund is a lost game (in this specific circumstance at least, i.e. secret revealed and payment spend doesn't wanna confirm)).

But approving this anyways since we will implement spend recovery (or rather re-trying to spend) in a later PR and the refund path introduced doesn't degrade the swap success rate or something.

[dimxy]

nit: could this not be simply fn my_addr_as_string(&self) -> String;?
why convert arbitrary addresses?

[laruh]

you are right, we can just use my_addr inside this function and do to string conversion
0f31d07

[laruh]

(trying to refund is a lost game (in this specific circumstance at least, i.e. secret revealed and payment spend doesn't wanna confirm)).

As discussed in dm, currently TPU supports only automatic refund functionality.
It’s always worth attempting a refund, even if the secret has been revealed, as other party could also encounter issues when trying to spend the payment. In such a case, both sides would simply follow the "refund" path.

In the future, we plan to implement an automatic "refund or spend" functionality if error occur. The loop will determine the result based on whichever action (refund or spend) succeeds first. In the worst-case scenario, if nothing happens for an long time, user can call "stop swap" and mark it as failed.

I think it’s worth providing user the ability to manually trigger a "refund or spend" with RPC on TPU too. If user has to stop the automated loop, they might still be able to refund or spend the payment manually later.

[laruh]

#2261 (comment)
I also think we should use a simple integer swap_version, as the discussion has dragged on. New features should simply bump the swap_version, allowing users to access new features by default if they update the app (since we were planning to make "stable" features mandatory anyway).