Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Solana Vault Swaps Elections #5355

Merged
merged 66 commits into from
Nov 27, 2024
Merged

Conversation

ramizhasan111
Copy link
Contributor

@ramizhasan111 ramizhasan111 commented Oct 24, 2024

Pull Request

Closes: PRO-1521 , PRO-1522

Checklist

Please conduct a thorough self-review before opening the PR.

  • I am confident that the code works.
  • I have written sufficient tests.
  • I have written and tested required migrations.
  • I have updated documentation where appropriate.

Summary

note: please ignore some of the naming around Contract Swaps. That will be fixed before merging. Renaming done.

@ramizhasan111 ramizhasan111 changed the base branch from main to feat/solana-program-swaps-close-accounts October 24, 2024 15:40
@ramizhasan111 ramizhasan111 changed the base branch from feat/solana-program-swaps-close-accounts to feat/closing-solana-swap-accounts October 24, 2024 15:49
@ramizhasan111 ramizhasan111 changed the base branch from feat/closing-solana-swap-accounts to feat/solana-program-swaps-close-accounts October 25, 2024 11:59
@ramizhasan111 ramizhasan111 marked this pull request as ready for review October 25, 2024 12:13
@albert-llimos albert-llimos requested review from ahasna and removed request for a team November 1, 2024 13:39
@albert-llimos
Copy link
Contributor

I just merged #5375 on top of this so it has the entire vault swap feature so it can be reviewed together (CCM, vaultSwapParameters).
The bouncer changes can be reviewed at the end after we merge this to the base Solana Vault branch if needed, no need to do here necessarily.

engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
state-chain/node/src/chain_spec/sisyphos.rs Show resolved Hide resolved
} else {
sp_std::mem::take(&mut known_accounts.witnessed_open_accounts)
};
match Hook::close_accounts(accounts_to_close.clone()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think all of the above logic could be in the close_accounts hook (could make it close_accounts(last_close_block_number, &mut known_accounts)).

This would also make it easier to mock and test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

depending on whether we were successfully able to close accounts, we have to set the unsynchronized state so it makes sense to have this logic here.

Copy link
Contributor

@kylezs kylezs Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If all the logic were in the close_accounts hook, then, we wouldn't need to have two separate hooks for get_number_of_available_sol_nonce_accounts and close_accounts - close accounts would already have access to it -> which makes it easier to mock and test, and for the writing of state, all you need is to return a result as is already the case, and just set the block number.

Then for testing this ElectoralSystem's on_finalize, it's very simple, you can just check "has hook been called" - we already have tests that do this for other ElectoralSystems so it would be very similar. Then you could also write tests for the close_accounts hook implementation itself, separately, and can do so just focussing on that logic, free from the needing to test everything in the context of the whole ElectoralSystem

Also if you pass in SolanaVaultSwapsVote could just have one hook function provided, since the api call would be out too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered moving all the logic concerning all three hooks inside one hook but realized it would not actually make testing easier. This is because in that case, writing the test for one big hook impl would then be difficult since the logic in hook impl calls external code (constructing apicall, accessing sol nonces etc). So we would have to mock those. So basically we are back where we started. We wanted to avoid create 3 mocks of 3 election hooks, instead we create one. However, that means we create those extra mocks while testing the hook impl so I dont think we gain anything. In fact, creating the mocks for the hook impl is even harder since we dont have a framework for it at the Runtime level. creating 3 mocks instead of one while writing tests for the on_finalize shouldnt be a big deal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the apicall construction be a separate trait function is probably reasonable, it's also kind of a different part of the flow, so conceptually could make things easier to follow. Though I think putting the get nonce and close accounts in one should definitely be simpler

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand what you mean. The apicall construction is inside the close_accounts function. trying to combine apicall construction and get nonces into one hook would mean we have to move the whole logic inside that hook which creates the problem of testability that I mentioned above.

In general, I think it is a pretty arbitrary and subjective choice here between having the logic in one place or splitting it. In the end logic is logic, one has to follow it to make sure it works. Its not like we don't have long pieces of logic inside single functions in other places in the codebase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, should've been clearer. I mean leaving initiate_vault_swap separate makes sense. But since the only reason we need the number of sol accounts is to decide whether we construct the close accounts call, seems to make sense to include it.

I'm not against having long pieces of logic, it's just about keeping conceptually similar things together and the boundaries between things clean. Happy to see how the tests look keeping it like this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes of course but in this case we cant test the hook impl easily as i said before so it makes sense to keep it here since we want to test and keep the hook impl trivial.

engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
@albert-llimos
Copy link
Contributor

albert-llimos commented Nov 5, 2024

For visibility, after the two last PRs I merged into this PR, these are the last items to get this done:

  • Finish tests on the SC ( test: solana vault swap electoral system tests #5382 and not sure if there's any more needed)
  • Address some minor comments left
  • Small change in the engine witnessing to not omit event accounts upon decoding error but rather submit the event account information to the SC to trigger the closure but without triggering a swap.

Afaik @ramizhasan111 is working on those 👌 . I will test the last item once completed.

Copy link
Contributor

@kylezs kylezs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still going through it, but thought I'd drop some comments now is looking good so far 👍

} else {
sp_std::mem::take(&mut known_accounts.witnessed_open_accounts)
};
match Hook::close_accounts(accounts_to_close.clone()) {
Copy link
Contributor

@kylezs kylezs Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If all the logic were in the close_accounts hook, then, we wouldn't need to have two separate hooks for get_number_of_available_sol_nonce_accounts and close_accounts - close accounts would already have access to it -> which makes it easier to mock and test, and for the writing of state, all you need is to return a result as is already the case, and just set the block number.

Then for testing this ElectoralSystem's on_finalize, it's very simple, you can just check "has hook been called" - we already have tests that do this for other ElectoralSystems so it would be very similar. Then you could also write tests for the close_accounts hook implementation itself, separately, and can do so just focussing on that logic, free from the needing to test everything in the context of the whole ElectoralSystem

Also if you pass in SolanaVaultSwapsVote could just have one hook function provided, since the api call would be out too

bouncer/shared/vault_swap.ts Outdated Show resolved Hide resolved
bouncer/tests/all_concurrent_tests.ts Show resolved Hide resolved
bouncer/tests/all_swaps.ts Show resolved Hide resolved
engine/src/witness/sol.rs Outdated Show resolved Hide resolved
engine/src/witness/sol/program_swaps_witnessing.rs Outdated Show resolved Hide resolved
engine/src/witness/sol.rs Outdated Show resolved Hide resolved
engine/src/witness/sol.rs Show resolved Hide resolved
@ramizhasan111
Copy link
Contributor Author

Apart from tests, which I am doing as another PR, this PR should be ready and I would like to merge it if there are no more open points.

Base automatically changed from feat/solana-program-swaps-close-accounts to main November 12, 2024 07:48
@albert-llimos
Copy link
Contributor

Just fyi, when this gets rebased I expect the Solana Vault swaps to fail because the encoding of cf_parameters have changed in main. I will fix that after the rebasing is done.

@dandanlen dandanlen enabled auto-merge November 20, 2024 12:50
@dandanlen dandanlen added this pull request to the merge queue Nov 20, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 20, 2024
@kylezs
Copy link
Contributor

kylezs commented Nov 20, 2024

For visibility @ramizhasan111 is going to do the missing migrations

state-chain/pallets/cf-elections/src/lib.rs Outdated Show resolved Hide resolved
state-chain/runtime/src/lib.rs Outdated Show resolved Hide resolved
@kylezs
Copy link
Contributor

kylezs commented Nov 21, 2024

The upgrade test might pass without those version changes until you run it in the queue because it'll be put on top of main, which will then require them

@dandanlen dandanlen enabled auto-merge November 21, 2024 09:26
@@ -79,7 +79,7 @@ jobs:
uses: ./.github/workflows/upgrade-test.yml
secrets: inherit
with:
run-job: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reminder: needs to be reverted before merge

@albert-llimos
Copy link
Contributor

I'm looking into the Vault swap failures after upgrade.

kylezs and others added 5 commits November 25, 2024 16:53
* chore: update transcationInId and bouncer

* chore: run CI

* chore: add logging and decrease timeout

* chore: not try runtime

* chore: add more prints

* chore: run only one Solana Vault swap

* chore: use processed commitment

* chore: use confirmed

* chore: revert sendAndConfirmTransaction

* chore: fix

* chore: add print and extra swap

* chore: remove unnecessary context transition

* chore: log electoral settings

* chore: readd missing line

* chore: improve logging

* chore: add fix to upgrade-test

* chore: add missing SOLANA_PROGRAMS_VERSION

* chore: replace nonces manually

* chore: manually skip solana vault swaps

* chore: revert logs

* chore: run upgrade

* chore: fix path

* chore: refactor env

* chore: restore files

* chore: force initial sol nonces

* chore: run upgrade ci

* chore: fix reset --hard

* chore: small refactor

* chore: remove unnecessary reset

* chore: revert files

* chore: refactor
@albert-llimos albert-llimos added this pull request to the merge queue Nov 27, 2024
@kylezs kylezs removed this pull request from the merge queue due to a manual request Nov 27, 2024
@albert-llimos albert-llimos added this pull request to the merge queue Nov 27, 2024
Merged via the queue into main with commit b3fae46 Nov 27, 2024
49 checks passed
@albert-llimos albert-llimos deleted the feat/solana-vault-swaps-election branch November 27, 2024 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants