Performance test suite #261
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This option allows to compile the testnet app in a mode that allows autoapproving any request, in order to measure the performance of various operations either in speculos, or on a real device.
bigspider
force-pushed
the
perftests
branch
2 times, most recently
from
ba8cbd1
to
16475d6
Compare
bigspider
force-pushed
the
perftests
branch
12 times, most recently
from
dc0d935
to
c943cea
Compare
These tests are no longer worth the maintenance burden, as they test specific conditions and only apply to an old version of the app protocol. The corresponding tests for the newer version of the protocol are kept in the codebase.
It is useful to test that large amounts are shown correctly; instead of generating the psbt synthetically, we hardcode a fixed psbt so we can hardcode the screenshots.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a test suite to measure the performance of the app on a series of synthetically generated testcases.
The
Makefileadds a newAUTOAPPROVE_FOR_PERF_TESTSthat compiles a version of the app that automatically approves any request, in order to remove the UX interactions from the measurements. As a sanity precaution, compilation is aborted if this flag is enabled when compiling on mainnet.The test suite can be ran either on speculos or on a real device; measured times on speculos are of course not a good indicator of the performance on real devices; nevertheless, the test suite is added as a CI job, since comparing the performance reports before and after a change can give a useful indicator about expected or unexpected changes in the running time. The performance report is also saved as a json artifact.
The PR adds an initial set of benchmarks related to signing different kinds of transactions. More extensive benchmarks can be added in the future.
Some legacy tests using the
txmakertool are deleted; no longer worth the maintenance burden.