Improve performance for test generation and execution #359
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GalRogozinski
approved these changes
Feb 1, 2024
types/testingutils/keymanager.go
Outdated
| ) | ||
|
|
||
| func getHash(data [][]byte) string { | ||
| hasher := sha256.New() |
There was a problem hiding this comment.
you can consider using MuHash or something else which is faster
There was a problem hiding this comment.
fnv is easy fast and simple for tasks like this. built in the go stdlib
MatusKysel
reviewed
Feb 5, 2024
types/testingutils/keymanager.go
Outdated
Comment on lines
49
to
52
| mu.Lock() | ||
| defer mu.Unlock() | ||
|
|
||
| hash := getHash(slashableDataRoots) |
There was a problem hiding this comment.
better approach would be getHash before locking, so we keep it locked as short as possible
MatusKysel
approved these changes
Feb 5, 2024
y0sher
reviewed
Feb 7, 2024
types/testingutils/keymanager.go
Outdated
Comment on lines
30
to
33
| var ( | ||
| instancesMap = make(map[string]*testingKeyManager) | ||
| mu sync.Mutex | ||
| ) |
There was a problem hiding this comment.
generally I would say global state is a bad idea, but since this is a test generation code and you are protecting it properly, its seems ok to me.
y0sher
approved these changes
Feb 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
The purpose of this PR is to optimize the performance of test generation and execution with the purpose of improving work quality on future changes.
The
ssvmodule is the one that takes the most time. After running a CPU profiling test, we noticed that theNewTestingKeyManagerfunction consumes 80% of the total time, which is explained by the fact that it's called multiple times and takes 26 milliseconds to run (according to benchmark tests).Nonetheless, the
NewTestingKeyManagerfunction returns atestingKeyManagerobject that is only used to validate and sign data but does not change state throughout a test execution. Thus, we propose adding a singleton pattern for this object.Metrics
To illustrate the number of times the
NewTestingKeyManagerfunction is called, take a look at the following tableNote: the number after the dash (/) is the number of different instances created, which depends on a
slashableDataRootsparameter.Performance comparison
With the singleton pattern, if the appropriate object already has been created, the function takes 5 microseconds, rather than 26 milliseconds.
All time values shown below are in seconds (or in minutes:seconds if it has ":" ).
For test generation, we got the following performance results.
For test execution, we got the following performance results.