Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Baseline to Benchmarks Report #25

Open
SkyeHoefling opened this issue Sep 5, 2021 · 3 comments
Open

Add Baseline to Benchmarks Report #25

SkyeHoefling opened this issue Sep 5, 2021 · 3 comments

Comments

@SkyeHoefling
Copy link
Contributor

Description

Add a baseline benchmark to each printed table. This will show the existing benchmark compared to the PR benchmark side-by-side. The data is saved in the repository, so we just need to extract it and merge it with the existing table

@SkyeHoefling
Copy link
Contributor Author

@mitchelsellers @kenny-sellers I have a couple options for us to include baseline benchmarks, which one do you like best? Please vote with the emojis below

👍

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

🚀

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

🎉 - Create 2 tables
PR Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Baseline Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Please leave any comments you may have

@kenny-sellers
Copy link
Collaborator

I voted. One thought and maybe I am an odd duck, what if baseline was first and then the PR result was below? In my mind, that makes it a little easier to compare the two because naturally I expect the baseline to be first.

@SkyeHoefling
Copy link
Contributor Author

Something like this?
Baseline Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

PR Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants