Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to benchmark pull requests #6274

Merged
merged 10 commits into from
Sep 23, 2024

Conversation

rashidnhm
Copy link
Contributor

Context: Add ability to benchmark a pull request

Description of the Change:
This pull request adds the ability to benchmark a PR.

When a user with either admin or maintain permissions comments /benchmark on a pull request, this workflow will trigger a webhook to benchmarks repo with sufficient details to benchmark the PR, it is then up to the benchmarks action to report back with results.

Once the webhook is sent, a 👍 reaction is added to the comment to indicate it was successful.

Benefits:
New feature.

Possible Drawbacks:
This is an MVP product, we may want to iterate on this to add further information or functionality. However, the current state is sufficient for the basic needs of benchmarking.

Related GitHub Issues:
None. sc-73314

Copy link
Contributor

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md with:

  • A one-to-two sentence description of the change. You may include a small working example for new features.
  • A link back to this PR.
  • Your name (or GitHub username) in the contributors section.

Copy link

codecov bot commented Sep 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.70%. Comparing base (f2e1e73) to head (019c0f9).
Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6274   +/-   ##
=======================================
  Coverage   99.70%   99.70%           
=======================================
  Files         444      444           
  Lines       42226    42226           
=======================================
  Hits        42103    42103           
  Misses        123      123           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

@jay-selby jay-selby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks Rashid!

I think there is some future work on the benchmark side to cleanly format the output reported back to the PR.

Copy link
Contributor

@mudit2812 mudit2812 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but is it possible to try it out before we merge? Just want to see it in action to verify it works. The code itself looks fine to me, but I also don't use javascript much. Or is it not possible since the PR on the benchmarks suite-side is still open?

@rashidnhm rashidnhm added the urgent Mark a pull request as high priority label Sep 20, 2024
@rashidnhm
Copy link
Contributor Author

Looks good to me, but is it possible to try it out before we merge? Just want to see it in action to verify it works. The code itself looks fine to me, but I also don't use javascript much. Or is it not possible since the PR on the benchmarks suite-side is still open?

I have tried to get the code to work as much as I could in a private fork ... however, yes, none of the changes here can be tested as part PR unfortunately

@mudit2812
Copy link
Contributor

Looks good to me, but is it possible to try it out before we merge? Just want to see it in action to verify it works. The code itself looks fine to me, but I also don't use javascript much. Or is it not possible since the PR on the benchmarks suite-side is still open?

I have tried to get the code to work as much as I could in a private fork ... however, yes, none of the changes here can be tested as part PR unfortunately

All good then. This is all pretty early iteration work anyway, we can always come back later with bug fixes 😄

@rashidnhm rashidnhm merged commit 9af9fd8 into master Sep 23, 2024
37 checks passed
@rashidnhm rashidnhm deleted the sc-73314-add-benchmarks-webhook branch September 23, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
urgent Mark a pull request as high priority
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants