Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not run benchcomp in every PR #2604

Merged
merged 1 commit into from
Jul 14, 2023

Conversation

celinval
Copy link
Contributor

Description of changes:

The performance benchmark PR is taking a lot of our CI time and many times it's not necessary. For example, benchmark job over 2h while all the other runs ran in less than 40min in this run

This PR moves the performance benchmark to a separate workflow that only runs on pushes to main / release tags and PR's that are labeled with 'Z-BenchCI'

Resolved issues:

Related RFC:

Call-outs:

We still need to fix the release tag runs to compare against previous release. We haven't had this working before, so I hope it's not a blocker for this PR. I could just remove the push to tag trigger for now if people prefer or create an issue to finish supporting release comparison.

I created a label for now, but we can rename it or remove it if this PR doesn't get accepted.

Testing:

Checklist

  • Each commit message has a non-empty body, explaining why the change was made
  • Methods or procedures are documented
  • Regression or unit tests are included, or existing tests cover the modified code
  • My PR is restricted to a single feature or bugfix

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

Move performance benchmark to a separate workflow that only runs on
push to main / release tags and PR's that are labeled with 'Z-BenchCI'
@celinval celinval requested a review from a team as a code owner July 14, 2023 20:35
@celinval celinval added the Z-BenchCI Tag a PR to run benchmark CI label Jul 14, 2023
Copy link
Contributor

@jaisnan jaisnan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I agree with the change. Should we document the need for the label in our dev-documentation guide?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Z-BenchCI Tag a PR to run benchmark CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants