Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
The performance benchmark PR is taking a lot of our CI time and many times it's not necessary. For example, benchmark job over 2h while all the other runs ran in less than 40min in this run
This PR moves the performance benchmark to a separate workflow that only runs on pushes to main / release tags and PR's that are labeled with 'Z-BenchCI'
Resolved issues:
Related RFC:
Call-outs:
We still need to fix the release tag runs to compare against previous release. We haven't had this working before, so I hope it's not a blocker for this PR. I could just remove the push to tag trigger for now if people prefer or create an issue to finish supporting release comparison.
I created a label for now, but we can rename it or remove it if this PR doesn't get accepted.
Testing:
How is this change tested? I tested this in my fork. Here is the job run for a PR where I applied the label and here is the run after I pushed to my fork main branch.
Is this a refactor change? It's a CI refactoring change
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.