Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce build times in CI #2085

Merged
merged 1 commit into from
Oct 2, 2024
Merged

reduce build times in CI #2085

merged 1 commit into from
Oct 2, 2024

Conversation

vroldanbet
Copy link
Contributor

@vroldanbet vroldanbet commented Oct 2, 2024

optimized tests with various changes. 40% improvement in CI in some cases.

  • datastore tests now run parallel. CRDB and Postgres had issues so it's disabled parallelization there
  • Parallelized integrity / no integrity tests in CRDB
  • removed code coverage by default in unit tests - was not being used
  • added t.Parallel() in the unit tests taking the longest

before

image

after

image

@github-actions github-actions bot added area/schema Affects the Schema Language area/api v1 Affects the v1 API area/datastore Affects the storage system area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) area/dispatch Affects dispatching of requests labels Oct 2, 2024
@vroldanbet vroldanbet force-pushed the make-unit-tests-faster branch 3 times, most recently from 3ec25a8 to b369aa2 Compare October 2, 2024 11:38
@github-actions github-actions bot added the area/CLI Affects the command line label Oct 2, 2024
@vroldanbet vroldanbet force-pushed the make-unit-tests-faster branch 3 times, most recently from 2560b66 to 642c5af Compare October 2, 2024 12:21
by enabling parallel tests.
Made unit tests run 30% faster

Also added some parallelism to Datastore
tests, but opt-in. MemDB and datastore proxy
tests are now parallel.

Also changed CRDB test entry points to run in
parallel, since the introduction of integrity tests
meant double the tests to run, and sequentially
@vroldanbet vroldanbet marked this pull request as ready for review October 2, 2024 13:23
@vroldanbet vroldanbet requested a review from a team as a code owner October 2, 2024 13:23
@vroldanbet vroldanbet changed the title make unit tests faster reduce build times in CI Oct 2, 2024
@vroldanbet vroldanbet self-assigned this Oct 2, 2024
Copy link

@Verolop Verolop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@vroldanbet vroldanbet added this pull request to the merge queue Oct 2, 2024
Merged via the queue into main with commit d07d490 Oct 2, 2024
22 checks passed
@vroldanbet vroldanbet deleted the make-unit-tests-faster branch October 2, 2024 13:42
@github-actions github-actions bot locked and limited conversation to collaborators Oct 2, 2024
Copy link
Contributor

@tstirrat15 tstirrat15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious about goleak change

@@ -30,7 +29,7 @@ import (
var ONR = tuple.ObjectAndRelation

func TestSimpleCheck(t *testing.T) {
defer goleak.VerifyNone(t, append(testutil.GoLeakIgnores(), goleak.IgnoreCurrent())...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this no longer necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no - it's moved into a TestMain function which makes it execute for all tests in the package

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/api v1 Affects the v1 API area/CLI Affects the command line area/datastore Affects the storage system area/dispatch Affects dispatching of requests area/schema Affects the Schema Language area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants