Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only cuda jobs generate empty cache on build-main and build-main-self-hosted workflows #4944

Closed
3 tasks done
youtalk opened this issue Jul 4, 2024 · 0 comments · Fixed by #4945
Closed
3 tasks done
Assignees
Labels
type:ci Continuous Integration (CI) processes and testing.

Comments

@youtalk
Copy link
Member

youtalk commented Jul 4, 2024

Checklist

  • I've read the contribution guidelines.
  • I've searched other issues and no duplicate issues were found.
  • I'm convinced that this is not my fault but a bug.

Description

According to Autoware CI Metrics, it has been confirmed that the no-cuda jobs can build 2 to 5 times faster due to the PR #4865. However the build acceleration is unstable for the cuda jobs.

https://autowarefoundation.github.io/autoware-ci-metrics/
Screenshot from 2024-07-02 10-42-02

This is because the GitHub Actions cache generated by the cuda jobs is highly likely to be empty.

https://github.com/autowarefoundation/autoware/actions/caches
Screenshot from 2024-07-04 09-01-57

Expected behavior

Both no-cuda and cuda jobs are consistently more than twice as fast.

Actual behavior

The cuda jobs remain as slow as before.

Steps to reproduce

Run build-main or build-main-self-hosted workflows.

Versions

No response

Possible causes

A similar issue has been posted for buildkit-cache-dance action, which is used to save the --mount=type=cache cache of docker build on GitHub Actions to the GitHub Actions cache.
reproducible-containers/buildkit-cache-dance#33

We would like to take a workaround for the time being.

Additional context

No response

@youtalk youtalk added the type:ci Continuous Integration (CI) processes and testing. label Jul 4, 2024
@youtalk youtalk self-assigned this Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:ci Continuous Integration (CI) processes and testing.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant