Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Beebop databases in CI #86

Merged
merged 11 commits into from
Dec 9, 2024
Merged

Cache Beebop databases in CI #86

merged 11 commits into from
Dec 9, 2024

Conversation

absternator
Copy link
Contributor

@absternator absternator commented Dec 6, 2024

This PR does work to allow caching of the beebop databases. The playwright CI and jest integration CI both require pulling databases which can take a long time. These don't change thus we are now caching, so every subsequent run we don't have to redownload databases.

To allow caching having added mount cli option to run_tests that allows to mount GHA directory into the /storage folder (where dbs are downloaded)

Testing: can re-run pipeline jest CI / unit_and_integration_backend (20.x) (pull_request) and you will see in logs of _run all components _ step that Database already exists for the multiple databases

@absternator absternator marked this pull request as ready for review December 6, 2024 09:34
@absternator absternator changed the base branch from poppunk-version-dev to main December 6, 2024 09:34
Copy link

codecov bot commented Dec 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.83%. Comparing base (97b5726) to head (52bede2).
Report is 12 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #86   +/-   ##
=======================================
  Coverage   98.83%   98.83%           
=======================================
  Files          34       34           
  Lines        2060     2060           
  Branches      270      270           
=======================================
  Hits         2036     2036           
  Misses         24       24           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@absternator absternator changed the title feat: jest try cache Cache Beebop databases Dec 6, 2024
@absternator absternator changed the title Cache Beebop databases Cache Beebop databases in CI Dec 6, 2024
Copy link
Collaborator

@EmmaLRussell EmmaLRussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

So the database ends up in the cache because the first time it runs, the script will write it out to the host location of the bind mount?

README.md Outdated
@@ -77,6 +77,8 @@ You can also run everything outside pm2, by separately running:
- `./scripts/run_server`
- `./scripts/run_client`

*Note: If you wish to override the volune with a custom bind mount pass in -mount {MOUNT_NAME} into `run_test` script*
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*Note: If you wish to override the volune with a custom bind mount pass in -mount {MOUNT_NAME} into `run_test` script*
*Note: If you wish to override the storage volume with a custom bind mount pass in -mount {MOUNT_NAME} into `run_test` script*

Comment on lines 8 to 18
while [ $# -gt 0 ]; do
case "$1" in
-mount)
MOUNT="$2"
shift 2
;;
*)
shift
;;
esac
done
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is shared logic with run _test to find the mount arg - could pull this out into a shared script - or just use a positional arg if it's easier! But it's quite nice and clear to have a named arg.

@absternator
Copy link
Contributor Author

Nice!

So the database ends up in the cache because the first time it runs, the script will write it out to the host location of the bind mount?

Yup exactly 😄

@absternator absternator merged commit 1c753a7 into main Dec 9, 2024
11 checks passed
@absternator absternator deleted the cache-pipeline branch December 9, 2024 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants