-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache parsed results to speed up initial load #5659
Labels
Comments
liady
added a commit
that referenced
this issue
Oct 4, 2024
This PR introduces `IndexedDB` cache for parse results, keyed by file name and string contents (so every change in the file contents invalidates the parse cache). This greatly improves the >1st time load on every project. <video src="https://github.com/user-attachments/assets/faa3560b-81ca-4f1d-9d92-52526bf7f004"></video> Important points: 1. The cache is implemented in the worker level - to keep our current parse flow as identical to now as possible. If the feature flag is on, the worker tries to look for the file in the cache, compares the content - and if there is a cache hit it returns it from the cache instead of parsing it. 2. When a parsing does happen - if the feature flag is on, the parsed results are stored in the cache. 3. The cache is not project specific - allowing for cached results to be shared between projects (if the file name and contents are similar) 4. Currently arbitrary code (chunks of code we send as `code.tsx`) is not being cached, this can be controlled using a feature flag. 5. This PR also adds a settings pane for controlling cache behavior. The pane allows controlling the cache, the cache log and manually clear the cache if necessary. The cache settings toggle (with sound 🔉) : <video src="https://github.com/user-attachments/assets/a3c901c6-1b87-4013-a9ab-a2f531fb4457"></video> **Commit Details:** - The main logic changes are in `parser-printer-worker.ts` and `parse-cache-utils.worker.ts`. - The worker now gets a `parsingCacheOptions` argument, which controls whether or not to use the cache (and also logging) **Manual Tests:** I hereby swear that: - [X] I opened a hydrogen project and it loaded - [X] I could navigate to various routes in Preview mode Fixes #5659
liady
added a commit
that referenced
this issue
Dec 13, 2024
This PR introduces `IndexedDB` cache for parse results, keyed by file name and string contents (so every change in the file contents invalidates the parse cache). This greatly improves the >1st time load on every project. <video src="https://github.com/user-attachments/assets/faa3560b-81ca-4f1d-9d92-52526bf7f004"></video> Important points: 1. The cache is implemented in the worker level - to keep our current parse flow as identical to now as possible. If the feature flag is on, the worker tries to look for the file in the cache, compares the content - and if there is a cache hit it returns it from the cache instead of parsing it. 2. When a parsing does happen - if the feature flag is on, the parsed results are stored in the cache. 3. The cache is not project specific - allowing for cached results to be shared between projects (if the file name and contents are similar) 4. Currently arbitrary code (chunks of code we send as `code.tsx`) is not being cached, this can be controlled using a feature flag. 5. This PR also adds a settings pane for controlling cache behavior. The pane allows controlling the cache, the cache log and manually clear the cache if necessary. The cache settings toggle (with sound 🔉) : <video src="https://github.com/user-attachments/assets/a3c901c6-1b87-4013-a9ab-a2f531fb4457"></video> **Commit Details:** - The main logic changes are in `parser-printer-worker.ts` and `parse-cache-utils.worker.ts`. - The worker now gets a `parsingCacheOptions` argument, which controls whether or not to use the cache (and also logging) **Manual Tests:** I hereby swear that: - [X] I opened a hydrogen project and it loaded - [X] I could navigate to various routes in Preview mode Fixes #5659
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Since parsing of large projects takes so long, and we do not store the parsed model (only the code) on the server (mainly due to the size of the requests), we would like to cache the parsed results locally. There are a few possible goals we can achieve here, so it is important when taking this ticket to assess a solution's suitability based on which of these goals it can achieve, how long it will take to implement and test, and how complex it is.
Possible goals:
Important caveats:
projectVersion
each time) or time consuming (moving all parsing into a separate package so we can use the version number of that package).Note this work should not be started until #5655 has been completed
The text was updated successfully, but these errors were encountered: