Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Request are not cached resulting in high network usage #1141

Open
SeDemal opened this issue Sep 17, 2024 · 1 comment
Open

bug: Request are not cached resulting in high network usage #1141

SeDemal opened this issue Sep 17, 2024 · 1 comment
Labels
bug Something isn't working
Milestone

Comments

@SeDemal
Copy link
Collaborator

SeDemal commented Sep 17, 2024

Provide environment information

any

Describe the bug

Some queries are not cached but should. This results in an insane amount of queries (This is a 7h span).
This is because the media information for each element needs to be queried separately, resulting in 20 calls every 5 seconds.
image

To reproduce

Mostly visible using the media request widgets.
Let the page opened for a while then go check your queries in pihole.

Additional information

No response

@SeDemal SeDemal changed the title bug: bug: Request are not cached resulting in high network usage Sep 17, 2024
@Meierschlumpf Meierschlumpf added this to the 1.0.0-alpha.1 milestone Oct 18, 2024
@Meierschlumpf Meierschlumpf added the bug Something isn't working label Dec 4, 2024
@Meierschlumpf Meierschlumpf modified the milestones: 1.0.0-beta, 1.0.0 Dec 17, 2024
@manuel-rw
Copy link
Member

manuel-rw commented Dec 31, 2024

I think the requests are behaving as designed - we don't know whether the remote data has changed or not - therefore we must fetch all the time.
However, 128K DNS requests do seem extreme - I wonder whether we don't properly cache DNS resolve requests.
@SeDemal can you confirm, that the problem is the amount of DNS requests and not the amount & frequency of HTTP requests? As far as I saw, we cannot reduce the amount of HTTP requests (except the frequency) because the data we need is distributed over multiple API endpoints.
I don't think this is a critical issue and we should not consider this as a blocker for the 1.0 release.
I think that our fetch() simply does not respect any of the TTL nor caches any lookups. See https://www.npmjs.com/package/cacheable-lookup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants