Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Each release, cache the ICS files and only push the changed files #462

Open
beyarkay opened this issue Aug 13, 2023 · 0 comments
Open

Each release, cache the ICS files and only push the changed files #462

beyarkay opened this issue Aug 13, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers waiting-on-time Not a high priority or require more time to implement than I have at the moment

Comments

@beyarkay
Copy link
Owner

At the moment when there's a change in the schedule, each of the 600+ calendar files will be updated, even if the old file and the new file are identical. It's quite common for there to be no change to the old and new files because the Cape Town loadshedding schedules and the Eskom loadshedding schedules are largely disjoint, so if Eskom announces an update then none of the CPT schedules will be changed.

This is a problem because uploading the files is the most expensive step of the release process. Uploading takes up to 10minutes each time, and it also consumes the hourly rate limit. In the past, there have been times when Eskom and Cape Town announced multiple schedule updates within an hour, and eskom-calendar exceeded the GitHub rate limit quota, and so had to wait about 30 minutes until the quota got reset and the files could be uploaded. Also, the number of calendar files isn't going down any time soon, so this problem of the rate limit quota will only get worse.

This could be fixed by modifying the beyarkay/update-existing-release repo to also upload a cachefile .cache which would be a YAML-formatted file containing a list of all the files, their hashes, and the last time they were updated. Logic would need to be added to only upload a file if the new hash and the existing hash have changed.

Ideally this change would be upstreamed to the original IsaacShelton/update-existing-release repo, but the in-house fork beyarkay/update-existing-release is quite behind and I'm not sure if upstreaming these changes might cause other issues.

@beyarkay beyarkay self-assigned this Aug 13, 2023
@beyarkay beyarkay added enhancement New feature or request good first issue Good for newcomers waiting-on-time Not a high priority or require more time to implement than I have at the moment labels Aug 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers waiting-on-time Not a high priority or require more time to implement than I have at the moment
Projects
None yet
Development

No branches or pull requests

1 participant