Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check for rate limiting #298

Open
IanButterworth opened this issue Nov 14, 2023 · 4 comments
Open

Add check for rate limiting #298

IanButterworth opened this issue Nov 14, 2023 · 4 comments

Comments

@IanButterworth
Copy link
Member

The inexplicable 403 errors we're seeing on repos that once worked but started to fail may be rate limited.

We can probe the status here https://docs.github.com/en/rest/overview/rate-limits-for-the-rest-api?apiVersion=2022-11-28#checking-the-status-of-your-rate-limit

@IanButterworth
Copy link
Member Author

IanButterworth commented Nov 14, 2023

Note that my theory is that GitHub are serving the wrong error message.
We're seeing

github.GithubException.GithubException: 403 {"message": "Resource not accessible by integration", "documentation_url": "https://docs.github.com/rest/git/refs#create-a-reference"}

I suspect we should be seeing something along the docs here


Exceeding the rate limit
If you exceed your primary rate limit, you will receive a 403 or 429 response, and the x-ratelimit-remaining header will be 0. You should not retry your request until after the time specified by the x-ratelimit-reset header.

If you exceed a secondary rate limit, you will receive a 403 or 429 response and an error message that indicates that you exceeded a secondary rate limit. If the retry-after response header is present, you should not retry your request until after that many seconds has elapsed. If the x-ratelimit-remaining header is 0, you should not retry your request until after the time, in UTC epoch seconds, specified by the x-ratelimit-reset header. Otherwise, wait for at least one minute before retrying. If your request continues to fail due to a secondary rate limit, wait for an exponentially increasing amount of time between retries, and throw an error after a specific number of retries.

Continuing to make requests while you are rate limited may result in the banning of your integration.


@DilumAluthge
Copy link
Member

The easiest would be to (when we get a 4xx error) dump the HTTP headers that GitHub sent back as part of the 4xx. And then we can e.g. inspect the x-ratelimit-remaining header.

@IanButterworth
Copy link
Member Author

Agreed

@omus
Copy link
Contributor

omus commented Jul 23, 2024

We can configure retries with PyGithub like in this example: PyGithub/PyGithub#2759 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants