"Request limit exceeded" error while running steampipe check against an AWS account

[esundeep1]

We were trying to run steampipe checks against our AWS accounts but for one of the accounts, we are getting a "Request limit exceeded" for an event name 'DescribeSnapshotAttribute'. Can someone help me with implementing some kind of throttling so to prevent the rate limit issues?

Presently we are running 'steampipe check all' with the aws plugin installed. Any help is appreciated.

[cbruno10]

Hi @esundeep1 , in the account(s) where you're hitting the request limit, how many snapshots do you have approximately? And are they spread out over multiple regions, or in a single region?

In terms of configuration options, we have several config arguments you can set in your ~/.steampipe/config/aws.spc file, including max_error_retry_attempts and min_error_retry_delay. So one suggestion would be is to try increasing both of these arguments to find a set of values that works for you, so the plugin will make more attempts when encountering throttling or other non-fatal errors, and will have a longer period in between retries, which may help space out the retry periods. More information on each of these arguments and their defaults can be found at https://hub.steampipe.io/plugins/turbot/aws#configuration.

Please let us know if adjusting these arguments helps at all!

[esundeep1]

Hi @cbruno10, we have about 1900 snapshots in a single region.

I have tried with those configuration changes (increased max_error_retry_attempts and min_error_retry_delay) but now it is failing with "timeout: context deadline exceeded" error.

Any suggestions to avoid this?

Thanks for earlier reply.

[cbruno10]

@esundeep1 Can you please provide more details on what values you had tested for both of the arguments?

[esundeep1]

max_error_retry_attempts was increased to 50
min_error_retry_delay was increased to 1000

@cbruno10

[ryanprior]

I'm having a similar rate limiting issue with a different plugin (GitLab) and I tried setting max_error_retry_attempts and min_error_retry_delay but then that plugin won't start. Are these features of the AWS plugin specifically? Is there any general-purpose way to make steampipe work slower so I can work around rate limits?

[cbruno10]

@ryanprior The GitHub plugin currently doesn't support the max_error_retry_attempts and min_error_retry_delay config arguments. I believe the AWS plugin is the only plugin to support these config args today. We started with this plugin as the AWS SDK client supports these types of arguments out of the box.

For the GitHub plugin queries you're running, can you please share more about what queries you're running, and what your organizations and repositories look like in relation to the query, e.g., how many repositories in the organization, how many users, branches, issues, etc., in the repository? This will help us better understand which tables to take a look at, as some tables use different APIs and have different limits. Thanks!