Skip to content

Commit

Permalink
source-kinesis: bump maximum retry attempts
Browse files Browse the repository at this point in the history
The AWS v2 Go SDK has a built in retry mechanism, but the default only retries 3
times before returning a fatal error.

This bumps that limit up so that the connector will continue retrying for ~5
minutes before failing with an error. This is the same configuration as we use
for source-dynamodb.
  • Loading branch information
williamhbaker committed Nov 25, 2024
1 parent bf70257 commit 3fdcc30
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions source-kinesis/connection.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"slices"
"sync"

"github.com/aws/aws-sdk-go-v2/aws/retry"
"github.com/aws/aws-sdk-go-v2/service/kinesis"
"golang.org/x/sync/errgroup"

Expand Down Expand Up @@ -52,6 +53,14 @@ func connect(ctx context.Context, cfg *Config) (*kinesis.Client, error) {
credentials.NewStaticCredentialsProvider(cfg.AWSAccessKeyID, cfg.AWSSecretAccessKey, ""),
),
awsConfig.WithRegion(cfg.Region),
awsConfig.WithRetryer(func() aws.Retryer {
// Bump up the number of retry maximum attempts from the default of 3. The maximum retry
// duration is 20 seconds, so this gives us around 5 minutes of retrying retryable
// errors before giving up and crashing the connector.
//
// Ref: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/retries-timeouts/
return retry.AddWithMaxAttempts(retry.NewStandard(), 20)
}),
}

if cfg.Advanced.Endpoint != "" {
Expand Down

0 comments on commit 3fdcc30

Please sign in to comment.