Skip to content

Commit

Permalink
source-stripe-native: increase resource update intervals
Browse files Browse the repository at this point in the history
We've been seeing a significant number of rate-limiting errors, and
users have seen their other Stripe integrations getting rate-limited
because the connector sends a massive number of API requests. Increasing
the default interval to 5 minutes should help reduce the number of API
requests the connector makes.

Other improvements could be made to reduce the number of API requests
the connector makes. Primarily, the connector could dispatch events to
different bindings based on the event type instead of having each
binding check the `/events` endpoint. However, that's a significant
refactor that will take some time & should be prioritized against all of
the other work that needs done.
  • Loading branch information
Alex-Bair committed Dec 9, 2024
1 parent a47f146 commit 164238c
Show file tree
Hide file tree
Showing 2 changed files with 87 additions and 46 deletions.
10 changes: 5 additions & 5 deletions source-stripe-native/source_stripe_native/resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ def open(
inc=ResourceState.Incremental(cursor=cutoff),
backfill=ResourceState.Backfill(next_page=None, cutoff=cutoff)
),
initial_config=ResourceConfig(name=cls.NAME),
initial_config=ResourceConfig(name=cls.NAME, interval=timedelta(minutes=5)),
schema_inference=True,
)

Expand Down Expand Up @@ -184,7 +184,7 @@ def open(
inc=ResourceState.Incremental(cursor=cutoff),
backfill=ResourceState.Backfill(next_page=None, cutoff=cutoff)
),
initial_config=ResourceConfig(name=child_cls.NAME),
initial_config=ResourceConfig(name=child_cls.NAME, interval=timedelta(minutes=5)),
schema_inference=True,
)

Expand Down Expand Up @@ -224,7 +224,7 @@ def open(
inc=ResourceState.Incremental(cursor=cutoff),
backfill=ResourceState.Backfill(next_page=None, cutoff=cutoff)
),
initial_config=ResourceConfig(name=child_cls.NAME),
initial_config=ResourceConfig(name=child_cls.NAME, interval=timedelta(minutes=5)),
schema_inference=True,
)

Expand Down Expand Up @@ -263,7 +263,7 @@ def open(
inc=ResourceState.Incremental(cursor=cutoff),
backfill=ResourceState.Backfill(next_page=None, cutoff=cutoff)
),
initial_config=ResourceConfig(name=child_cls.NAME),
initial_config=ResourceConfig(name=child_cls.NAME, interval=timedelta(minutes=5)),
schema_inference=True,
)

Expand Down Expand Up @@ -303,6 +303,6 @@ def open(
inc=ResourceState.Incremental(cursor=cutoff),
backfill=ResourceState.Backfill(next_page=None, cutoff=cutoff)
),
initial_config=ResourceConfig(name=cls.NAME),
initial_config=ResourceConfig(name=cls.NAME, interval=timedelta(minutes=5)),
schema_inference=True,
)
Loading

0 comments on commit 164238c

Please sign in to comment.