From e3c4f840aa2f138597066137fa477b6b14ad97f5 Mon Sep 17 00:00:00 2001 From: Bart de Water <496367+bdewater@users.noreply.github.com> Date: Sun, 20 Oct 2024 15:13:48 -0400 Subject: [PATCH] Add example --- guides/throttling.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/guides/throttling.md b/guides/throttling.md index 005f51f0..37598ba7 100644 --- a/guides/throttling.md +++ b/guides/throttling.md @@ -38,9 +38,31 @@ def build_enumerator(_params, cursor:) end ``` +If you want to apply throttling on all jobs, you can subclass your own EnumeratorBuilder and override the default +enumerator builder. The builder always wraps the returned enumerators from `build_enumerator` + +```ruby +class MyOwnBuilder < JobIteration::EnumeratorBuilder + class Wrapper < Enumerator + class << self + def wrap(_builder, enum) + ThrottleEnumerator.new( + enum, + nil, + throttle_on: -> { DatabaseStatus.unhealthy? }, + backoff: 30.seconds + ) + end + end + end +end + +JobIteration.enumerator_builder = MyOwnBuilder +``` + Note that it's up to you to implement `DatabaseStatus.unhealthy?` that works for your database choice. At Shopify, a helper like `DatabaseStatus` checks the following MySQL metrics: * Replication lag across all regions * DB threads * DB is available for writes (otherwise indicates a failover happening) -* [Semian](https://github.com/shopify/semian) open circuits \ No newline at end of file +* [Semian](https://github.com/shopify/semian) open circuits