Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TooManyRequestsError doc questions (doubts) #1701

Open
guilherme90 opened this issue Jun 27, 2024 · 2 comments
Open

TooManyRequestsError doc questions (doubts) #1701

guilherme90 opened this issue Jun 27, 2024 · 2 comments

Comments

@guilherme90
Copy link

Hello. I apologize for creating this ticket. It is not a bug but a question about using the tool.

I'm reading the doc, I even read the code a lot. However, I have doubts about the example in the documentation. After all, where is this "TooManyRequestsError" class? Was this just an example of how we should work around when a timeout occurs on the consumer?

If yes, then should I consider doing something like

try {
 // something
} catch (e: Error | any) {
    if (error.name === 'KafkaJSNumberOfRetriesExceeded') {
        consumer.pause([{ topic, partitions: [partition] }])
        setTimeout(() => {
            consumer.resume([{
                topic,
                partitions: [partition],
                ...consumerOptions
            }])
        }, 5000)
    }
}

Doc example

consumer.run({
    partitionsConsumedConcurrently: 3, // Default: 1
    eachMessage: async ({ topic, partition, message }) => {
      // This will be called up to 3 times concurrently
        try {
            await sendToDependency(message)
        } catch (e) {
            if (e instanceof TooManyRequestsError) {
                consumer.pause([{ topic, partitions: [partition] }])
                // Other partitions will keep fetching and processing, until if / when
                // they also get throttled
                setTimeout(() => {
                    consumer.resume([{ topic, partitions: [partition] }])
                    // Other partitions that are paused will continue to be paused
                }, e.retryAfter * 1000)
            }

            throw e
        }
    },
})

Thanks

@yohaansunnie
Copy link

Thanks for posting the question @guilherme90 , I also have the same question going by the docs.
My work has this use-case and hoping someone could provide the right direction here.

@unematiii
Copy link

unematiii commented Jul 30, 2024

I believe the TooManyRequestsError is st that your service/consumer should raise when it can't keep up with the workload. Name is just an example. Also, I'd be careful with re-throwing the error, as it counts towards the max retries, eventually causing the consumer to crash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants