SQS Fixed Polling (throttling)

[jeroenvandevelde]

Hi, I checked out the rewrite of SQS that is currently happening. However I couldn't find a way to use SQS in a fixed polling mode as in poll every 5 seconds for 5 messages.

Is this something that is being considered, I am willing to help with this if necessary?

Currently we are using https://github.com/JaidenAshmore/java-dynamic-sqs-listener for this however we are looking at switching to Spring's implementation.

Use Case: We have an application which sends messages over SQS to connectors. We use a fixed polling mechanism to avoid overflowing (throttling) the third parties that sit behind the connectors.

[maciejwalkowiak]

I think with the new implementation you could manually start container and stop it 5 seconds later? Perhaps @tomazfernandes has other ideas?

[tomazfernandes]

Hi @jeroenvandevelde, thanks for bringing this up, and thanks @maciejwalkowiak for the ping.

Interesting library, I didn't know it. To be honest, I didn't fully understand what's the back pressure strategy it uses - it seems that you can set this 5 second poll, but if all 5 consumer threads finish processing it'll fetch for more messages earlier, did I get it right?

We currently have a Semaphore-based back pressure strategy, where you can set the maximum number of concurrent messages and the framework will try to keep this many messages being processed. If your third party supports 5 concurrent messages at a time, just set the maxInflightMessagesPerQueue property to 5, and the framework will make sure no more than 5 messages are active at a given time for that queue.

Does this sound like it would work for your use-case?

Getting a bit into the implementation, we have the SemaphoreBackPressureHandler that implements the BackPressureHandler interface and is responsible for this control. I've thought we could eventually implement a RateLimiterBackPressureHandler that would allow a N polls per M seconds back pressure control. Google Guava has an interesting RateLimiter implementation we could use.

Considering we already have a lot for this first milestone, I think I'd rather rely on the current approach for now before introducing a new one though.

Let me know your thoughts on this.

[jeroenvandevelde]

No, our current behaviour is: we set a 5 seconds poll and if any/all consumers are finished before the 5 seconds are over they need to wait until they are over. In this way we know a max number of messages per second we process/send to external systems.

Here, you can find Jaiden Ashmore explain it with an example of how to do it in his library although it is a "bit of a hack".

If we limit our system to max 5 concurrent messages, we have no control over the max throughput as in messages/ per second.
As you don't the know time a message takes upfront, could be faster/slower than a second.

The reason we do this is because we have external systems who can't handle more than 50 calls/second, and we want to stay below that.

Thanks for the reply!

[tomazfernandes]

I see, looks like the use case for a RateLimiter-based back pressure handler.

A simpler option would be including a minDelayBetweenPolls property. The drawback would be that if a poll returns less than 5 messages, it would wait 5 seconds before fetching new ones.

I think we can look into adding one of the two options in M3, WDYT @maciejwalkowiak? Maybe we can open an issue for this.

If you want to experiment with creating your own BackPressureHandler it should be doable too - please let me know if you try that and run into any issues.

I'm not sure what would be a feasible workaround - I've thought of allowing maxInflightMessages < maxMessagesPerPoll but that would mean buffering messages and potential messageVisibility problems, so I'd rather not go there unless strictly necessary.

Maybe you can try holding off message execution completion until 5 seconds are passed, or as @maciejwalkowiak suggested, experiment with starting and stopping the containers.

Please let me know your thoughts on this, thanks.

[maciejwalkowiak]

This can be added either in M3 or even later. @jeroenvandevelde please open an issue.

[jeroenvandevelde]

Thanks @tomazfernandes @maciejwalkowiak for your input. I created an issue here. Feel free to give feedback if more information or whatsoever is required.

@tomazfernandes I will try to take a look at the branch with the SQS rework and try to figure out how to best fit it in, with @bramvanswalm.

[tomazfernandes]

Actually, if I understand this library's behavior correctly, we could achieve the same behavior by setting permitAcquireTimeout to 5 seconds and maxInflight to 5.

This way if all 5 messages are processed before 5 seconds, the permits will be acquired and the next poll will happen immediately. If e.g. at 5 seconds only 3 messages have been processed, the framework will fetch for 3 messages.

Maybe this permitAcquireTimeout option could have a better name that better expresses this kind of behavior.

[tomazfernandes]

Maybe maxDelayBetweenPolls, with a disclaimer that if by this time we still have all maxInflight messages being processed it won't poll and will start waiting again.

[riteshmaurya]

has maxDelayBetweenPolls been implemented?

[tomazfernandes]

has maxDelayBetweenPolls been implemented?

Yes, it's in SqsContainerOptions.

Please check the docs at:
https://docs.awspring.io/spring-cloud-aws/docs/3.1.1/reference/html/index.html#sqscontaineroptions-descriptions