[8.12](backport #40231) update max_number_of_messages parameter description #40339
Conversation
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
(cherry picked from commit c75f9bc)
botelastic
bot
added
the
needs_team
|
This pull request doesn't have a |
|
This pull request has not been merged yet. Could you please review and merge it @kaiyan-sheng? 🙏 |
kaiyan-sheng
requested review from
leehinman,
VihasMakwana,
v1v and
pazone
and removed request for
a team
|
@kaiyan-sheng , AFAIR, backports from Mergify only need approval from the former owner. I don't know if that policy has changed since I'm not actively involved in Beats, cc @pazone |
|
This pull request has not been merged yet. Could you please review and merge it @kaiyan-sheng? 🙏 |
| @@ -261,7 +261,13 @@ The default is `10 MiB`. | |||
| ==== `max_number_of_messages` | |||
|
|
|||
| The maximum number of SQS messages that can be inflight at any time. Defaults | |||
| to 5. | |||
| to 5. When processing large amount of large size S3 objects and each object has | |||
| large amount of events, if this parameter sets too high, it can cause the input | |||
There was a problem hiding this comment.
When processing large amount of large size S3 objects and each object has large amount of events, if this parameter sets too high, it can cause the input to process too many messages concurrently, overload the agent and cause ingest failure.
Perhaps something like:
Setting this parameter too high can overload Elastic Agent and cause ingest failures in situations where the SQS messages contain many S3 objects or the S3 objects themselves contain large numbers of messages.
Or minimally:
When processing a large number of large size S3 objects that each have a large number of events, if this parameter is set too high, it can cause the input to process too many messages concurrently, overloading the agent and causing ingest failure.
There was a problem hiding this comment.
@strawgate Thanks for the review!! Ugh sorry this is a backup PR. Let me change it in a main branch in a separate PR and backport it into these branches.
| to process too many messages concurrently, overload the agent and cause ingest failure. | ||
| We recommend to keep the default value 5 and use the | ||
| {fleet-guide}/es-output-settings.html#es-output-settings-performance-tuning-settings[preset] | ||
| option to tune your Elastic Agent performance. You can optimize for throughput, |
There was a problem hiding this comment.
We should specifically recommend either Balanced or Throughput for this use-case
pazone
dismissed
their stale review
The approval is not relevant to the changes
|
This pull request has not been merged yet. Could you please review and merge it @kaiyan-sheng? 🙏 |
|
closing this PR with #40513 |
Proposed commit message
This PR is to update the documentation for max_number_of_messages to inform users to be cautious when using a large value for this var.
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.This is an automatic backport of pull request #40231 done by [Mergify](https://mergify.com).