Add a basic metric for tracking the capacity in VecDeque buffer #383
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ayirr7
changed the title
ayirr7
changed the title
ayirr7
changed the title
untitaker
approved these changes
Aug 28, 2024
untitaker
approved these changes
Sep 11, 2024
onkar
reviewed
Sep 11, 2024
rust-arroyo/src/processing/dlq.rs
Outdated
Comment on lines
411
to
418
| if metric_prob <= 0.01 { | ||
| // Number of partitions in the buffer map | ||
| gauge!( | ||
| "arroyo.consumer.dlq_buffer.assigned_partitions", | ||
| self.buffered_messages.len() as u64, | ||
| ); | ||
| } | ||
|
|
There was a problem hiding this comment.
This pattern is repeated several times. Can this be moved to a function?
There was a problem hiding this comment.
I didn't notice this code. In fact, this sampling should not be required at all. it has some performance benefits in python but they should not be present for rust. let's remove the sampling entirely.
untitaker
requested changes
Sep 11, 2024
rust-arroyo/src/processing/dlq.rs
Outdated
Comment on lines
411
to
418
| if metric_prob <= 0.01 { | ||
| // Number of partitions in the buffer map | ||
| gauge!( | ||
| "arroyo.consumer.dlq_buffer.assigned_partitions", | ||
| self.buffered_messages.len() as u64, | ||
| ); | ||
| } | ||
|
|
There was a problem hiding this comment.
I didn't notice this code. In fact, this sampling should not be required at all. it has some performance benefits in python but they should not be present for rust. let's remove the sampling entirely.
untitaker
approved these changes
Sep 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We can also measure the length of the queue, but as far as I can tell
VecDequeonly really has thecapacity()function.If we see this metric drop to 0 and then spike up to a new very high value, we would be able to detect the scenario where the buffer is being reallocated. Not sure if we need anything more complicated than this since we have the mutable buffer being accessed in both of these functions anyways and we can just check in these functions.
Also tracks the number of partitions in the BTreeMap