-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ref(am1): Collect span counts for orgs prior to migrating #75485
Conversation
Some organizations are not yet on AM2, and are unsampled without span extraction running, which makes it hard to estimate usage for future plan changes to AM3. This allows us to collect metrics for only those specific customers who we can't estimate through outcomes.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #75485 +/- ##
========================================
Coverage 78.18% 78.19%
========================================
Files 6809 6816 +7
Lines 302915 303037 +122
Branches 52132 52149 +17
========================================
+ Hits 236844 236954 +110
- Misses 59686 59690 +4
- Partials 6385 6393 +8
|
@@ -187,6 +187,8 @@ def process_event( | |||
event_id=event_id, | |||
project_id=project_id, | |||
) | |||
collect_span_metrics(project, data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we move this into save_event_transaction
, and / or wrap it in a try / catch?
This line is part of a try
block, but it only DLQs KeyErrors
. Any other failure might block the consumer and cause a P0 incident.
sentry/src/sentry/ingest/consumer/processors.py
Lines 141 to 142 in 503314b
# Raise the retriable exception and skip DLQ if anything below this point fails as it may be caused by | |
# intermittent network issue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wrap in a try/catch is a good idea, sure 👍
PR reverted: 2680cfa |
@k-fish this was reverted for INC-842, please review when you're back. |
We ran into trouble with the previous PR due to a few issues; a typo in the name, exception logging in the 'has' causing a thundering herd of exceptions, and the condition order causing the problem to not trip tests / soak time. original PR for context: #75485 Notes: - `am3_tier` vs `am3-tier` as a result of a bad copy paste (out of a thread instead of the file it's registered) - `logger.exception` circumventing the `try / except`, already fixed [here](687086f#diff-9cd8ed7a349a276392e908cabcd613d3647a94eaa5956ae79e4efb604e2c9232L309) - Conditional order has `organizations:dynamic-sampling` first, which causes an early shortcut of the `and`
Summary
Some organizations are not yet on AM2, without span extraction running, which makes it hard to estimate usage for future plan changes to AM3. This allows us to collect metrics for only those specific customers who we can't estimate through outcomes.
Flag I'm using in lieu of a new option is from the subscription plan handler, should be similarly fast to an option in region mode.