Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(alerts): Debug anomaly detection #77109

Closed
wants to merge 1 commit into from

Conversation

ceorourke
Copy link
Member

This is not actually code that will be merged, but since we can't test metric alerts locally fully I pulled out the code we are having trouble with in production to be able to test it locally and make sure everything is working.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 6, 2024
expected_seasonality=alert_rule.seasonality,
)
############################
now = datetime.datetime.now()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I don't have access to self.last_update from the subscription processor, I am using the current timestamp

@@ -103,7 +104,10 @@ def format_historical_data(data: SnubaTSResult, dataset: Any) -> list[TimeSeries
formatted_data.append(ts_point)
else:
for datum in nested_data:
ts_point = TimeSeriesPoint(timestamp=datum.get("time"), value=datum.get("count", 0))
#########################
count = randrange(1, 100)
Copy link
Member Author

@ceorourke ceorourke Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no historical data locally so I populate this with random numbers. Seer wasn't happy if I did it all the same number. Put this back to falling back to 0 to test the 500 error we were getting

Copy link

codecov bot commented Sep 6, 2024

❌ 54 Tests Failed:

Tests completed Failed Passed Skipped
21670 54 21616 206
View the top 3 failed tests by shortest run time
tests.sentry.incidents.test_subscription_processor.ProcessUpdateTest test_enable_dynamic_alert_rule_and_fire
Stack Traces | 0.35s run time
#x1B[1m#x1B[.../testutils/helpers/features.py#x1B[0m:144: in wrapped
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1387: in patched
    with self.decoration_helper(patched,
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:137: in __enter__
    return next(self.gen)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1369: in decoration_helper
    arg = exit_stack.enter_context(patching)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:526: in enter_context
    result = _enter(cm)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1442: in __enter__
    self.target = self.getter()
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/pkgutil.py#x1B[0m:528: in resolve_name
    result = getattr(result, p)
#x1B[1m#x1B[31mE   AttributeError: type object 'SubscriptionProcessor' has no attribute 'seer_anomaly_detection_connection_pool'#x1B[0m
tests.sentry.incidents.test_subscription_processor.ProcessUpdateTest test_seer_call_bad_status
Stack Traces | 0.354s run time
#x1B[1m#x1B[.../testutils/helpers/features.py#x1B[0m:144: in wrapped
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1387: in patched
    with self.decoration_helper(patched,
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:137: in __enter__
    return next(self.gen)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1369: in decoration_helper
    arg = exit_stack.enter_context(patching)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:526: in enter_context
    result = _enter(cm)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1442: in __enter__
    self.target = self.getter()
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/pkgutil.py#x1B[0m:528: in resolve_name
    result = getattr(result, p)
#x1B[1m#x1B[31mE   AttributeError: type object 'SubscriptionProcessor' has no attribute 'seer_anomaly_detection_connection_pool'#x1B[0m
tests.sentry.incidents.test_subscription_processor.ProcessUpdateTest test_seer_call_timeout_error
Stack Traces | 0.355s run time
#x1B[1m#x1B[.../testutils/helpers/features.py#x1B[0m:144: in wrapped
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1387: in patched
    with self.decoration_helper(patched,
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:137: in __enter__
    return next(self.gen)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1369: in decoration_helper
    arg = exit_stack.enter_context(patching)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/contextlib.py#x1B[0m:526: in enter_context
    result = _enter(cm)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/unittest/mock.py#x1B[0m:1442: in __enter__
    self.target = self.getter()
#x1B[1m#x1B[.../hostedtoolcache/Python/3.12.3................../x64/lib/python3.12/pkgutil.py#x1B[0m:528: in resolve_name
    result = getattr(result, p)
#x1B[1m#x1B[31mE   AttributeError: type object 'SubscriptionProcessor' has no attribute 'seer_anomaly_detection_connection_pool'#x1B[0m

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@getsantry getsantry bot added the Stale label Oct 1, 2024
@getsantry
Copy link
Contributor

getsantry bot commented Oct 1, 2024

This pull request has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you add the label WIP, I will leave it alone unless WIP is removed ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot removed the Stale label Oct 2, 2024
@getsantry
Copy link
Contributor

getsantry bot commented Oct 24, 2024

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot added the Stale label Oct 24, 2024
@ceorourke ceorourke closed this Oct 24, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Nov 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Do Not Merge Don't merge Scope: Backend Automatically applied to PRs that change backend components Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants