Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Process delayed alert conditions in batches of 10,000 #75302

Merged
merged 20 commits into from
Jul 31, 2024
Merged

Process delayed alert conditions in batches of 10,000 #75302

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
79ba7d0
WIP
saponifi3d Jul 30, 2024
1ed44ad
Add the ability to batch the apply delayed function.
saponifi3d Jul 30, 2024
70cbbb4
WIP
saponifi3d Jul 31, 2024
5225441
move mock redis buffer into the create event test case, and provide p…
saponifi3d Jul 31, 2024
8b20402
Cleanup the code, add some test cases around single / batch execution
saponifi3d Jul 31, 2024
7aafed8
raise a not implemented error
saponifi3d Jul 31, 2024
cdf288f
add a docstring tot he process grops, move logic for fetching teh rul…
saponifi3d Jul 31, 2024
d39202e
i am the walrus
saponifi3d Jul 31, 2024
9af164b
fix comment
saponifi3d Jul 31, 2024
c94c18e
-1 line changed
saponifi3d Jul 31, 2024
41fbcb7
add hmset and use bulk updating to redis
saponifi3d Jul 31, 2024
c1a7982
add tests for the all the new buffer code
saponifi3d Jul 31, 2024
ba6b28d
reduce diff
saponifi3d Jul 31, 2024
b767878
use options automator to store the batch size
saponifi3d Jul 31, 2024
267d71d
move from a module vairable to getting it from the options automator.…
saponifi3d Jul 31, 2024
c29c5e3
make the comment _better_
saponifi3d Jul 31, 2024
babca17
make it modifiable
saponifi3d Jul 31, 2024
1cd92fe
update cleanup_redis_buffers to remove batched keys as well
saponifi3d Jul 31, 2024
834445d
add a note about hmset v hset
saponifi3d Jul 31, 2024
4bac969
remove all these extra mocks
saponifi3d Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/sentry/options/defaults.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2636,3 +2636,7 @@
default=1,
flags=FLAG_AUTOMATOR_MODIFIABLE,
)
register(
"delayed_processing.batch_size",
default=10000,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You just need to add flags=FLAG_AUTOMATOR_MODIFIABLE and then you'll be good to go

4 changes: 2 additions & 2 deletions src/sentry/rules/processing/delayed_processing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import sentry_sdk
from django.db.models import OuterRef, Subquery

from sentry import buffer, nodestore
from sentry import buffer, nodestore, options
from sentry.buffer.redis import BufferHookEvent, redis_buffer_registry
from sentry.db import models
from sentry.eventstore.models import Event, GroupEvent
Expand Down Expand Up @@ -47,7 +47,7 @@
logger = logging.getLogger("sentry.rules.delayed_processing")
EVENT_LIMIT = 100
COMPARISON_INTERVALS_VALUES = {k: v[1] for k, v in COMPARISON_INTERVALS.items()}
CHUNK_BATCH_SIZE = 10000
CHUNK_BATCH_SIZE = options.get("delayed_processing.batch_size")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't set this as a module variable - it'll never update (and might cause problems on load). We just need to perform the check in function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heh, ty, i was just about to ask to check if did that right. thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, just updated again, anything else i'm missing with the options? (I was looking at the docs here: https://develop.sentry.dev/backend/options/ to add it). ps, thanks for the help!



class UniqueConditionQuery(NamedTuple):
Expand Down
Loading