Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref: Make eventstore.processing use LazyServiceWrapper #51870

Merged
merged 1 commit into from
Jun 29, 2023

Conversation

untitaker
Copy link
Member

@untitaker untitaker commented Jun 29, 2023

LazyServiceWrapper is a horrible old beast but it has a few sanity
checks that a simple import_string lacks. For starters, whether the
imported string even implements the relevant service baseclass.

Also LazyServiceWrapper supports typing now (which is the trigger
for doing this), while import_string would require one additional type
hint. That's easy to add but I don't particularly care for having N
ways to do services in Sentry.

LazyServiceWrapper is a horrible old beast but it has a few sanity
checks that a simple `import_string` lacks. For starters, whether the
imported string even implements the relevant service baseclass.

Also LazyServiceWrapper supports typing now, while import_string would
require one additional type hint. That's easy to add but I don't
particularly care for having N ways to do services in Sentry.
@untitaker untitaker requested review from beezz and a team June 29, 2023 13:08
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 29, 2023
@untitaker untitaker requested a review from a team June 29, 2023 13:11
@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #51870 (05bcf2e) into master (d8ca28d) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #51870   +/-   ##
=======================================
  Coverage    81.32    81.32           
=======================================
  Files        4901     4901           
  Lines      205867   205870    +3     
  Branches    11048    11048           
=======================================
+ Hits       167414   167419    +5     
+ Misses      38208    38206    -2     
  Partials      245      245           
Impacted Files Coverage Δ
src/sentry/eventstore/processing/__init__.py 100.00% <100.00%> (ø)
src/sentry/eventstore/processing/base.py 93.93% <100.00%> (+0.18%) ⬆️
src/sentry/eventstore/processing/default.py 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

Copy link
Member

@asottile-sentry asottile-sentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@untitaker untitaker merged commit b2634bf into master Jun 29, 2023
@untitaker untitaker deleted the ref/eventstore-processing-servicewrapper branch June 29, 2023 14:04
@asottile-sentry
Copy link
Member

appears to be breaking processing:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/src/sentry/src/sentry/__main__.py", line 3, in <module>
    main()
  File "/usr/src/sentry/src/sentry/runner/__init__.py", line 193, in main
    func(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 69, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 29, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/src/sentry/src/sentry/runner/commands/run.py", line 726, in basic_consumer
    run_processor_with_signals(processor)
  File "/usr/src/sentry/src/sentry/utils/kafka.py", line 13, in run_processor_with_signals
    processor.run()
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/processor.py", line 288, in run
    self._run_once()
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/processor.py", line 381, in _run_once
    self.__processing_strategy.submit(message)
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/guard.py", line 82, in submit
    self.__inner_strategy.submit(message)
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/run_task.py", line 52, in submit
    self.__next_step.submit(Message(value))
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/guard.py", line 34, in submit
    self.__next_step.submit(message)
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/guard.py", line 82, in submit
    self.__inner_strategy.submit(message)
  File "/usr/local/lib/python3.8/site-packages/arroyo/processing/strategies/run_task.py", line 50, in submit
    result = self.__function(cast(Message[TStrategyPayload], message))
  File "/usr/src/sentry/src/sentry/ingest/consumer_v2/ingest.py", line 54, in process_ingest_message
    process_event(message, project)
  File "/usr/src/sentry/src/sentry/ingest/ingest_consumer.py", line 40, in inner
    return f(*args, **kwargs)
  File "/usr/src/sentry/src/sentry/utils/metrics.py", line 199, in inner
    return f(*args, **kwargs)
  File "/usr/src/sentry/src/sentry/ingest/ingest_consumer.py", line 127, in process_event
    cache_key = event_processing_store.store(data)
  File "/usr/src/sentry/src/sentry/utils/services.py", line 103, in __getattr__
    self._setup()
  File "/usr/src/sentry/src/sentry/utils/services.py", line 119, in _setup
    assert issubclass(backend, Service)
TypeError: issubclass() arg 1 must be a class

@asottile-sentry asottile-sentry added the Trigger: Revert Add to a merged PR to revert it (skips CI) label Jun 29, 2023
@getsentry-bot
Copy link
Contributor

PR reverted: 81dcc62

getsentry-bot added a commit that referenced this pull request Jun 29, 2023
This reverts commit b2634bf.

Co-authored-by: asottile-sentry <103459774+asottile-sentry@users.noreply.github.com>
untitaker added a commit that referenced this pull request Jun 30, 2023
Redis event processing store is currently unused, meaning we can change
it to not use a cache backend and move forward with Ted's original
vision of getting rid of the cache backend.

getsentry already uses a "transitional" backend that does not use the
cache layer, and now that the redis backend is identical to that one we
can move over to it.

Previously I broke sentry by deploying
#51870 (comment),
where I forgot that getsentry had its own processingstore backend. I
could fix the backend but I can also just delete it.
untitaker added a commit that referenced this pull request Jul 6, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Jul 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components Trigger: Revert Add to a merged PR to revert it (skips CI)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants