Skip to content

Commit

Permalink
ref(server): Submit envelopes directly from processor (#2936)
Browse files Browse the repository at this point in the history
Removes circular roundtrips between the envelope processor service and
the now obsolete envelope manager by encoding and submitting envelopes
directly in the processor.

Previously, `EnvelopeManager` was the service responsible to supervise
the lifecycle of an envelope. This included dispatching it to the right
backend (Kafka or HTTP upstream) and logging outcomes in the event of an
error. However, in order to submit envelopes via HTTP, they
conditionally need to be compressed, which was sent back to the envelope
processor. The overall chain was therefore Processor -> manager ->
processor -> manager -> upstream -> manager. Additionally, the manager
had to maintain a long-running future to await the result of sending the
envelope upstream.

With this refactor, the envelope manager no longer exists. Instead, the
processor directly encodes the envelope and submits a message to the
correct backend. As a positive side-effect of this change, encoding of
large envelopes no longer blocks the main runtime. If submission fails,
either the request's `respond` method logs outcomes, or in case the
message is lost `ManagedEnvelope::drop` does so.

The public `SubmitEnvelope` message still exists and has moved to
`EnvelopeProcessor`, which allows other services to send envelopes.
  • Loading branch information
jan-auer committed Jan 12, 2024
1 parent 118cafc commit aa272ed
Show file tree
Hide file tree
Showing 10 changed files with 276 additions and 491 deletions.
357 changes: 0 additions & 357 deletions relay-server/src/actors/envelopes.rs

This file was deleted.

8 changes: 3 additions & 5 deletions relay-server/src/actors/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@
//! - [`ProjectCache`](project_cache::ProjectCache): A cache that serves queries for project
//! configurations. Its requests are debounced and batched based on a configured interval (100ms
//! by default). Also, missing projects are cached for some time.
//! - [`EnvelopeManager`](envelopes::EnvelopeManager) and
//! [`EnvelopeProcessor`](processor::EnvelopeProcessor): Handle a queue of envelopes, verify
//! their projects, execute PII stripping and finally send the envelope to the upstream. The
//! processor is spawned in multiple synchronous worker threads.
//! - [`EnvelopeProcessor`](processor::EnvelopeProcessor): A worker pool for CPU-intensive tasks.
//! Most importantly, this implements envelope processing to verify their projects, execute PII
//! stripping and finally send the envelope to the upstream.
//! - [`UpstreamRelay`](upstream::UpstreamRelay): Abstraction for communication with the upstream
//! (either another Relay or Sentry). It manages an internal client connector to throttle
//! requests and ensures this relay is authenticated before sending queries (e.g. project config
Expand All @@ -28,7 +27,6 @@
//! Controller::run(|| Server::start())
//! .expect("failed to start relay");
//! ```
pub mod envelopes;
pub mod global_config;
pub mod health_check;
pub mod outcome;
Expand Down
Loading

0 comments on commit aa272ed

Please sign in to comment.