Send attachment data inline when possible (#3654)

For `invidual_attachments`, we can now send the `data` inline if it is
small enough. This avoids having to store temporary `attachment_chunks`
in the processing redis store.

Right now, every "event" attachment is sent with outlined chunks, even
for things like logfiles, even though only minidumps and other
processing-relevant attachments need to be treated that way.
It might be possible to also inline logfiles with more logic. That way,
such attachments could be saved as `EventAttachment` directly without
putting load on the attachments store. Doing that however would create
problems when such events are discarded within the rest of the pipeline.

Support for consuming the `data` property is being added in
getsentry/sentry#71522 to the Sentry codebase.

CHANGELOG.md

@@ -22,6 +22,7 @@
 - Forward `received_at` timestamp for buckets sent to Kafka. ([#3561](https://github.com/getsentry/relay/pull/3561))
 - Limit metric name to 150 characters. ([#3628](https://github.com/getsentry/relay/pull/3628))
 - Add validation of Kafka topics on startup. ([#3543](https://github.com/getsentry/relay/pull/3543))
+- Send `attachment` data inline when possible. ([#3654](https://github.com/getsentry/relay/pull/3654))
 
 ## 24.5.0
 

relay-dynamic-config/src/global.rs

@@ -198,6 +198,17 @@ pub struct Options {
  )]
  pub feedback_ingest_topic_rollout_rate: f32,
 
+ /// Rollout rate for sending attachment chunks inline.
+ ///
+ /// Rate needs to be between `0.0` and `1.0`.
+ /// If set to `1.0` all eligible attachment chunks will be sent as part of the attachment message.
+ #[serde(
+ rename = "relay.inline-attachments.rollout-rate",
+ deserialize_with = "default_on_error",
+ skip_serializing_if = "is_default"
+ )]
+ pub inline_attachments_rollout_rate: f32,
+
  /// Overall sampling of span extraction.
  ///
  /// This number represents the fraction of transactions for which