ref(server): Use multer directly #3978
Conversation
| // Set the single-attachment limit that applies only for raw minidumps. Multipart bypasses the | ||
| // limited body and applies its own limits. | ||
| post(handle).route_layer(DefaultBodyLimit::max(config.max_attachment_size())) |
There was a problem hiding this comment.
This might be surprising behavior. DefaultBodyLimit applies by default for all extractors that axum defines, since they use with_limited_body() internally. Among others, this applies to the Bytes extractor, which is used in one of the branches of handle.
In our case, we want two different limits:
- For raw minidumps, use the single attachment limit
- For multipart, use the multiple attachments limit.
Our multipart extractor does that automatically, as it has access to our config, and that also bypasses with_limited_body. Hence, the limit here applies to just one of the two branches.
I've been looking into ways to layer the limit directly onto the Bytes extractor, but haven't found a good way to do so.
| headers={ | ||
| "content-encoding": "gzip", | ||
| "content-length": str(size), | ||
| "content-type": "application/octet-stream", |
There was a problem hiding this comment.
This is an important fix. Before, we didn't set this content type, which had the following effect:
- The minidump endpoint did not detect this as a standalone minidump
- It went into multipart parsing, so the parser tried to find a boundary
- There's no boundary, nor a multipart payload, so the endpoint fails immediately.
This was the 400 status code we received, which had nothing to do with the zipbomp and content sizes. By adding the content type, we force the standalone minidump branch, and at least test that part.
* master: (27 commits) build: Update dialoguer and hostname (#4009) build: Update opentelemetry-proto to 0.7.0 (#4000) build: Update lru to 0.12.4 (#4008) build: Update cookie to 0.18.1 (#4007) feat(spans): Extract standalone CLS span metrics and performance score (#3988) build: Update cadence to 1.4.0 and statsdproxy to 0.2.0 (#4005) build: Update maxminddb to 0.24.0 (#4003) build: Update multer to 3.1.0 (#4002) build: Update regex and aho-corasick (#4001) build: Update sentry-kafka-schemas to 1.0.107 (#3999) build: Update dev-dependencies (#3998) build: Update itertools to 0.13.0 (#3993) build: Update brotli, zstd, flate2 (#3996) build: Update rdkafka to 0.36.2 (#3995) build: Update tikv-jemallocator to 0.6.0 (#3994) build: Update minidump to 0.22.0 (#3992) build: Update bindgen to 0.70.1 (#3991) build: Update chrono to 0.4.38 (#3990) feat(spans): initial MongoDB description scrubbing support (#3912) fix(spooler): Reduce number of disk reads (#3983) ...
jan-auer
force-pushed
the
ref/direct-multer
branch
from
30905ea
to
c71db88
Compare
Switches to direct use of
multerinstead ofaxum's wrapper. Thisallows us to use constraints to configure the maximum body size and
maximum field size directly on the
Multipartinstance instead ofhandling this manually.
In order to keep using the multipart type as an extractor, we define
Remotewhich is a shallow transparent wrapper around any remote typewe would like to implement
FromRequest,FromRequestParts, orIntoResponsefor. This way, we can use theMultipartandmulter::Errortypes directly in our signatures. Usage of this type isdocumented on the type.
Fixes #2218
See also the issue in axum
#skip-changelog