Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: make it possible to disable Content-Type sniffing #9430

Closed
stanhu opened this issue Feb 15, 2024 · 1 comment · Fixed by #9431
Closed

storage: make it possible to disable Content-Type sniffing #9430

stanhu opened this issue Feb 15, 2024 · 1 comment · Fixed by #9431
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@stanhu
Copy link
Contributor

stanhu commented Feb 15, 2024

Is your feature request related to a problem? Please describe.

As described in google/go-cloud#3298 (comment), we want to disable automatic Content-Type detection when inserting an object to Google Cloud Storage (GCS).

To write a blob to GCS, the Go Cloud project calls httpStorageClient.OpenWriter, which calls this to insert the blob:

call := c.raw.Objects.Insert(params.bucket, rawObj).
Media(pr, mediaOpts...).
Projection("full").
Context(params.ctx).
Name(params.attrs.Name)

It appears that googleapis.MediaOptions has ForceEmptyContentType that can be used to disable this content sniffing: https://github.com/googleapis/google-api-go-client/blob/260fc1ef78c67de11cc5d654fcbe08edd8c01843/internal/gensupport/media.go#L152-L153

However, there's currently no way to set this value.

Describe the solution you'd like

Likely the easiest option would be to add an option to openWriterParams to enable ForceEmptyContentType.

@stanhu stanhu added the triage me I really want to be triaged. label Feb 15, 2024
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Feb 15, 2024
@stanhu
Copy link
Contributor Author

stanhu commented Feb 15, 2024

I've confirmed that this hack properly disables the sniffing, since googleapi.ContentType will set ForceEmptyContentType to be true if the value is blank (https://github.com/googleapis/google-api-go-client/blob/260fc1ef78c67de11cc5d654fcbe08edd8c01843/googleapi/googleapi.go#L230-L235):

diff --git a/storage/http_client.go b/storage/http_client.go
index a0f3c00a73..e3e0d761bb 100644
--- a/storage/http_client.go
+++ b/storage/http_client.go
@@ -885,7 +885,7 @@ func (c *httpStorageClient) OpenWriter(params *openWriterParams, opts ...storage
        mediaOpts := []googleapi.MediaOption{
                googleapi.ChunkSize(params.chunkSize),
        }
-       if c := attrs.ContentType; c != "" {
+       if c := attrs.ContentType; c != "" || true {
                mediaOpts = append(mediaOpts, googleapi.ContentType(c))
        }
        if params.chunkRetryDeadline != 0 {

stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 16, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 16, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 16, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 16, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
@BrennaEpp BrennaEpp added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Feb 16, 2024
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 20, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 20, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 21, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 21, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 22, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
stanhu added a commit to stanhu/google-cloud-go that referenced this issue Feb 22, 2024
As described in
google/go-cloud#3298 (comment),
we want to disable automatic `Content-Type` detection when
inserting an object to Google Cloud Storage (GCS).

Previously it wasn't possible to disable this auto-detection, even
though `googleapi.MediaOptions` provides a `ForceEmptyContentType`
option (https://github.com/googleapis/google-api-go-client/blob/v0.165.0/googleapi/googleapi.go#L283).

We enable this by adding a `Writer` option to set this value.

Closes googleapis#9430
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants