Error in curl::curl_fetch_memory(url, handle = handle): Unrecognized content encoding type.

[MarkEdmondson1234]

This was reported to start occurring when upgrading to R 4.0.2 at this issue cloudyr/bigQueryR#67

Full error message:

2020-08-22 17:44:30 -- Uploading local data.frame 2020-08-22 17:44:31 -- Upload job made... 
2020-08-22 17:44:36 -- Waiting for job: job_57rJOEQf4vFavmoccgYW-dAbPS4x - Job timer: 5.000396 secs 
Error in curl::curl_fetch_memory(url, handle = handle): Unrecognized content encoding type. libcurl understands identity content encodings. 
...retries...
Request failed before finding status code: Unrecognized content encoding type. libcurl understands identity content encodings.

It seems related to #179 and #160 but the content-encoding which is cited as the reason at those issues is on the response is a valid type, gzip:

-> GET /bigquery/v2/projects/XXX/jobs/job_hr_FlMBJtoHi0qdBEJzyZhxZklC- HTTP/1.1
-> Host: www.googleapis.com
-> User-Agent: googleAuthR/1.3.0 (gzip)
-> Accept: application/json, text/xml, application/xml, /
-> Accept-Encoding: gzip
-> Authorization: Bearer XXX
->
<- HTTP/1.1 200 OK
<- ETag: YsspnhsRtTEmCaY40rNEwg==
<- Content-Type: application/json; charset=UTF-8
<- Vary: Origin
<- Vary: X-Origin
<- Vary: Referer
<- Content-Encoding: gzip
<- Date: Mon, 24 Aug 2020 06:16:30 GMT
<- Server: ESF
<- Cache-Control: private
<- X-XSS-Protection: 0
<- X-Frame-Options: SAMEORIGIN
<- X-Content-Type-Options: nosniff
<- Transfer-Encoding: chunked

The user's sessionInfo:

R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-openmp/libopenblasp-r0.3.8.so

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] teamr_0.0.1 RCurl_1.98-1.2 purrr_0.3.4 stringr_1.4.0 bigQueryR_0.5.0 googleCloudStorageR_0.5.1
[7] googleAuthR_1.3.0 dplyr_1.0.1 httr_1.4.2

loaded via a namespace (and not attached):
[1] zip_2.1.0 Rcpp_1.0.5 pillar_1.4.6 compiler_4.0.2 bitops_1.0-6 tools_4.0.2 digest_0.6.25 jsonlite_1.7.0 memoise_1.1.0
[10] lifecycle_0.2.0 tibble_3.0.3 gargle_0.5.0 lattice_0.20-41 pkgconfig_2.0.3 rlang_0.4.7 Matrix_1.2-18 cli_2.0.2 rstudioapi_0.11
[19] curl_4.3 yaml_2.2.1 generics_0.0.2 vctrs_0.3.2 fs_1.5.0 askpass_1.1 hms_0.5.3 grid_4.0.2 tidyselect_1.1.0
[28] glue_1.4.1 R6_2.4.1 fansi_0.4.1 readr_1.3.1 magrittr_1.5 ellipsis_0.3.1 assertthat_0.2.1 stringi_1.4.6 openssl_1.4.2
[37] crayon_1.3.4

[jeroen]

What version of libcurl are you using? Can you include your curl::curl_version() ? It seems like it was built without gzip support.

[jay]

I agree. That error message should show all the built in content types and as you can see it does not show gzip. Likely the user or rcurl asked for gzip explicitly via CURLOPT_ACCEPT_ENCODING, which should not be done unless gzip support was built in. Rather than check the version info for what encodings are supported, the better way is CURLOPT_ACCEPT_ENCODING set to "".

To aid applications not having to bother about what specific algorithms this particular libcurl build supports, libcurl allows a zero-length string to be set ("") to ask for an Accept-Encoding: header to be used that contains all built-in supported encodings.

[jurrigerretsen]

No gzip seems to be the case:

$version
[1] "7.70.0"

$ssl_version
[1] "OpenSSL/1.1.1f"

$libz_version
[1] NA

$libssh_version
[1] "libssh2/1.8.0"

$libidn_version
[1] NA

$host
[1] "x86_64-pc-linux-gnu"

$protocols
[1] "dict" "file" "ftp" "ftps" "gopher" "http" "https" "imap" "imaps" "pop3" "pop3s" "rtsp" "scp" "sftp" "smb" "smbs" "smtp" "smtps"
[19] "telnet" "tftp"

$ipv6
[1] TRUE

$http2
[1] FALSE

$idn
[1] FALSE

[MarkEdmondson1234]

Ok thanks for tracking it down, where should be the fix? In the user's environment or is it something to fix in the R httr request?

[jeroen]

@jurrigerretsen I don't know how you got that version of libcurl, but it is not properly built. I recommend you remove any custom libcurl installations from your server, and install the official libcurl4-openssl-dev from ubuntu.

[github-actions]

This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. If the
problem still exists in the latest version, feel free to raise it again
in a new issue. Please remember to keep the description terse and include
a minimal example to help us look into it.