Bug #9467: Net::HTTP does unzip zipped content

• Status: Closed
• Assignee: dknight (Amitav Mohanty)
• Target version: -
• ruby -v: ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]

[dknight (Amitav Mohanty)]

According to the Compression section of the documentation of Net::HTTP, it should automatically uncompress compressed responses. However, when I tried with a number of JSON requests with different servers, I am not getting automatic decompression.

require 'net/http'
uri = URI("http://yourhost.com")
http = Net::HTTP.new(uri.host, uri.port)
res = http.get(uri.request_uri)

You can test hitting http://vimeo.com/api/oembed.json?url=http://vimeo.com/69655841&width=150

[drbrain (Eric Hodel)]

• Tracker changed from Backport to Bug
• Project changed from Backport200 to Ruby master
• Category set to lib
• Status changed from Open to Feedback
• Priority changed from 5 to Normal

I can't reproduce. The URI you've given is correctly decompressed by Net::HTTP using ruby 2.0.0:

$ cat test.rb
require 'net/http'
require 'pp'

uri = URI("http://vimeo.com/api/oembed.json?url=http://vimeo.com/69655841&width=150")
http = Net::HTTP.new(uri.host, uri.port)
http.set_debug_output $stderr

res = http.get(uri.request_uri)

pp res.to_hash

p res.body[0, 30]

$ ruby20 -v test.rb 
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-darwin13.0.0]
opening connection to vimeo.com:80...
opened
<- "GET /api/oembed.json?url=http://vimeo.com/69655841&width=150 HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nConnection: close\r\nHost: vimeo.com\r\n\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Server: Apache\r\n"
-> "Access-Control-Allow-Origin: *\r\n"
-> "Access-Control-Allow-Headers: X-Requested-With\r\n"
-> "Expires: Thu, 30 Jan 2014 20:52:07 GMT\r\n"
-> "Last-Modified: Thu, 30 Jan 2014 20:12:02 GMT\r\n"
-> "Etag: \"215c14df5b7b774ff7b23cfb02b34da62c9ec022\"\r\n"
-> "X-UA-Compatible: IE=edge\r\n"
-> "X-DNS-Prefetch-Control: on\r\n"
-> "Vary: Accept-Encoding\r\n"
-> "Content-Encoding: gzip\r\n"
-> "Content-Type: application/json\r\n"
-> "Content-Length: 745\r\n"
-> "Accept-Ranges: bytes\r\n"
-> "Date: Thu, 30 Jan 2014 20:51:07 GMT\r\n"
-> "X-Varnish: 2058928850\r\n"
-> "Age: 0\r\n"
-> "Via: 1.1 varnish\r\n"
-> "X-Varnish-Cache: 0\r\n"
-> "Connection: close\r\n"
-> "X-VServer: 10.90.128.147\r\n"
-> "\r\n"
reading 745 bytes...
-> "\x1F\x8B\b\x00\x00\x00\x00\x00\x00\x03[snipped gzip response]"
read 745 bytes
Conn close
{"server"=>["Apache"],
 "access-control-allow-origin"=>["*"],
 "access-control-allow-headers"=>["X-Requested-With"],
 "expires"=>["Thu, 30 Jan 2014 20:52:07 GMT"],
 "last-modified"=>["Thu, 30 Jan 2014 20:12:02 GMT"],
 "etag"=>["\"215c14df5b7b774ff7b23cfb02b34da62c9ec022\""],
 "x-ua-compatible"=>["IE=edge"],
 "x-dns-prefetch-control"=>["on"],
 "vary"=>["Accept-Encoding"],
 "content-type"=>["application/json"],
 "content-length"=>["745"],
 "accept-ranges"=>["bytes"],
 "date"=>["Thu, 30 Jan 2014 20:51:07 GMT"],
 "x-varnish"=>["2058928850"],
 "age"=>["0"],
 "via"=>["1.1 varnish"],
 "x-varnish-cache"=>["0"],
 "connection"=>["close"],
 "x-vserver"=>["10.90.128.147"]}
"{\"type\":\"video\",\"version\":\"1.0"

[dknight (Amitav Mohanty)]

Hey

I found the issue to be occurring in rails console but not in IRB. I should have tested it before. The gem right_aws was causing the issue it seems.

Regards,

[mathieulbouchard (Mathieu L. Bouchard)]

Got the same thing and also this error when the block ends :
/usr/lib/ruby/1.9.1/net/http.rb:1032:in initialize': can't convert Net::ReadAdapter into String (TypeError) from /usr/lib/ruby/1.9.1/net/http.rb:1032:in new'
from /usr/lib/ruby/1.9.1/net/http.rb:1032:in block in get' from /usr/lib/ruby/1.9.1/net/http.rb:1322:in block (2 levels) in transport_request'
from /usr/lib/ruby/1.9.1/net/http.rb:2671:in reading_body' from /usr/lib/ruby/1.9.1/net/http.rb:1321:in block in transport_request'
from /usr/lib/ruby/1.9.1/net/http.rb:1316:in catch' from /usr/lib/ruby/1.9.1/net/http.rb:1316:in transport_request'
from /usr/lib/ruby/1.9.1/net/http.rb:1293:in request' from /usr/lib/ruby/1.9.1/net/http.rb:1286:in block in request'
from /usr/lib/ruby/1.9.1/net/http.rb:745:in start' from /usr/lib/ruby/1.9.1/net/http.rb:1284:in request'
from /usr/lib/ruby/1.9.1/net/http.rb:1026:in get' from téléphones_LEP.rb:11:in block in '

This is when using ruby 1.9.3p194 (2012-04-20 revision 35410) [i686-linux].

It works on ruby 1.8.7 (2012-02-08 patchlevel 358) [i686-linux]

[jeremyevans0 (Jeremy Evans)]

• Status changed from Feedback to Closed