Bug #16672

Updated by jmreid (Justin Reid) over 1 year ago

When using net/http to make a request to a resource, the default request headers are the following (when you have ZLIB available): 
 `"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"], "accept"=>["*/*"], "user-agent"=>["Ruby"]` 

 This means that a resource will return a gzipped response if it can provide it. Take this URL for example: 

 This is a JS file that has a `content-length` of `2733` when gzipped and `9995` when inflated: 

 curl --silent GET "" -H "accept-encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3" | wc -c 

 curl --silent GET "" | wc -c 

 When making a simple request for this asset using net/http: 
 uri = URI('') 
 res = Net::HTTP.get_response(uri) 

 Ruby will ( 
 - Delete the `content-encoding` header 
 - inflate the body 
 - return the inflated body 

 The issue here is that Ruby also leaves the `content-length` header set to the original request's value: 
 require 'net/http' 

 uri = URI('') 
 res = Net::HTTP.get_response(uri) 

 puts "Fetching:" 
 puts "Body size using String#bytesize: #{res.body.to_s.bytesize}" 
 puts "Content-Length response header: #{res.content_length}" 

 Results in: 
 Body size using String#bytesize: 9995 
 Content-Length response header: 2733 

 This means that an incorrect `content-length` header is passed back when net/http makes requests for gzip objects and inflates them.  

 This issue was noticed when Rack changed their behaviour in how they compute content-length. They used to compute the content-length for each body, but that changed in 2.0.8: 

 Using `Rack::ContentLength` is now the method they prefer if you need to compute the content-length. However, `Rack::ContentLength` will not try to re-compute the value if that header already exists: 

 Should Ruby: 
 - Do a `self.delete 'content-length'` in the inflater? 
 - Compute the `content-length` itself and update the header? (Hacky example: