Project

General

Profile

Bug #11268

Zlib::DataError: incorrect header check

Added by stevenspiel (Steven Spiel) almost 4 years ago. Updated over 3 years ago.

Status:
Assigned
Priority:
Normal
Target version:
-
[ruby-core:69609]

Description

I'm having an issue opening a webpage with open-uri

$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-darwin14]
$ irb

require 'open-air'
=> true
open('https://www.hoveround.com')
Zlib::DataError: incorrect header check
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:377:in inflate'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:377:in
block in inflate_adapter'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/protocol.rb:411:in call_block'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/protocol.rb:402:in
<<'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/protocol.rb:102:in read'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:399:in
read'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:317:in read_chunked'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:281:in
block in read_body_0'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:260:in inflater'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:279:in
read_body_0'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:201:in read_body'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:333:in
block (2 levels) in open_http'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1421:in block (2 levels) in transport_request'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http/response.rb:162:in
reading_body'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1420:in block in transport_request'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in
catch'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in transport_request'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:1384:in
request'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:324:in block in open_http'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/net/http.rb:853:in
start'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:318:in open_http'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:736:in
buffer_open'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:211:in block in open_loop'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:209:in
catch'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:209:in open_loop'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:150:in
open_uri'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:716:in open'
from /Users/stevenspiel/.rbenv/versions/2.2.2/lib/ruby/2.2.0/open-uri.rb:34:in
open'
from (irb):2
from /Users/stevenspiel/.rbenv/versions/2.2.2/bin/irb:11:in `'


Files

net_http.patch (6.01 KB) net_http.patch nkmrya (Yasuhiro Nakamura), 09/08/2015 03:32 PM

History

Updated by Hanmac (Hans Mackowiak) almost 4 years ago

i can reproduce it with "ruby 2.3.0dev (2015-06-15 trunk 50908) [x86_64-linux]"

but i did use require 'open-uri' instead of require 'open-air'

#2

Updated by nkmrya (Yasuhiro Nakamura) over 3 years ago

I check the response from https://www.hoveround.com .

$ openssl s_client -connect www.hoveround.com:443

Request header

GET / HTTP/1.1
Host: www.hoveround.com
Accept-Encoding: gzip,deflate

Response

HTTP/1.1 200 OK
Cache-Control: private, no-store, must-revalidate
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
Content-Encoding: deflate
Server: Microsoft-IIS/8.5
X-Frame-Options: SAMEORIGIN
Set-Cookie: CMSPreferredCulture=en-US; expires=Sat, 20-Aug-2016 14:26:58 GMT; path=/; HttpOnly
Set-Cookie: ASP.NET_SessionId=ksufrr40usfo3ecjvxmudpus; path=/; HttpOnly
Set-Cookie: CurrentContact=788a7c31-f770-4eba-bd0f-f322b42b3f71; expires=Thu, 20-Aug-2065 14:26:58 GMT; path=/; HttpOnly
X-UA-Compatible: IE=Edge
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Strict-Transport-Security: max-age=10886400
Date: Thu, 20 Aug 2015 14:26:59 GMT

5361
... encoded content

encoded content

edbde972db48b230fa...

That server encodes the content by Deflate. Deflate has not header. (see RFC 1951) Encoded content is not satisfy Gzib header and Zlib header, so Zlib::Inflate can not inflate it.

I think, the solution is to remove deflate from request header or to make new method for inflate raw 'Deflate'. Which solution is better?

#3

Updated by nagachika (Tomoyuki Chikanaga) over 3 years ago

  • Status changed from Open to Assigned
  • Assignee set to akr (Akira Tanaka)

Yasuhiro, thank you for your investigation!
So is it server side issue?

Anyway, you can specify request header via open-uri.

require "open-uri"
open('https://www.hoveround.com', "Accept-Encoding" => "plain") do |f| puts f.read end

This works fine on my environments.

#4

Updated by nkmrya (Yasuhiro Nakamura) over 3 years ago

I think it is issue of ruby.

Test case of test/net/http/test_httpresponse.rb is wrong.
That test case say "x\x9C\xCBH\xCD\xC9\xC9\a\x00\x06,\x02\x15" is raw Deflate format (at line 85) but actually that hex is Zlib format.
A case of deflated 'hello' is "\xCBH\xCD\xC9\xC9\a\x00".
I think ruby can not inflate a data which is compressed by raw Deflate format.
So ruby can not inflate the response of https://www.hoveround.com .
I examined these problems by using the php.

I am writing a path for problem of "open-uri".

#5

Updated by nkmrya (Yasuhiro Nakamura) over 3 years ago

This patch has passed tests.

Also available in: Atom PDF