Bug #9129

Regression in support for IPv6 literals in URIs with Net::HTTP

Added by Daniel DeLeo almost 2 years ago. Updated 8 months ago.

[ruby-core:58438]
Status:Closed
Priority:Normal
Assignee:Yui NARUSE
ruby -v:ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0] Backport:2.0.0: DONE, 2.1: DONE

Description

The proxy code in Net::HTTP constructs a URI string by concatenating the address with the port. If the address is IPv6, this is incorrect, the address should be surrounded by square brackets. My code is failing in the Net::HTTP#proxy_uri method. The error can be reproduced by:

ruby -r'net/http' -e 'Net::HTTP.new("::1", 80).proxy_uri'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/generic.rb:214:in initialize': the scheme http does not accept registry part: ::1:80 (or bad hostname?) (URI::InvalidURIError)
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/http.rb:84:in
initialize'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in new'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in
parse'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:747:in parse'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:994:in
URI'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1027:in proxy_uri'
from -e:1:in
'

Though I've not tried to reproduce on current trunk, the implementation of Net::HTTP#proxy_uri is unchanged from 2.0.0p0 and current trunk.

My application's tests pass if I monkey patch that method as follows:

  def proxy_uri # :nodoc:
    ipv6_safe_addr = address.to_s.include?(":") ? "[#{address}]" : address
    @proxy_uri ||= URI("http://#{ipv6_safe_addr}:#{port}").find_proxy
  end

Related issues

Related to Backport200 - Backport #10530: Backport r47079 to 2.0.0 Closed 11/20/2014

Associated revisions

Revision 47079
Added by Yui NARUSE about 1 year ago

  • lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead of parser to handle IPv6 address. [Bug #9129]

Revision 47079
Added by Yui NARUSE about 1 year ago

  • lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead of parser to handle IPv6 address. [Bug #9129]

Revision 48632
Added by Usaku NAKAMURA 9 months ago

merge revision(s) 47079: [Backport #10530]

* lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead
  of parser to handle IPv6 address. [Bug #9129]

Revision 49076
Added by Tomoyuki Chikanaga 8 months ago

merge revision(s) r47079: [Backport #9129] [Backport #10531]

* lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead
  of parser to handle IPv6 address. [Bug #9129]

History

#1 Updated by Eric Hodel almost 2 years ago

  • Assignee set to Yui NARUSE
  • Category set to lib
  • Status changed from Open to Assigned
  • Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: UNKNOWN

=begin
I can't find a regression here as no prior version has #proxy_uri:

$ ruby19 -v -rnet/http -e 'Net::HTTP.new("::1", 80).proxy_uri'
ruby 1.9.3p374 (2013-01-15 revision 38858) [x86_64-darwin12.2.1]
-e:1:in <main>': undefined methodproxy_uri' for # (NoMethodError)

IIRC net/http has always required square brackets for IPv6 addresses:

$ ruby -v -rnet/http -e 'Net::HTTP.new("[::1]", 80).proxy_uri'
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0]
$

This is consistent with URI:

$ ruby -ruri -e 'u = URI "http://[::1]"; u.host = "[::2]"; puts u'
http://[::2]
$ ruby -ruri -e 'u = URI "http://[::1]"; u.host = "::2"; puts u'
[…]/uri/generic.rb:605:in `check_host': bad component(expected host component): ::2 (URI::InvalidComponentError)

If you are using URI to figure out the host portion, use #host, not #hostname:

$ ruby -ruri -e 'u = URI "http://[::1]"; p u.host, u.hostname'
"[::1]"
"::1"

See also the URI::HTTP#host and #hostname documentation.

I have assigned this to naruse as he is Net::HTTP maintainer, maybe I am wrong.

=end

#2 Updated by Daniel DeLeo almost 2 years ago

Hi Eric,
Earlier ruby versions handled proxy support differently (I'm not very familiar with either the old or new code), so my repro won't show that. The issue is that HTTP#proxy_uri is in the path for all HTTP requests, at least the way I'm using Net::HTTP. Here is a section of stack trace from my application:

/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/generic.rb:214:in initialize'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/http.rb:84:in
initialize'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in new'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in
parse'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:747:in parse'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:994:in
URI'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1027:in proxy_uri'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1014:in
proxy?'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:868:in connect'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in
do_start'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in start'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1367:in
request'

If I understand correctly, the first argument to Net::HTTP.new should be an address that you can safely pass on to the socket layer code. For example, run this on Ruby 1.9.3:

ruby -rnet/http -e "Net::HTTP.new('::1', 80).get('/')" # works if you're running an HTTP server, Errno::ECONNREFUSED otherwise

Whereas this:

ruby -rnet/http -e "Net::HTTP.new('[::1]', 80).get('/')"

Gives you "getaddrinfo: nodename nor servname provided, or not known (SocketError)" because it's treating the whole thing with the brackets as the hostname/address.

It's true that in URIs you need to wrap an IPv6 literal in square brackets, which is why I believe this particular line of code is incorrect: https://github.com/ruby/ruby/blob/trunk/lib/net/http.rb#L1028

Thanks

#3 Updated by Daniel DeLeo almost 2 years ago

I will add also that this example gives a SocketError (as I would expect it to and is consistent with 1.9.3) on Ruby 2.0:

ruby -rnet/http -e "Net::HTTP.new('[::1]', 80).get('/')"
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError)
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in
open'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in block in connect'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in
timeout'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in connect'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in
do_start'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in start'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1367:in
request'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1126:in get'
from -e:1:in
'

#4 Updated by Yui NARUSE almost 2 years ago

  • Target version set to 2.1.0

#5 Updated by Hiroshi SHIBATA over 1 year ago

  • Target version changed from 2.1.0 to current: 2.2.0

#6 Updated by Yui NARUSE about 1 year ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

Applied in changeset r47079.


  • lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead of parser to handle IPv6 address. [Bug #9129]

#7 Updated by Tomoyuki Chikanaga 9 months ago

  • Backport changed from 1.9.3: DONTNEED, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: REQUIRED

according to #10531 it seems required to be backported into ruby_2_1.

#8 Updated by Usaku NAKAMURA 9 months ago

#9 Updated by Usaku NAKAMURA 9 months ago

  • Backport changed from 1.9.3: DONTNEED, 2.0.0: REQUIRED to 2.0.0: DONE, 2.1: REQUIRED

#10 Updated by Tomoyuki Chikanaga 8 months ago

  • Backport changed from 2.0.0: DONE, 2.1: REQUIRED to 2.0.0: DONE, 2.1: DONE

Backported at r49076.

Also available in: Atom PDF