Bug #15394: Ruby adds unexpected HTTP header value when using symbol key

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN

[shia (Sangyong Sim)]

Problem

Because of this, Ruby will set Host header twice if user uses symbol key.
In other words, host header should be set when host header is not given.
See code: https://github.com/ruby/ruby/blob/0777262b1b08722ee99d73fb8c9703aa9d1a0e98/lib/net/http/generic_request.rb#L49

I guess passing symbol key is not considered...?

Handle symbol key as different with string key from: https://github.com/ruby/ruby/commit/acce0b7ec4f5b6682b94e458675020263068d306
Default host header logic introduced: https://github.com/ruby/ruby/commit/159fa373f8e913a5464ab88c308e4375c946af8b

Therefore, this impacts to > 2.2

Reproduce code

require "net/http"
uri = URI("https://www.google.com")
req = Net::HTTP::Get.new(uri, host: "google")

req.each_capitalized do |k,v|
  puts "#{k}: #{v}\r\n"
end

# => Actual behavior
# Host: google
# Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
# Accept: */*
# User-Agent: Ruby
# Host: www.google.com

# => Expected behavior
# Host: google
# Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
# Accept: */*
# User-Agent: Ruby

Use case

We use HTTP host header to proxy request for internal api call.
This issue was found when debugging the problem that a request was not proxied properly.
The case, setting host header and using it for routing, is increasing by the needs of microservices.
I hope this fixed to prevent introducing bugs by accidentally.

Proposal

Preserve HTTP header key as string.
I think it's better to persist internal state as same as final output in this case.

write_header implementation: https://github.com/ruby/ruby/blob/0777262b1b08722ee99d73fb8c9703aa9d1a0e98/lib/net/http/generic_request.rb#L324-L336

This may affect to set 'Accept', 'User-Agent', 'Host' headers with symbol key, but the other cases not:

• These headers will be set when there is no same key. (net/http)
• Most of header values could be added or deleted, there are few cases which set value if not exists.

I attach sample patch for this.

Thanks for reading.

[naruse (Yui NARUSE)]

• Status changed from Open to Closed

Applied in changeset trunk|r66319.

---

Preserve HTTP header key as string [Bug #15394]

to prevent send Host header twice accidentally.

From: Sangyong Sim [e-mail]

[naruse (Yui NARUSE)]

Sounds reasonable.
I merged your patch with replacing to_s by to_str because to_s is too loose for the original code
which accepts only key which receives downcase method.
thanks!

[shia (Sangyong Sim)]

Thanks!!