User-Agent HTTP header not being set on CONNECT requests
(This was originally reported as RubyGems issue https://github.com/rubygems/rubygems/issues/1012, contributors determined this is actually a bug in core Ruby.)¶
Per issue https://github.com/rubygems/rubygems/issues/825, RubyGems is supposed to be setting the HTTP User-Agent header, like:
RubyGems/2.2.2 x86_64-darwin-13 Ruby/2.1.0 (2013-12-25 patchlevel 0)
However, network tracing reveals that the User-Agent header is not being sent on HTTP CONNECT requests to the proxy server, which is needed for some companies to be able to white-list RubyGem traffic. Only bare minimal headers are present:
CONNECT api.rubygems.org:443 HTTP/1.1
Please set the HTTP User-Agent header on HTTP CONNECT requests.
Updated by usa (Usaku NAKAMURA) about 5 years ago
I found that the internet-draft (Dec.1995) mentioned about the User-Agent header for CONNECT, but the final RFC (RFC2817, May.2000) doesn't.
Thus, sending the User-Agent header is not official, so I think it's the problem of the proxy server which requires it.
Could you refute me with some concrete evidence?
Enumerating the implementations of proxy servers which need the User-Agent header, showing the specification document or others...
Updated by Feldhacker (Chris Feldhacker) about 5 years ago
In the (RFC2817, May.2000) doc you reference, section "5.2 Requesting a Tunnel with CONNECT" indicates that "Other HTTP mechanisms can be used normally with the CONNECT method..." and then goes on to provide an example where a "Proxy-Authorization" header is also sent to the proxy.
Unfortunately, the doc doesn't seem to elaborate on what "other HTTP mechanisms" means, nor does it provide what other examples might be beyond the Proxy-Authorization header...
It's hard to provide proxy server examples as many of the common enterprise-level "web gateways" are hidden behind support logins, but I did manage to find these:
(BlueCoat, McAfee, Symantec, Trend Micro, etc -- a good list is available in this chart: http://images.machspeed.bluecoat.com/EloquaImages/clients/BlueCoat/%7B814625d3-ff03-444b-b846-1b04817a14d5%7D_gartner-2014-mq-graph.jpg)