Feature #20567: Net::HTTPResponse#detect_encoding should default to UTF-8 for application/javascript

• Status: Open
• Assignee: -
• Target version: -

[postmodern (Hal Brodigan)]

It is fairly common now for JavaScript to include UTF-8 characters. If a misconfigured web server returns an application/javascript response for a JavaScript file, but without charset=utf-8 in the Content-Type, Net::HTTPResponse will return a response body with encoding of ASCII-8BIT. This can cause lots of subtle String parsing/encoding issues when the response body is parsed or transformed. Net::HTTPResponse#detect_encoding should default the response encoding to UTF-8 if the Content-Type includes application/javascript.

RFC 4329 also seems to encourage defaulting to UTF-8.