Feature #21773: Support for setting encoding when a block is passed to `Net::HTTPResponse.read_body`

• Status: Open
• Assignee: -
• Target version: -

[rosa (Rosa Gutierrez )]

Hi everyone,

This feature request could be considered a follow-up to https://bugs.ruby-lang.org/issues/2567. Opting in to automatic detection of the right encoding for the body when passing a block to read_body results in an error as follows:

require "net/http"

uri = URI.parse("https://rosa.codes")

http = Net::HTTP.new(uri.host, uri.port).tap { it.use_ssl = true }
req = Net::HTTP::Get.new(uri)

http.request(req) do |response|
  response.body_encoding = true

  body = +""
  response.read_body do |str|
    body << str
  end
end

# => lib/net/http/response.rb:380:in 'Net::HTTPResponse#read_body': undefined method 'force_encoding' for an instance of Net::ReadAdapter (NoMethodError)

It's expected this doesn't work because when passing a block, the body is not kept anywhere in the Net::HTTPResponse object, each segment read is just passed to the block via the wrapping Net::ReadAdapter.

In the case I was handling, I required both to read the body using a block and also set the encoding in the final body. I didn't want to duplicate all the code to do the encoding detection, as it's not trivial. I preferred to rely on the Net::HTTP implementation of it. I ended up doing something different, along these lines:

require "net/http"
require "active_support"

class Body
  attr_reader :content

  delegate_missing_to :content

  def initialize(&block)
    @block = block
    @content = +""
  end

  def <<(str)
    @content << str
    @block.call(str) if @block
  end
end

uri = URI.parse("https://rosa.codes")

http = Net::HTTP.new(uri.host, uri.port).tap { it.use_ssl = true }
req = Net::HTTP::Get.new(uri)

http.request(req) do |response|
  response.body_encoding = true

  dest = Body.new
  response.read_body(dest)
end

But this got me wondering whether:

• The error that happens when calling force_encoding on an instance of Net::ReadAdapter should be handled more gracefully or in a more informative way for the user.
• This could be better supported by Net::HTTP. All the pieces are there; there just isn't an easy way to put them together. There's a way to detect the encoding, there's a way to set a body directly in the response, via Net::HTTPResponse#body=, but there's no way to trigger the encoding detection and set it in the body besides the first time the body is read.

Thanks for the consideration!

[mame (Yusuke Endoh)]

• Related to Bug #21786: Net::HTTP::Response#read_body crashes on nil body when response_body_encoding is a String added