Bug #17933: `Net::HTTP#write_timeout` doesn't work with `body_stream`

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin20]
• Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN

[miguelfteixeira (Miguel Teixeira)]

While testing Net::HTTP#write_timeout using the server example from the Feature Issue, I've noticed that Faraday multipart requests (with a file parameter) didn't trigger WriteTimeout. The root cause is that Net::HTTPGenericRequest#send_request_with_body_stream is not using Net::BufferedIO (the class that implements the write timeout).

The following patch fixes the issue, but looking at the comments, it's unclear if it will break some existing functionality. Albeit all the tests pass.

--- a/lib/net/http/generic_request.rb
+++ b/lib/net/http/generic_request.rb
@@ -204,7 +204,7 @@ def send_request_with_body_stream(sock, ver, path, f)
     else
       # copy_stream can sendfile() to sock.io unless we use SSL.
       # If sock.io is an SSLSocket, copy_stream will hit SSL_write()
-      IO.copy_stream(f, sock.io)
+      IO.copy_stream(f, sock)
     end
   end

➜  ruby git:(fix-net-http-write-timeout-body-stream) ✗ ../mspec/bin/mspec run library/net/
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin20]
[- | ==================100%================== | 00:00:00]      0F      0E

Finished in 2.029623 seconds

187 files, 876 examples, 1214 expectations, 0 failures, 0 errors, 0 tagged

Any thoughts on this? It looks like Net::HTTP#write_timeout is partially implemented, and we should ensure that it either works on all use cases or an error is raised when it's used in a non-supported use case.

I'm more than happy to help with the implementation. If this patch is the correct approach, I can also create new unit tests to assert the proper behaviour.

[byroot (Jean Boussier)]

it's unclear if it will break some existing functionality

Well, copy_stream can only use its optimizations with real IO objects, so functionally this would still work, but the performance would be heavily degraded. But this optimization is already disabled for SSL sockets, so maybe it doesn't matter as much in practice?

[byroot (Jean Boussier)]

Previous discussion where write_timeout was added and it was noted that copy_stream didn't have a timeout: https://bugs.ruby-lang.org/issues/13396

There was talks about the possibility to add it, which everybody agreed it would be best.

[miguelfteixeira (Miguel Teixeira)]

byroot (Jean Boussier) wrote in #note-2:

Previous discussion where write_timeout was added and it was noted that copy_stream didn't have a timeout: https://bugs.ruby-lang.org/issues/13396

There was talks about the possibility to add it, which everybody agreed it would be best.

Are you talking about moving the write_timeout implementation from Net::BufferedIO to IO.copy_stream? I was reading the comments and got the idea that it would be another layer of "security", not a replacement.

I may be biased but it looks like the current implementation of Net::HTTP creates a Net::BufferedIO instance so it would be safer if we always use it when writing to a stream to make it future proof.

[byroot (Jean Boussier)]

Are you talking about moving the write_timeout implementation from Net::BufferedIO to IO.copy_stream?

No. IO.copy_stream is a low level API that leverage sendfile(2) and other similar APIs so that the data transfer happens in the Kernel rather than in user space. For large uploads the performance difference can be quite massive.

However this low-level API has no timeout support. The discussion was about adding timeout support to that so that Net::HTTP could pass a timeout argument.

[miguelfteixeira (Miguel Teixeira)]

I've created a pull request in the Ruby Net::HTTP Github project with the proposed solution: https://github.com/ruby/net-http/pull/27

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

[byroot (Jean Boussier)]

• Related to Feature #18228: Add a `timeout` option to `IO.copy_stream` added