Bug #17933
closed`Net::HTTP#write_timeout` doesn't work with `body_stream`
Description
While testing Net::HTTP#write_timeout
using the server example from the Feature Issue, I've noticed that Faraday multipart requests (with a file parameter) didn't trigger WriteTimeout
. The root cause is that Net::HTTPGenericRequest#send_request_with_body_stream is not using Net::BufferedIO
(the class that implements the write timeout).
The following patch fixes the issue, but looking at the comments, it's unclear if it will break some existing functionality. Albeit all the tests pass.
--- a/lib/net/http/generic_request.rb
+++ b/lib/net/http/generic_request.rb
@@ -204,7 +204,7 @@ def send_request_with_body_stream(sock, ver, path, f)
else
# copy_stream can sendfile() to sock.io unless we use SSL.
# If sock.io is an SSLSocket, copy_stream will hit SSL_write()
- IO.copy_stream(f, sock.io)
+ IO.copy_stream(f, sock)
end
end
➜ ruby git:(fix-net-http-write-timeout-body-stream) ✗ ../mspec/bin/mspec run library/net/
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin20]
[- | ==================100%================== | 00:00:00] 0F 0E
Finished in 2.029623 seconds
187 files, 876 examples, 1214 expectations, 0 failures, 0 errors, 0 tagged
Any thoughts on this? It looks like Net::HTTP#write_timeout
is partially implemented, and we should ensure that it either works on all use cases or an error is raised when it's used in a non-supported use case.
I'm more than happy to help with the implementation. If this patch is the correct approach, I can also create new unit tests to assert the proper behaviour.
Updated by byroot (Jean Boussier) over 3 years ago
it's unclear if it will break some existing functionality
Well, copy_stream
can only use its optimizations with real IO objects, so functionally this would still work, but the performance would be heavily degraded. But this optimization is already disabled for SSL sockets, so maybe it doesn't matter as much in practice?
Updated by byroot (Jean Boussier) over 3 years ago
Previous discussion where write_timeout
was added and it was noted that copy_stream
didn't have a timeout: https://bugs.ruby-lang.org/issues/13396
There was talks about the possibility to add it, which everybody agreed it would be best.
Updated by miguelfteixeira (Miguel Teixeira) over 3 years ago
byroot (Jean Boussier) wrote in #note-2:
Previous discussion where
write_timeout
was added and it was noted thatcopy_stream
didn't have a timeout: https://bugs.ruby-lang.org/issues/13396There was talks about the possibility to add it, which everybody agreed it would be best.
Are you talking about moving the write_timeout
implementation from Net::BufferedIO
to IO.copy_stream
? I was reading the comments and got the idea that it would be another layer of "security", not a replacement.
I may be biased but it looks like the current implementation of Net::HTTP
creates a Net::BufferedIO
instance so it would be safer if we always use it when writing to a stream to make it future proof.
Updated by byroot (Jean Boussier) over 3 years ago
Are you talking about moving the write_timeout implementation from Net::BufferedIO to IO.copy_stream?
No. IO.copy_stream
is a low level API that leverage sendfile(2)
and other similar APIs so that the data transfer happens in the Kernel rather than in user space. For large uploads the performance difference can be quite massive.
However this low-level API has no timeout support. The discussion was about adding timeout support to that so that Net::HTTP could pass a timeout argument.
Updated by miguelfteixeira (Miguel Teixeira) over 3 years ago
I've created a pull request in the Ruby Net::HTTP Github project with the proposed solution: https://github.com/ruby/net-http/pull/27
Updated by jeremyevans0 (Jeremy Evans) over 3 years ago
- Status changed from Open to Closed
Updated by byroot (Jean Boussier) about 3 years ago
- Related to Feature #18228: Add a `timeout` option to `IO.copy_stream` added