Feature #855: HTTP/1.1 fixes and other enhancements to webrick

• Status: Closed
• Assignee: normalperson (Eric Wong)
• Target version: 2.6

[candlerb (Brian Candler)]

=begin
I raised the following issues on ruby-core:

1. When returning an open IO object (without Content-Length or chunking), Webrick fails to close the HTTP/1.1 connection, and hence the client waits forever for the end of the data.
   http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18454
   http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18565

2. Webrick makes it very difficult to send a '100 continue' response when a HTTP/1.1 client requests one, and yet the RFC2616 says it must do so.
   http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18459

3. It would be convenient to be able to stream not just real IO objects, but other objects which duck-type like them (such as an open zip file entry from rubyzip)
   http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18460

4. It would be convenient to be able to provide a proc object to generate a streamed response:
   proc { |out| out << "data"; out << "more data"; etc }

5. The default block size of 4K is too small to be efficient when serving large files. This was already fixed for 1.9 in
   http://redmine.ruby-lang.org/repositories/revision/ruby-19?rev=10167
   (default now 64K and tunable). Please consider this for 1.8, especially since a similar improvement has been backported for Net::HTTP in
   http://redmine.ruby-lang.org/repositories/revision/ruby-18?rev=12092

The attached file contains small monkey-patches to address these issues. If there is interest these could be rewritten as actual patches against webrick.
=end

[shyouhei (Shyouhei Urabe)]

• Assignee set to gotoyuzo (GOTOU Yuuzou)

=begin

=end

[candlerb (Brian Candler)]

=begin
There is an additional problem in setup_header, which prevents use of send_body_proc to stream bodies to HTTP/1.0 clients where no Content-Length has been set.

   elsif @header['content-length'].nil?
     unless @body.is_a?(IO)  # <<<<< PROBLEM: Proc is not an IO!
       @header['content-length'] = @body ? @body.size : 0
     end
   end

I suggest something like this:

   elsif @header['content-length'].nil?
     if @body.nil?
       @header['content-length'] = 0
     elsif @body.respond_to?(:size)
       @header['content-length'] = @body.size
     else
       @header['connection'] = 'close'
     end
   end

=end

[candlerb (Brian Candler)]

=begin
I have made individual patches for all these issues and posted them onto github. There are three separate branches:

http://github.com/candlerb/webrick/commits/master
http://github.com/candlerb/webrick/commits/ruby18
http://github.com/candlerb/webrick/commits/ruby186

Most of my testing has been on the ruby186 branch, although I have checked that the test suites still pass for ruby18 (with 1.8.7p72) and master (with 1.9.1p0)

The main difference between these branches is that the master branch uses 'bytesize' instead of 'size'

I hope this work will make it easier for people to test the changes, and/or for them to be committed into subversion.

=end

[nahi (Hiroshi Nakamura)]

• Category set to lib
• Assignee changed from gotoyuzo (GOTOU Yuuzou) to nahi (Hiroshi Nakamura)

=begin

=end

[nahi (Hiroshi Nakamura)]

=begin
Sorry for late response.
Applied the patch No. 2 for '100-continue' to ruby_1_9: http://redmine.ruby-lang.org/repositories/revision/ruby-19?rev=29218
Thank you!

I'll give a look to other changes later. For now, No.3 sounds good and No.1 and No.4 could be OK.
=end

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Assigned

=begin

=end

[nahi (Hiroshi Nakamura)]

• Project changed from Ruby 1.8 to Ruby master
• Category changed from lib to lib

[nahi (Hiroshi Nakamura)]

• Target version set to 2.0.0

r32192 is a fix for No.1. User can set HTTPResponse#chunked = true but it might not be a choice (Some client does not support chunked encoding for example.)

No.3 and No.4 should be discussed in the future...

[mame (Yusuke Endoh)]

NaHi, are you still willing to discuss about No.3 and 4?
If not, could you please close this?

--
Yusuke Endoh [e-mail]

[nahi (Hiroshi Nakamura)]

• Priority changed from 3 to Normal

Sorry that I kept this open long time.

Yes, the feature request for streaming (No.3 and No.4) is reasonable. I'll re-evaluate the patch before 2.0.0

I keep the assignee to me and raise the priority.

[mame (Yusuke Endoh)]

• Target version changed from 2.0.0 to 2.6

[mame (Yusuke Endoh)]

• Assignee changed from nahi (Hiroshi Nakamura) to normalperson (Eric Wong)

Eric Wong,

Could you handle this very old ticket about webrick?

[normalperson (Eric Wong)]

[e-mail] wrote:

Could you handle this very old ticket about webrick?

Sure; it looks like a lot are already done except 4. Will take
a closer look tomorrow or next week. Thanks for the ping.

• Status changed from Assigned to Closed

Applied in changeset trunk|r60584.

---

webrick: support Proc objects as body responses

• lib/webrick/httpresponse.rb (send_body): call send_body_proc
  (send_body_proc): new method
  (class ChunkedWrapper): new class

• test/webrick/test_httpresponse.rb (test_send_body_proc): new test
  (test_send_body_proc_chunked): ditto
  [Feature #855]