Feature #855

HTTP/1.1 fixes and other enhancements to webrick

Added by Brian Candler over 5 years ago. Updated over 1 year ago.

[ruby-core:20474]
Status:Assigned
Priority:Normal
Assignee:Hiroshi Nakamura
Category:lib
Target version:next minor

Description

=begin
I raised the following issues on ruby-core:

  1. When returning an open IO object (without Content-Length or chunking), Webrick fails to close the HTTP/1.1 connection, and hence the client waits forever for the end of the data.
    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18454
    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18565

  2. Webrick makes it very difficult to send a '100 continue' response when a HTTP/1.1 client requests one, and yet the RFC2616 says it must do so.
    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18459

  3. It would be convenient to be able to stream not just real IO objects, but other objects which duck-type like them (such as an open zip file entry from rubyzip)
    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18460

  4. It would be convenient to be able to provide a proc object to generate a streamed response:
    proc { |out| out << "data"; out << "more data"; etc }

  5. The default block size of 4K is too small to be efficient when serving large files. This was already fixed for 1.9 in
    http://redmine.ruby-lang.org/repositories/revision/ruby-19?rev=10167
    (default now 64K and tunable). Please consider this for 1.8, especially since a similar improvement has been backported for Net::HTTP in
    http://redmine.ruby-lang.org/repositories/revision/ruby-18?rev=12092

    The attached file contains small monkey-patches to address these issues. If there is interest these could be rewritten as actual patches against webrick.
    =end

webrick-patches.rb Magnifier - Monkey patches to webrick (2.69 KB) Brian Candler, 12/11/2008 07:08 PM

Associated revisions

Revision 29218
Added by Hiroshi Nakamura over 3 years ago

  • lib/webrick/httprequest.rb (WEBrick::HTTPRequest#continue): add
    method for generating HTTP/1.1 100 continue response if the client
    expects it, otherwise does nothing. Patch by Brian Candler.
    ref #855.

    * test/webrick/test_httprequest.rb: test added.
    

Revision 32188
Added by Hiroshi Nakamura almost 3 years ago

  • lib/webrick/httpresponse.rb (HTTPResponse#setup_header): Close
    HTTP/1.1 connection when returning an IO object as response body
    without setting HTTPResponse#chunked to true. See #855 no.1.

  • test/webrick/test_httpserver.rb: Test it.

History

#1 Updated by Shyouhei Urabe about 5 years ago

  • Assignee set to GOTOU Yuuzou

=begin

=end

#2 Updated by Brian Candler about 5 years ago

=begin
There is an additional problem in setupheader, which prevents use of sendbody_proc to stream bodies to HTTP/1.0 clients where no Content-Length has been set.

   elsif @header['content-length'].nil?
     unless @body.is_a?(IO)  # <<<<< PROBLEM: Proc is not an IO!
       @header['content-length'] = @body ? @body.size : 0
     end
   end

I suggest something like this:

   elsif @header['content-length'].nil?
     if @body.nil?
       @header['content-length'] = 0
     elsif @body.respond_to?(:size)
       @header['content-length'] = @body.size
     else
       @header['connection'] = 'close'
     end
   end

=end

#3 Updated by Brian Candler about 5 years ago

=begin
I have made individual patches for all these issues and posted them onto github. There are three separate branches:

http://github.com/candlerb/webrick/commits/master
http://github.com/candlerb/webrick/commits/ruby18
http://github.com/candlerb/webrick/commits/ruby186

Most of my testing has been on the ruby186 branch, although I have checked that the test suites still pass for ruby18 (with 1.8.7p72) and master (with 1.9.1p0)

The main difference between these branches is that the master branch uses 'bytesize' instead of 'size'

I hope this work will make it easier for people to test the changes, and/or for them to be committed into subversion.

=end

#4 Updated by Hiroshi Nakamura over 3 years ago

  • Category set to lib
  • Assignee changed from GOTOU Yuuzou to Hiroshi Nakamura

=begin

=end

#5 Updated by Hiroshi Nakamura over 3 years ago

=begin
Sorry for late response.
Applied the patch No. 2 for '100-continue' to ruby19: http://redmine.ruby-lang.org/repositories/revision/ruby-19?rev=29218
Thank you!

I'll give a look to other changes later. For now, No.3 sounds good and No.1 and No.4 could be OK.
=end

#6 Updated by Shyouhei Urabe over 3 years ago

  • Status changed from Open to Assigned

=begin

=end

#7 Updated by Hiroshi Nakamura almost 3 years ago

  • Project changed from Ruby 1.8 to ruby-trunk
  • Category changed from lib to lib

#8 Updated by Hiroshi Nakamura almost 3 years ago

  • Target version set to 2.0.0

r32192 is a fix for No.1. User can set HTTPResponse#chunked = true but it might not be a choice (Some client does not support chunked encoding for example.)

No.3 and No.4 should be discussed in the future...

#9 Updated by Yusuke Endoh about 2 years ago

NaHi, are you still willing to discuss about No.3 and 4?
If not, could you please close this?

Yusuke Endoh mame@tsg.ne.jp

#10 Updated by Hiroshi Nakamura about 2 years ago

  • Priority changed from Low to Normal

Sorry that I kept this open long time.

Yes, the feature request for streaming (No.3 and No.4) is reasonable. I'll re-evaluate the patch before 2.0.0

I keep the assignee to me and raise the priority.

#11 Updated by Yusuke Endoh over 1 year ago

  • Target version changed from 2.0.0 to next minor

Also available in: Atom PDF