Project

General

Profile

Feature #14268

[PATCH] net/protocol: optimize large read case

Added by normalperson (Eric Wong) over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Target version:
-
[ruby-core:84586]

Description

Any comment?  Will commit in a few days if none.

net/protocol: optimize large read case

There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.

The following script exhibits reduced memory usage and
runtimes using the time(1) command:

    2.9s =>  2.6s
    70MB => 12 MB

---------
require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
  c = s.accept
  c.readpartial(16384)
  c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
  IO.copy_stream('/dev/zero', c, len)
  c.close
end

addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
  http.request_get('/') do |res|
    dig = Digest::MD5.new
    res.read_body { |buf|
      dig.update(buf)
      # String#clear is important to reduce malloc overhead,
      # but most Ruby programmers don't do this :<
      buf.clear
    }
    puts dig.hexdigest
  end
end
----------

* lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len

Files

Associated revisions

Revision d9beb769
Added by normal over 1 year ago

net/protocol: optimize large read case

There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.

The following script exhibits reduced memory usage and
runtimes using the time(1) command:

    2.9s =>  2.6s
    70MB => 12 MB

require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end

addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end

end

  • lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len [Feature #14268]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61602 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 61602
Added by normalperson (Eric Wong) over 1 year ago

net/protocol: optimize large read case

There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.

The following script exhibits reduced memory usage and
runtimes using the time(1) command:

2.9s =>  2.6s
70MB => 12 MB

require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end

addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end

end

  • lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len [Feature #14268]

Revision 61602
Added by normal over 1 year ago

net/protocol: optimize large read case

There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.

The following script exhibits reduced memory usage and
runtimes using the time(1) command:

2.9s =>  2.6s
70MB => 12 MB

require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end

addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end

end

  • lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len [Feature #14268]

History

#1

Updated by Anonymous over 1 year ago

  • Status changed from Open to Closed

Applied in changeset trunk|r61602.


net/protocol: optimize large read case

There are several places where rbuf_consume is called with
@rbuf.size as its length arg; simplify that case by avoiding
the slow String#slice! operation in favor of a lightweight
replacement.

The following script exhibits reduced memory usage and
runtimes using the time(1) command:

2.9s =>  2.6s
70MB => 12 MB

require 'net/http'
require 'digest/md5'
Thread.abort_on_exception = true
s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
th = Thread.new do
c = s.accept
c.readpartial(16384)
c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
IO.copy_stream('/dev/zero', c, len)
c.close
end

addr = s.addr
Net::HTTP.start(addr[3], addr[1]) do |http|
http.request_get('/') do |res|
dig = Digest::MD5.new
res.read_body { |buf|
dig.update(buf)
# String#clear is important to reduce malloc overhead,
# but most Ruby programmers don't do this :<
buf.clear
}
puts dig.hexdigest
end

end

  • lib/net/protocol (rbuf_consume): optimize for @rbuf.size == len [Feature #14268]

Also available in: Atom PDF