Project

General

Profile

Bug #2742

IO#read/gets can be very slow in doze

Added by rogerdpack (Roger Pack) over 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.2dev (2010-02-12 trunk 26649) [i386-mingw32]
Backport:
[ruby-core:28162]

Description

=begin
this code

n = 100 * 1000 * 1000
puts "writing"
File.open("foo", 'wb'){|f| f.write(" " * n) }
puts "reading"
File.open("foo", 'r') do |io|
io.read
end

takes something like 700s in windows, most of the time is spent in rb_str_resize. Cause seems to be that it is rb_str_resize'ing the string by 1024B at a time, so N2 in the amount of time it takes to read it in because it has to keep copying it to the newly sized string.

The same thing happens for

File.open("foo", 'rb') do |io|
io.gets
end

except that it reads (and resizes) in 8K bytes. Not sure why they're different sizes. I'd imagine this problem exists in Linux, but without as much of a negative impact as it reads in larger sizes.
=end


Related issues

Related to Backport187 - Backport #2741: gets with large file is slow in windozeClosedActions
Related to Ruby master - Bug #1332: Reading file on Windows is 500x slower then with previous Ruby versionClosed03/30/2009Actions

History

#1

Updated by rogerdpack (Roger Pack) over 9 years ago

=begin
This seems to help, though I'm not sure if it's the best way.

Index: io.c
===================================================================
--- io.c (revision 26655)
+++ io.c (working copy)
@@ -1170,7 +1170,7 @@
if (fptr->rbuf == NULL) {
fptr->rbuf_off = 0;
fptr->rbuf_len = 0;

  • fptr->rbuf_capa = 8192;
  • fptr->rbuf_capa = 64*1024; fptr->rbuf = ALLOC_N(char, fptr->rbuf_capa); } if (fptr->rbuf_len == 0) { @@ -1583,7 +1583,7 @@ rb_exc_raise(rb_econv_open_exc(sname, dname, ecflags)); fptr->cbuf_off = 0; fptr->cbuf_len = 0;
  • fptr->cbuf_capa = size < 1024 ? 1024 : size;
  • fptr->cbuf_capa = size < (64*1024) ? (64*1024) : size; fptr->cbuf = ALLOC_N(char, fptr->cbuf_capa); } }

=end

#2

Updated by nobu (Nobuyoshi Nakada) over 9 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

=begin
This issue was solved with changeset r26997.
Roger, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

=end

Also available in: Atom PDF