Project

General

Profile

Backport #2722

gets on a large file takes a very very long time

Added by ghazel (Greg Hazel) over 9 years ago. Updated 7 days ago.

Status:
Closed
Priority:
Normal
Assignee:
-
[ruby-core:28103]

Description

=begin
This problem occurs on 1.8.6, 1.8.7 and 1.9.1 (all the versions I tested)

This simple script demonstrates the problem:

# 100 MB
n = 100 * 1000 * 1000
puts "writing"
File.open("foo", 'wb'){|f| f.write(" " * n) }
puts "reading"
File.open("foo", 'rb') do |io|
io.gets
end

This takes about 1911 seconds. Using a 10MB file completes in 19 seconds, instead of in 1/10th the time as you would imagine. Similarly a 1MB file completes in 0.18 seconds.
=end


Related issues

Related to Backport187 - Backport #2741: gets with large file is slow in windozeClosedActions

History

#1

Updated by matz (Yukihiro Matsumoto) over 9 years ago

=begin
Hi,

In message "Re: [ruby-core:28103] [Bug #2722] gets on a large file takes a very very long time"
on Mon, 8 Feb 2010 17:33:09 +0900, Greg Hazel redmine@ruby-lang.org writes:

|This problem occurs on 1.8.6, 1.8.7 and 1.9.1 (all the versions I tested)

I could reproduce the problem on 1.9, but not on 1.8. 1.9 has been
fixed by r26622. Thank you for the report.

                        matz.

=end

#2

Updated by matz (Yukihiro Matsumoto) over 9 years ago

=begin
Hi,

In message "Re: [ruby-core:28138] Re: [Bug #2722] gets on a large file takes a very very long time"
on Wed, 10 Feb 2010 06:09:54 +0900, Roger Pack rogerdpack2@gmail.com writes:

|For me it performs fast when I replace .gets with .read
|Perhaps there is a reason for this?

gets need to scan for the newline. Here's my numbers

ruby 1.9.2dev (2010-02-09 trunk 26623) [i686-linux]
gets: 0.13s user 0.42s system 88% cpu 0.626 total
read: 0.08s user 0.46s system 93% cpu 0.578 total

ruby 1.8.8dev (2010-02-07 revision 26612) [i486-linux]
gets: 2.54s user 0.44s system 97% cpu 3.073 total
read: 2.47s user 0.45s system 97% cpu 3.007 total

1.8 has bottleneck on String#times, maybe we can work on it.

                        matz.

=end

#3

Updated by naruse (Yui NARUSE) over 9 years ago

  • Priority changed from Normal to 3

=begin

=end

#4

Updated by jeremyevans0 (Jeremy Evans) 7 days ago

  • ruby -v deleted (ruby 1.8.6 (2009-08-04 patchlevel 383) [i386-mingw32])
  • Status changed from Open to Closed
  • Description updated (diff)
  • Project changed from Ruby 1.8 to Backport187
  • Tracker changed from Bug to Backport

Also available in: Atom PDF