Bug #5202

RubyLex very slow when lexing large files

Added by Ryan Melton over 2 years ago. Updated over 1 year ago.

[ruby-core:39018]
Status:Closed
Priority:Normal
Assignee:Keiju Ishitsuka
Category:-
Target version:-
ruby -v:ruby 1.9.2p290 (2011-07-09) [i386-mingw32] Backport:

Description

RubyLex exponentially slower when lexing large files based on the size of the file. This is because of a call to @readed.reverse inside of the get_readed and ungetc methods. The following updated versions of these methods greatly improve performance by using rindex instead of reverse.index.

def getreaded
if idx = @readed.rindex("\n")
@base
charno = @readed.size - (idx + 1)
else
@base
char_no += @readed.size
end

readed = @readed.join("")
@readed = []
readed

end

def ungetc(c = nil)
if @herereaded.empty?
c2 = @readed.pop
else
c2 = @here
readed.pop
end
c = c2 unless c
@rests.unshift c #c =
@seek -= 1
if c == "\n"
@lineno -= 1
if idx = @readed.rindex("\n")
@char
no = idx + 1
else
@charno = @basecharno + @readed.size
end
else
@char
no -= 1
end
end

Associated revisions

Revision 38609
Added by Keiju Ishitsuka over 1 year ago

  • lib/irb/ruby-lex.rb: improve RubyLex performance for large files [Bug #5202]. Patch by ryanmelt.

History

#1 Updated by Yui NARUSE over 2 years ago

  • Status changed from Open to Assigned
  • Assignee set to Keiju Ishitsuka

#2 Updated by Keiju Ishitsuka over 1 year ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r38609.
Ryan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/irb/ruby-lex.rb: improve RubyLex performance for large files [Bug #5202]. Patch by ryanmelt.

Also available in: Atom PDF