Project

General

Profile

Bug #3765

Ripper::Lexer missed out tokens after '=>' operator

Added by ngtzeyang (TzeYang Ng) about 9 years ago. Updated over 8 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.1p376 (2009-12-07 revision 26041) [x86_64-linux]
Backport:
[ruby-core:31941]

Description

=begin
Given the following ripper use case:

pp Ripper.lex(':x => 1')

I'm getting:

[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"]]

I'm thinking the output should be:

[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"],
[[1, 4], :on_sp, " "],
[[1, 5], :on_int, "1"]]

Somehow the trailing ' 1' is missed out. Seems like a bug ?
=end

Associated revisions

Revision cff3c3d5
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@29120 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

Revision 29120
Added by shyouhei (Shyouhei Urabe) about 9 years ago

  • string.c (rb_str_prepend): new method by Sora Harakami [Feature #3765]

History

#1

Updated by eitoball (Eito Katagiri) about 9 years ago

=begin
I think that it is because the provided statement was illegal and Ripper stopped parsing it.

% ruby -ve ":x => 1"
ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-darwin9.8.0]
-e:1: syntax error, unexpected tASSOC, expecting $end
:x => 1
^

=end

#2

Updated by mame (Yusuke Endoh) about 9 years ago

  • Status changed from Open to Rejected

=begin
Eito is right.

Ruby lexer has very complex states.
Such an incomplete code fragment may not parse.
You should write complete code:

$ ruby -rripper -e 'p *Ripper.lex("foo :x => 1")'
[[1, 0], :on_ident, "foo"]
[[1, 3], :on_sp, " "]
[[1, 4], :on_symbeg, ":"]
[[1, 5], :on_ident, "x"]
[[1, 6], :on_sp, " "]
[[1, 7], :on_op, "=>"]
[[1, 9], :on_sp, " "]
[[1, 10], :on_int, "1"]

--
Yusuke Endoh mame@tsg.ne.jp
=end

Also available in: Atom PDF