Bug #8383

Ripper.lex does not handle correctly symbols whose identifiers coincide with keyword names

Added by Bozhidar Batsov about 2 years ago. Updated about 2 years ago.

[ruby-core:<unknown>]
Status:Rejected
Priority:Normal
Assignee:-
ruby -v:ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin12.3.0] Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN

Description

This problem is best illustrated with an example

[23] pry(main)> Ripper.lex(':one')
=> [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_ident, "one"]]
[24] pry(main)> Ripper.lex(':alias')
=> [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_kw, "alias"]]

I'm pretty sure that the correct tokenization of the second example should yield :on_ident instead of :on_kw. Or am I missing something?

This is the behaviour on both 1.9.3 and 2.0.0.


Related issues

Duplicates Ruby trunk - Bug #6306: Ripper lexes :on_kw when it should find :on_ident Rejected 04/17/2012

History

#1 Updated by Bozhidar Batsov about 2 years ago

On a related note I don't think this works correctly with method definitions as well:

[30] pry(main)> Ripper.lex('def alias(arg)')
=> [[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_kw, "alias"],
[[1, 9], :on_lparen, "("],
[[1, 10], :on_ident, "arg"],
[[1, 13], :on_rparen, ")"]]

[31] pry(main)> Ripper.lex('def aliass(arg)')
=> [[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_ident, "aliass"],
[[1, 10], :on_lparen, "("],
[[1, 11], :on_ident, "arg"],
[[1, 14], :on_rparen, ")"]]

#2 Updated by Nobuyoshi Nakada about 2 years ago

  • Status changed from Open to Rejected

Duplicates #6306.

Also available in: Atom PDF