Bug #8383: Ripper.lex does not handle correctly symbols whose identifiers coincide with keyword names

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin12.3.0]
• Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN

[bozhidar (Bozhidar Batsov)]

This problem is best illustrated with an example

[23] pry(main)> Ripper.lex(':one')
=> [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_ident, "one"]]
[24] pry(main)> Ripper.lex(':alias')
=> [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_kw, "alias"]]

I'm pretty sure that the correct tokenization of the second example should yield :on_ident instead of :on_kw. Or am I missing something?

This is the behaviour on both 1.9.3 and 2.0.0.

[bozhidar (Bozhidar Batsov)]

On a related note I don't think this works correctly with method definitions as well:

[30] pry(main)> Ripper.lex('def alias(arg)')
=> [[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_kw, "alias"],
[[1, 9], :on_lparen, "("],
[[1, 10], :on_ident, "arg"],
[[1, 13], :on_rparen, ")"]]

[31] pry(main)> Ripper.lex('def aliass(arg)')
=> [[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_ident, "aliass"],
[[1, 10], :on_lparen, "("],
[[1, 11], :on_ident, "arg"],
[[1, 14], :on_rparen, ")"]]

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Rejected

Duplicates #6306.