Project

General

Profile

Actions

Bug #18962

closed

Ripper.lex does not handle \0, ^D and ^Z correctly.

Added by tompng (tomoya ishida) over 1 year ago. Updated over 1 year ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
[ruby-core:109474]

Description

Ripper.lex does not end with \0, ^D and ^Z, and it contains weird tokens including "\u0000"

# for syntax ok code
Ripper.lex("hello\0\0world").map{_3} #=> ["hello"]
Ripper.lex("hello\0world").map{_3}   #=> ["hello", "\u0000world"]
Ripper.lex("hello\0@world").map{_3}  #=> ["hello", "@world"]
Ripper.lex("hello\0:world").map{_3}  #=> ["hello", "\u0000:", "world"]
Ripper.lex("hello\0))").map{_3}      #=> ["hello", "\u0000)", ")"]
# for syntax invalid code
Ripper.lex("end\0*").map{_3} # => ["end", "\u0000*"]
Ripper.lex("end\0\0*").map{_3} # => ["end"]
Ripper.lex("end\0@end").map{_3} # => ["end", "@end"]
Actions

Also available in: Atom PDF

Like0
Like0