Backport #8375

Lexer error when specific character sequence occurs in string literals

Added by Dimitri Geshef over 2 years ago. Updated about 2 years ago.

[ruby-core:54846]
Status:Closed
Priority:Normal
Assignee:Usaku NAKAMURA

Description

The next line causes lexer/parser error:

problematic_string = "#$%"

The error message is: unterminated string meets end of file

The bug occurs in MRI1.9.3... ruby -v = ruby 1.9.3p374 (2013-01-15) [i386-mingw32]

... and in 2.0.0 ruby -v = ruby 2.0.0p0 (2013-02-24) [i386-mingw32]

Associated revisions

Revision 40607
Added by Nobuyoshi Nakada over 2 years ago

parse.y: fail if invalid name

  • parse.y (parser_yylex): fail if $, @, @@ are not followed by a valid name character. [Bug #8375].

Revision 40635
Added by Nobuyoshi Nakada over 2 years ago

parse.y: invalid name as mere string

  • parse.y (parser_peek_variable_name): treat invalid global, class, and instance variable names as mere strings rather than errors. [Bug #8375]

Revision 42462
Added by Usaku NAKAMURA about 2 years ago

merge revision(s) 40606,40607,40635: [Backport #8375]

test_scanner_events.rb: assert_location

* test/ripper/test_scanner_events.rb (TestRipper#assert_location):

rename so skipped in backtraces.
* parse.y (parser_yylex): fail if $, @, @@ are not followed by a valid
name character. [Bug #8375].

* parse.y (parser_peek_variable_name): treat invalid global, class,
  and instance variable names as mere strings rather than errors.
   [Bug #8375]

History

#1 Updated by Benoit Daloze over 2 years ago

The complete error is:

$ ruby -e 'problematic_string = "#$%"'
-e:1: syntax error, unexpected $undefined
problematic_string = "#$%"
^
-e:1: unterminated string meets end of file

So this is because it expects a global named $% but this is an invalid global name.

followed by (@@ or @ or $) is interpolating the following variable in double-quoted strings.

It would be nice if the error message could be clearer.

(I personally dislike the special syntax for # (@@ or @ or $) instead of plain #{expr},
it leads to weird cases with $" ($LOADED_FEATURES): "#$" and "#$"").

It can be avoided by using "#\$%" or using simple quotes as '#$%'.

#2 Updated by Nobuyoshi Nakada over 2 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r40607.
Dimitri, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


parse.y: fail if invalid name

  • parse.y (parser_yylex): fail if $, @, @@ are not followed by a valid name character. [Bug #8375].

#3 Updated by Tomoyuki Chikanaga over 2 years ago

  • Status changed from Closed to Assigned
  • Assignee set to Tomoyuki Chikanaga
  • Tracker changed from Bug to Backport
  • Project changed from Ruby trunk to Backport200

#4 Updated by Yukihiro Matsumoto over 2 years ago

I'd like to treat them as mere strings rather than errors.

Matz.

#5 Updated by Tomoyuki Chikanaga over 2 years ago

  • Assignee changed from Tomoyuki Chikanaga to Nobuyoshi Nakada

#6 Updated by Tomoyuki Chikanaga over 2 years ago

  • Assignee changed from Nobuyoshi Nakada to Tomoyuki Chikanaga

add r40635 for backport.

#7 Updated by Tomoyuki Chikanaga over 2 years ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r40650.
Dimitri, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 40606,40607,40635: [Backport #8375]

test_scanner_events.rb: assert_location

* test/ripper/test_scanner_events.rb (TestRipper#assert_location):

rename so skipped in backtraces.
* parse.y (parser_yylex): fail if $, @, @@ are not followed by a valid
name character. [Bug #8375].

* parse.y (parser_peek_variable_name): treat invalid global, class,
  and instance variable names as mere strings rather than errors.
   [Bug #8375]

#8 Updated by Sakuro OZAWA about 2 years ago

=begin
# The bug occurs in MRI1.9.3... ruby -v = ruby 1.9.3p374 (2013-01-15) [i386-mingw32]

Is this to be backported to 1.9.3?
=end

#9 Updated by Tomoyuki Chikanaga about 2 years ago

  • Project changed from Backport200 to Backport193
  • Status changed from Closed to Assigned
  • Assignee changed from Tomoyuki Chikanaga to Usaku NAKAMURA

Thanks you for your notice! I move this ticket to Backport93.
usa-san will make a judgement about it.

#10 Updated by Usaku NAKAMURA about 2 years ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r42462.
Dimitri, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 40606,40607,40635: [Backport #8375]

test_scanner_events.rb: assert_location

* test/ripper/test_scanner_events.rb (TestRipper#assert_location):

rename so skipped in backtraces.
* parse.y (parser_yylex): fail if $, @, @@ are not followed by a valid
name character. [Bug #8375].

* parse.y (parser_peek_variable_name): treat invalid global, class,
  and instance variable names as mere strings rather than errors.
   [Bug #8375]

Also available in: Atom PDF