Project

General

Profile

Actions
Copy link

Bug #19838

closed

Ripper nested heredocs

Added by kddnewton (Kevin Newton) over 1 year ago. Updated about 1 year ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
Backport:
3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
[ruby-core:114381]

Description

I'm seeing some odd behavior on nested heredocs in the ripper lexer.

<<~H1
  1
  #{<<~H2}
    2
  H2
  3
H1

$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'

[[[1, 0], :on_heredoc_beg, "<<~H1", BEG],
 [[1, 5], :on_nl, "\n", BEG],
 [[2, 0], :on_ignored_sp, "  ", BEG],
 [[2, 2], :on_tstring_content, "1\n", BEG],
 [[3, 0], :on_ignored_sp, "  ", BEG],
 [[3, 2], :on_embexpr_beg, "\#{", BEG],
 [[3, 4], :on_heredoc_beg, "<<~H2", BEG],
 [[3, 9], :on_embexpr_end, "}", END],
 [[3, 10], :on_tstring_content, "\n", BEG],
 [[4, 0], :on_ignored_sp, "    ", BEG],
 [[4, 4], :on_tstring_content, "2\n", BEG],
 [[5, 0], :on_heredoc_end, "  H2\n", BEG],
 [[6, 0], :on_ignored_sp, "  ", BEG],
 [[6, 2], :on_tstring_content, "3\n", BEG],
 [[7, 0], :on_heredoc_end, "H1\n", BEG]]

This seems totally correct. But if you change it to a non-indenting heredoc:

<<-H1
  1
  #{<<~H2}
    2
  H2
  3
H1

$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'

[[[1, 0], :on_heredoc_beg, "<<-H1", BEG],
 [[1, 5], :on_nl, "\n", BEG],
 [[2, 0], :on_tstring_content, "  1\n" + "  ", BEG],
 [[3, 2], :on_embexpr_beg, "\#{", BEG],
 [[3, 4], :on_heredoc_beg, "<<~H2", BEG],
 [[3, 9], :on_embexpr_end, "}", END],
 [[3, 10], :on_tstring_content, "\n" + "  3\n", BEG],
 [[4, 0], :on_ignored_sp, "    ", BEG],
 [[4, 4], :on_tstring_content, "2\n", BEG],
 [[5, 0], :on_heredoc_end, "  H2\n", BEG],
 [[7, 0], :on_heredoc_end, "H1\n", BEG]]

Now the newline on line 3 gets concatenated with the content from line 6, and that token is dropped from the list entirely. This seems unintentional, but I'm not sure.

Actions
Copy link

Also available in: Atom PDF

Like0
Like0