Project

General

Profile

Actions

Bug #19838

open

Ripper nested heredocs

Added by kddnewton (Kevin Newton) 4 months ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:114381]

Description

I'm seeing some odd behavior on nested heredocs in the ripper lexer.

<<~H1
  1
  #{<<~H2}
    2
  H2
  3
H1

$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'

[[[1, 0], :on_heredoc_beg, "<<~H1", BEG],
 [[1, 5], :on_nl, "\n", BEG],
 [[2, 0], :on_ignored_sp, "  ", BEG],
 [[2, 2], :on_tstring_content, "1\n", BEG],
 [[3, 0], :on_ignored_sp, "  ", BEG],
 [[3, 2], :on_embexpr_beg, "\#{", BEG],
 [[3, 4], :on_heredoc_beg, "<<~H2", BEG],
 [[3, 9], :on_embexpr_end, "}", END],
 [[3, 10], :on_tstring_content, "\n", BEG],
 [[4, 0], :on_ignored_sp, "    ", BEG],
 [[4, 4], :on_tstring_content, "2\n", BEG],
 [[5, 0], :on_heredoc_end, "  H2\n", BEG],
 [[6, 0], :on_ignored_sp, "  ", BEG],
 [[6, 2], :on_tstring_content, "3\n", BEG],
 [[7, 0], :on_heredoc_end, "H1\n", BEG]]

This seems totally correct. But if you change it to a non-indenting heredoc:

<<-H1
  1
  #{<<~H2}
    2
  H2
  3
H1

$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'

[[[1, 0], :on_heredoc_beg, "<<-H1", BEG],
 [[1, 5], :on_nl, "\n", BEG],
 [[2, 0], :on_tstring_content, "  1\n" + "  ", BEG],
 [[3, 2], :on_embexpr_beg, "\#{", BEG],
 [[3, 4], :on_heredoc_beg, "<<~H2", BEG],
 [[3, 9], :on_embexpr_end, "}", END],
 [[3, 10], :on_tstring_content, "\n" + "  3\n", BEG],
 [[4, 0], :on_ignored_sp, "    ", BEG],
 [[4, 4], :on_tstring_content, "2\n", BEG],
 [[5, 0], :on_heredoc_end, "  H2\n", BEG],
 [[7, 0], :on_heredoc_end, "H1\n", BEG]]

Now the newline on line 3 gets concatenated with the content from line 6, and that token is dropped from the list entirely. This seems unintentional, but I'm not sure.

No data to display

Actions

Also available in: Atom PDF

Like0