Project

General

Profile

Actions

Bug #13536

closed

Ripper fails to tokenize nested squiggly heredoc

Added by shyouhei (Shyouhei Urabe) almost 7 years ago. Updated over 6 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 2.5.0dev (2017-05-02 trunk 58544) [x86_64-darwin15]
[ruby-core:80977]

Description

require 'ripper'

str = \
<<~'end'
<<~"do"
#{
<<~"begin"
this must be a valid ruby
begin
}
do
end
puts eval(str)
str == Ripper.tokenize(str)

This program fails at the last line.

Actions #1

Updated by nobu (Nobuyoshi Nakada) almost 7 years ago

  • Status changed from Open to Closed

Applied in changeset trunk|r58545.


ripper/lexer.rb: nested indented heredoc

  • ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): fix for
    nested indedented here documents, where Elems are nested too.
    [ruby-core:80977] [Bug #13536]

Updated by shyouhei (Shyouhei Urabe) almost 7 years ago

  • Status changed from Closed to Open

Thank you, the exception disappeared.

But it seems incomplete. Look at this slightly different example:

require 'ripper'

str = \
<<~'end'
<<~"do"
#{
<<~"begin"
 this must be a valid ruby
begin
}
do
end
puts eval(str)
str == Ripper.tokenize(str).join

The last line does not raise exception, but evaluates false.

Actions #3

Updated by nobu (Nobuyoshi Nakada) almost 7 years ago

  • Status changed from Open to Closed

Applied in changeset trunk|r58584.


ripper/lexer.rb: nested indented heredoc

  • ext/ripper/lib/ripper/lexer.rb (on_heredoc_dedent): insert
    stripped leading spaces as on_ignored_sp elements, so that the
    original source can be reconsructed.
    [ruby-core:80977] [Bug #13536]

Updated by usa (Usaku NAKAMURA) over 6 years ago

  • Backport changed from 2.2: REQUIRED, 2.3: REQUIRED, 2.4: REQUIRED to 2.2: REQUIRED, 2.3: DONE, 2.4: REQUIRED

ruby_2_3 r59214 merged revision(s) 58545,58584.

Updated by nagachika (Tomoyuki Chikanaga) over 6 years ago

  • Backport changed from 2.2: REQUIRED, 2.3: DONE, 2.4: REQUIRED to 2.2: REQUIRED, 2.3: DONE, 2.4: DONE

ruby_2_4 r59247 merged revision(s) 58545,58584.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0