Project

General

Profile

Actions
Copy link

Bug #19736

closed

Ripper.tokenize ignores token after heredoc_beg when embexpr inside heredoc is unclosed

Bug #19736: Ripper.tokenize ignores token after heredoc_beg when embexpr inside heredoc is unclosed
1

Added by tompng (tomoya ishida) over 3 years ago. Updated over 3 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.3.0dev (2023-06-19T09:28:10Z master 2a80bac9f0) [x86_64-linux]
Backport:
3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
[ruby-core:113938]

Description

I expect Ripper.tokenize(code) to tokenize +1+2+3\n part.

code = <<'RUBY'
<<A+1+2+3
#{4
RUBY
Ripper.tokenize code
# => ["<<A", "\#{", "4", "\n"]

It will be easy for IRB's internal code to handle indent and other things if Ripper.tokenize can tokenize all part of the code even if it is incomplete.

Actions
Copy link

Also available in: PDF Atom