Bug #21202: Ripper wrongly concats separate tstring_content within nested unterminated heredoc

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.5.0dev (2025-03-27T18:17:40Z master 67d1dd2ebd) +PRISM [x86_64-linux]
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN

[tompng (tomoya ishida)]

code = <<~'CODE'
  <<H1
  #{<<H2}a
  H2
  b
CODE

Ripper.tokenize(code) #=> ["<<H1", "\n", "\#{", "<<H2", "}", "a\nb\n", "H2\n"]

"a\nb\n" should be a separate token "a\n" and "b\n"

[tompng (tomoya ishida)]

• ruby -v set to ruby 3.5.0dev (2025-03-27T18:17:40Z master 67d1dd2ebd) +PRISM [x86_64-linux]

Pull request: https://github.com/ruby/ruby/pull/13000

[tompng (tomoya ishida)]

• Status changed from Open to Closed

Applied in changeset git|a4a60195502add094fb52a587411bbd0c19facce.

---

[Bug #21202] Fix wrong token concat while tokenizing nested unterminated heredoc (#13000)