Project

General

Profile

Actions

Bug #22600

closed

parse.y corrupts line-1 heredoc delimiter/resume offset by 3 bytes after a UTF-8 BOM

Bug #22600: parse.y corrupts line-1 heredoc delimiter/resume offset by 3 bytes after a UTF-8 BOM

Added by steviejj (Steven Johnstone) about 19 hours ago. Updated about 13 hours ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 4.0.9 (2026-10-23 revision 3e93b23afc) +PRISM [arm64-darwin27]
[ruby-core:127582]

Description

When a Ruby script or eval string starts with a UTF-8 BOM (\xEF\xBB\xBF) and declares a heredoc on line 1 (e.g. \xEF\xBB\xBF<<EOF\nEOF\n, \xEF\xBB\xBF(<<EOF)\nEOF\n, \xEF\xBB\xBFp(<<EOF)\nEOF\n, or \xEF\xBB\xBF[<<EOF]\nEOF\n), CRuby parse.y miscomputes the heredoc identifier/resume offset on line 1 by 3 bytes (the length of the UTF-8 BOM), looking up "..." shifted 3 bytes to the left (such as "(<<", "[<<", or "\xBF<<") and failing with SyntaxError: can't find string ... anywhere before EOF followed by unexpected constant.

If a newline separates the BOM from the heredoc (\xEF\xBB\xBF\n(<<EOF)\nEOF\n), both parse.y and --parser=prism succeed.

Discovered by differential fuzzing of prism against parse.y.

Reproduction

Tested on CRuby 4.1.0dev (2026-11-18 HEAD ed3e747aa4) and ruby 4.0.9 (2026-10-23 revision 3e93b23afc) +PRISM [arm64-darwin27].

eval("\xEF\xBB\xBF(<<EOF)\nEOF\n".b)
# --parser=prism:   nil  (exits 0)
# --parser=parse.y: SyntaxError: can't find string "(<<" anywhere before EOF

Updated by nobu (Nobuyoshi Nakada) about 13 hours ago Actions #1

  • Status changed from Open to Closed

Applied in changeset git|9036f16dc369aaf28ecdc9686293af1dbf88d087.


[Bug #22600] Count BOM just before here document

Actions

Also available in: PDF Atom