Bug #20039: Matching US-ASCII string to copied UTF-8 Regexp causes invalid multibyte character error

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-12-03 master 85bc80a)
• Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED

[[e-mail] (Dustin Brown)]

Matching a US-ASCII string to a UTF-8 encoded regexp with multibyte characters works as expected.

re = Regexp.new("\u2018".encode("UTF-8"))
"".encode("US-ASCII").match?(re) 

=> false

However, if that regexp is used to initialize a new regexp, the comparison fails with a Invalid mutibyte character error.

re = Regexp.new("\u2018".encode("UTF-8"))
"".encode("US-ASCII").match?(Regexp.new(re))

=> ArgumentError: regexp preprocess failed: invalid multibyte character

After a bunch of digging, I discovered that this error was due to the fixed encoding flag not being copied over from the original regexp. This pull request address the issue by copying the fixed encoding and no encoding flags during reg_copy.

Ref: https://github.com/ruby/ruby/pull/9120

[[e-mail] (Dustin Brown)]

• Description updated (diff)

[nobu (Nobuyoshi Nakada)]

• Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED

[[e-mail] (Dustin Brown)]

• Status changed from Open to Closed

Applied in changeset git|d89280e8bf6496aa83326b5f9c293724bd1cc1e9.

---

Copy encoding flags when copying a regex [Bug #20039]

• :bug: Fixes Bug #20039

When a Regexp is initialized with another Regexp, we simply copy the
properties from the original. However, the flags on the original were
not being copied correctly. This caused an issue when the original had
multibyte characters and was being compared with an ASCII string.
Without the forced encoding flag (KCODE_FIXED) transferred on to the
new Regexp, the comparison would fail. See the included test for an
example.

Co-authored-by: Nobuyoshi Nakada [e-mail]