Project

General

Profile

Actions

Bug #20517

closed

`Ripper.tokenize('"\\M-あ"')` separates encoding valid string to encoding invalid string.

Added by tompng (tomoya ishida) 6 months ago. Updated 4 months ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.4.0dev (2024-04-11T08:57:52Z master e7f8db9079) [x86_64-linux]
[ruby-core:118132]

Description

Ripper.tokenize '"\\M-あ"'
=> ["\"", "\\M-\xE3", "\x81", "\x82", "\""]

I expect all tokens to be valid_encoding if the source string is valid_encoding.
Similar to https://bugs.ruby-lang.org/issues/20030

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0