Bug #20305: commit 1d2d25dcadda0764f303183ac091d0c87b432566 breaks grapheme_clusters

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED

[fablestales (Fable Tales)]

given a script:

#script.rb
p "안녕".byteslice(0, 4).grapheme_clusters

The commit 1d2d25dcadda0764f303183ac091d0c87b432566 (https://github.com/ruby/ruby/commit/1d2d25dcadda0764f303183ac091d0c87b432566) breaks the grapheme_clusters method on a byte slice

(commit 1d2d25dcadda0764f303183ac091d0c87b432566)
 ((HEAD detached at 1d2d25dcad)) $ ./ruby --disable=gems script.rb
["안", "\xEB"]
 ((HEAD detached at 1d2d25dcad)) $ git checkout HEAD^
(114e71d06280f9c57b9859ee4405ae89a989ddb6)
 ((HEAD detached at 114e71d062)) $ make -j
...
 ((HEAD detached at 114e71d062)) $ ./ruby --disable=gems script.rb
["안"]
 ((HEAD detached at 114e71d062)) $ cat script.rb
p "안녕".byteslice(0, 4).grapheme_clusters

the expected result here is almost certainly the latter output, and not the former.

[fablestales (Fable Tales)]

fablestales (Fable Tales) wrote:

given a script:

#script.rb
p "안녕".byteslice(0, 4).grapheme_clusters

The commit 1d2d25dcadda0764f303183ac091d0c87b432566 (https://github.com/ruby/ruby/commit/1d2d25dcadda0764f303183ac091d0c87b432566) breaks the grapheme_clusters method on a byte slice

(commit 1d2d25dcadda0764f303183ac091d0c87b432566)
 ((HEAD detached at 1d2d25dcad)) $ ./ruby --disable=gems script.rb
["안", "\xEB"]
 ((HEAD detached at 1d2d25dcad)) $ git checkout HEAD^
(114e71d06280f9c57b9859ee4405ae89a989ddb6)
 ((HEAD detached at 114e71d062)) $ make -j
...
 ((HEAD detached at 114e71d062)) $ ./ruby --disable=gems script.rb
["안"]
 ((HEAD detached at 114e71d062)) $ cat script.rb
p "안녕".byteslice(0, 4).grapheme_clusters

the expected result here is almost certainly the latter output, and not the former.

to clarify: grapheme_clusters used to ignore partial characters from a byteslice, now it does not.

[fablestales (Fable Tales)]

I added a failing test to reproduce this issue in this PR: https://github.com/ruby/ruby/pull/10103

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|3a04ea2d0379dd8c6623c2d5563e6b4e23986fae.

---

[Bug #20305] Fix matching against an incomplete character

When matching against an incomplete character, some enclen calls are
expected not to exceed the limit, and some are expected to return the
required length and then the results are checked if it exceeds.

[byroot (Jean Boussier)]

• Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED, 3.3: REQUIRED