Bug #20924: IO#readline ignores the limit argument when the encoding is UTF-32LE and the limit would split a character

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.4.0dev (2024-11-28T12:38:16Z master 3af1a04741) +PRISM [x86_64-linux]
• Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: DONE

[javanthropus (Jeremy Bopp)]

require 'tempfile'

Tempfile.open(binmode: true, encoding: 'utf-32le') do |f|
  f.write('0123456789')
  f.rewind

  # A limit that would truncate a character becomes completely ignored
  f.readline(3).bytesize  # => 40; should be 4
end

Tempfile.open(binmode: true, encoding: 'utf-32le') do |f|
  f.write('0123456789')
  f.rewind

  # A limit on character boundaries is respected
  f.readline(4).bytesize  # => 4
end

Tempfile.open(encoding: 'utf-8:utf-32le') do |f|
  f.write('0123456789')
  f.rewind

  # A limit that would truncate a character becomes completely ignored
  f.readline(3).bytesize  # => 40; should be 4
end

Tempfile.open(encoding: 'utf-8:utf-32le') do |f|
  f.write('0123456789')
  f.rewind

  # A limit on character boundaries is respected
  f.readline(4).bytesize  # => 4
end

This doesn't happen with UTF-32BE. This also doesn't happen in Ruby 3.3, but it does happen in 3.4-dev and Ruby 3.0 - 3.2.

[javanthropus (Jeremy Bopp)]

Note that replacing #readline with #gets in the examples causes the issue to be seen in all Ruby versions, including 3.3.6.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|e90b447655dd39ad1eb645cdaae450efd605db00.

---

[Bug #20924] Fix reading with delimiter in wide character encodings

[nobu (Nobuyoshi Nakada)]

• Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED

ruby_3_2 3110d5c8abf8f710de42989fbc35870287a12f75 merged revision(s) e90b447655dd39ad1eb645cdaae450efd605db00.

[k0kubun (Takashi Kokubun)]

• Backport changed from 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: DONE

ruby_3_3 00147cbab567b05b8a4137875bbda341ef704760 merged revision(s) e90b447655dd39ad1eb645cdaae450efd605db00.