Bug #13874: String#valid_encoding? has side effects

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
• Backport: 2.2: REQUIRED, 2.3: DONE, 2.4: DONE

[ojundt (Oliver Jundt)]

I noticed the result of String#encode changes after a call to String#valid_encoding?. Other side effects were not visible so far.

Based on the documentation and intuition I would expect #valid_encoding? to be an operation without side effects.

content = "\xE5".dup.force_encoding(Encoding::ASCII_8BIT)

content.encode(Encoding::UTF_8, Encoding::UTF_8, invalid: :replace, replace: '?')
=> "?"

content.valid_encoding?

content.encode(Encoding::UTF_8, Encoding::UTF_8, invalid: :replace, replace: '?')
=> "\xE5"

[ojundt (Oliver Jundt)]

• Description updated (diff)

[ojundt (Oliver Jundt)]

Also reproducible on ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]

Not reproducible on ruby 2.2.5p319 (2016-04-26 revision 54774) [x86_64-linux]. Both calls result in "\xE5"

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset trunk|r59763.

---

string.c: fix false coderange

• string.c (rb_enc_str_scrub): enc can differ from the actual
  encoding of the string, the cached coderange is useless then.
  [ruby-core:82674] [Bug #13874]

[nobu (Nobuyoshi Nakada)]

• Related to Bug #12431: Strange behavior of String#encode('UTF-8', 'UTF-8', ...) when the encoding of the source string is not UTF-8 added

[nobu (Nobuyoshi Nakada)]

• Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: REQUIRED, 2.3: REQUIRED, 2.4: REQUIRED

2.2.6 and later need the backport.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.2: REQUIRED, 2.3: REQUIRED, 2.4: REQUIRED to 2.2: REQUIRED, 2.3: REQUIRED, 2.4: DONE

ruby_2_4 r59822 merged revision(s) 59763.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.2: REQUIRED, 2.3: REQUIRED, 2.4: DONE to 2.2: REQUIRED, 2.3: DONE, 2.4: DONE

ruby_2_3 r59883 merged revision(s) 59763.