Bug #19739: String coderange not cleared by String#slice!

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 3.1.4
• Backport: 3.0: REQUIRED, 3.1: DONE, 3.2: DONTNEED

[ilya.andreyuk (Ilya Andreyuk)]

Hello,
Key cannot be found in a Hash when slice! method is applied to the key. I cannot reproduce this behaviour with sub! or gsub! methods:

hash = {}
key = "ABC OÜ"
key.slice!(/ oü$/i) 
# key.sub!(/ oü$/i, '')
# key.gsub!(/ oü$/i, '')
hash[key] = true
"#{key} in #{hash}: value is #{hash[key]}"

=> "ABC in {\"ABC\"=>true}: value is "

Also it seems that ruby 2.7.6 and 3.2.2 don't have this issue.

[byroot (Jean Boussier)]

That is very interesting.

The two strings are identical (aside from one being frozen):

require "objspace"

hash = {}
key = "ABC OÜ"
p key.hash
key.slice!(/ oü$/i) 

p key.hash
puts ObjectSpace.dump(key)
hash[key] = true
p "#{key} in #{hash}: value is #{hash[key].inspect}"
puts ObjectSpace.dump(hash.keys.first)
p [hash.keys.first, key, hash.keys.first == key]
p [hash.keys.first.hash, key.hash]

1883393806954673666
-2878200511277543492
{"address":"0x10129f048", "type":"STRING", "class":"0x1011c2418", "embedded":true, "bytesize":3, "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
"ABC in {\"ABC\"=>true}: value is nil"
{"address":"0x10129ebc0", "type":"STRING", "class":"0x1011c2418", "frozen":true, "embedded":true, "fstring":true, "bytesize":3, "value":"ABC", "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
["ABC", "ABC", true]
[-2878200511277543491, -2878200511277543491]

The hash code is the same, and == returns true.

I had a quick look at recent changes in String#slice! that could explain why it's fixed in 3.2, but didn't spot anything yet.

What is interesting though is that the first access to key.hash returns -2878200511277543492, but then later it consistently returns -2878200511277543491. I wonder if there is an off by one bug somewhere.

[byroot (Jean Boussier)]

• Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED

I suspect https://github.com/ruby/ruby/pull/5867 is what fixed it, but don't have time to confirm just yet.

[byroot (Jean Boussier)]

• Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED

Ok, so after bisecting it appears that what fixed it was: https://github.com/ruby/ruby/commit/b0b9f7201acab05c2a3ad92c3043a1f01df3e17f

This won't be easy to backport though, as it was a performance patch. Especially since it had to be fixed later with https://github.com/ruby/ruby/pull/7437/commits/2d0c804e5640475202b7c24559bbe1b151367ebf

We might need to craft a patch specifically to fix that bug.

[byroot (Jean Boussier)]

OK fix is here: https://github.com/ruby/ruby/pull/7965

[byroot (Jean Boussier)]

cc @nagachika (Tomoyuki Chikanaga) is a PR targeting ruby_3_1 OK? Or should I submit the patch differently?

[byroot (Jean Boussier)]

• Subject changed from Key cannot be found in a Hash when slice! method is applied to the key to String coderange not cleared by String#slice!

[hsbt (Hiroshi SHIBATA)]

@byroot (Jean Boussier) ruby_3_1 is handled by @usa (Usaku NAKAMURA) now.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Closed

[byroot (Jean Boussier)]

The commit to backport would be: https://github.com/Shopify/ruby/commit/3b351ee62d4206bb72301c2e98dcb173f1e35be7

[usa (Usaku NAKAMURA)]

• Backport changed from 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: DONE, 3.2: DONTNEED

ruby_3_1 7e926c4d8c2e785d174161b38a8a688b19ea4b99.