Bug #19739
closedString coderange not cleared by String#slice!
Description
Hello,
Key cannot be found in a Hash when slice! method is applied to the key. I cannot reproduce this behaviour with sub! or gsub! methods:
hash = {}
key = "ABC OÜ"
key.slice!(/ oü$/i)
# key.sub!(/ oü$/i, '')
# key.gsub!(/ oü$/i, '')
hash[key] = true
"#{key} in #{hash}: value is #{hash[key]}"
=> "ABC in {\"ABC\"=>true}: value is "
Also it seems that ruby 2.7.6 and 3.2.2 don't have this issue.
Updated by byroot (Jean Boussier) 11 months ago
That is very interesting.
The two strings are identical (aside from one being frozen):
require "objspace"
hash = {}
key = "ABC OÜ"
p key.hash
key.slice!(/ oü$/i)
p key.hash
puts ObjectSpace.dump(key)
hash[key] = true
p "#{key} in #{hash}: value is #{hash[key].inspect}"
puts ObjectSpace.dump(hash.keys.first)
p [hash.keys.first, key, hash.keys.first == key]
p [hash.keys.first.hash, key.hash]
1883393806954673666
-2878200511277543492
{"address":"0x10129f048", "type":"STRING", "class":"0x1011c2418", "embedded":true, "bytesize":3, "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
"ABC in {\"ABC\"=>true}: value is nil"
{"address":"0x10129ebc0", "type":"STRING", "class":"0x1011c2418", "frozen":true, "embedded":true, "fstring":true, "bytesize":3, "value":"ABC", "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
["ABC", "ABC", true]
[-2878200511277543491, -2878200511277543491]
The hash code is the same, and ==
returns true.
I had a quick look at recent changes in String#slice!
that could explain why it's fixed in 3.2, but didn't spot anything yet.
What is interesting though is that the first access to key.hash
returns -2878200511277543492
, but then later it consistently returns -2878200511277543491
. I wonder if there is an off by one bug somewhere.
Updated by byroot (Jean Boussier) 11 months ago
- Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED
I suspect https://github.com/ruby/ruby/pull/5867 is what fixed it, but don't have time to confirm just yet.
Updated by byroot (Jean Boussier) 11 months ago
- Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED
Ok, so after bisecting it appears that what fixed it was: https://github.com/ruby/ruby/commit/b0b9f7201acab05c2a3ad92c3043a1f01df3e17f
This won't be easy to backport though, as it was a performance patch. Especially since it had to be fixed later with https://github.com/ruby/ruby/pull/7437/commits/2d0c804e5640475202b7c24559bbe1b151367ebf
We might need to craft a patch specifically to fix that bug.
Updated by byroot (Jean Boussier) 11 months ago
OK fix is here: https://github.com/ruby/ruby/pull/7965
Updated by byroot (Jean Boussier) 11 months ago
cc @nagachika (Tomoyuki Chikanaga) is a PR targeting ruby_3_1
OK? Or should I submit the patch differently?
Updated by byroot (Jean Boussier) 11 months ago
- Subject changed from Key cannot be found in a Hash when slice! method is applied to the key to String coderange not cleared by String#slice!
Updated by hsbt (Hiroshi SHIBATA) 11 months ago
@byroot (Jean Boussier) ruby_3_1
is handled by @usa (Usaku NAKAMURA) now.
Updated by hsbt (Hiroshi SHIBATA) 11 months ago
- Status changed from Open to Closed
Updated by byroot (Jean Boussier) 11 months ago
The commit to backport would be: https://github.com/Shopify/ruby/commit/3b351ee62d4206bb72301c2e98dcb173f1e35be7
Updated by usa (Usaku NAKAMURA) 10 months ago
- Backport changed from 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: DONE, 3.2: DONTNEED
ruby_3_1 7e926c4d8c2e785d174161b38a8a688b19ea4b99.