Bug #1587: Problem with string sharing

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 1.9.2dev (2009-06-08) [i386-mswin32_80]

[quetzal (Quet Zal)]

=begin
I have a problem building HEAD of Ruby 1.9 on windows if linking to debug libraries (/MDd).
After some debugging I've been able to reduce problem to this piece of code:

s1 = String.new() # make empty string
10.times { s1 << 'abc' } # make sure string is not embedded in RString
p "S1: ", s1
s2 = s1.dup # make s2 and s1 share same buffer
s1.gsub!("abc", "xxx"); # gsub! discards s1, making s2 point to non-valid memory
p "S1: ", s1 # ok
p "S2: ", s2 # ouch, some garbage

If debug versions of malloc/free are used, last p "S2: "... shows garbage. This is because in s2 RString.as.heap.ptr references to memory that has been already freed.
It happens like that:
after s2 = s1.dup we have two RString objects, with s2 sharing a data buffer with s1, meaning s2.as.heap.ptr == s1.as.heap.ptr and s2.as.heap.aux.shared = s1.
gsub! leads to following call sequence rb_str_gsub_bang -> str_gsub -> rb_str_shared_replace -> str_discard
str_discard is called on s1 and calls xfree -> ruby_xfree -> vm_xfree -> free on s1.as.heap.ptr buffer. If debug version of free is used, freed memory is filled
with some constant, but we STILL have s2.as.heap.ptr pointing to this (already freed) memory, which is obviously wrong.
There's some problem reproducing it in linux because linux memory allocator does not touch freed memory and it can be used just fine (unless its allocated for something else, which is a rare case).

Sorry for such long explanation, I'm messing with ruby internals for only few days and can hardly believe such bad bugs exists in ruby, so there's really big chance I'm overlooking something.
=end

[rue (Eero Saynatkari)]

=begin
Excerpts from redmine message on Mon Jun 08 15:07:21 +0300 2009:

Bug #1587: Problem with string sharing
http://redmine.ruby-lang.org/issues/show/1587

Author: Quet Zal
Status: Open, Priority: Normal
ruby -v: ruby 1.9.2dev (2009-06-08) [i386-mswin32_80]

I have a problem building HEAD of Ruby 1.9 on windows if linking to debug
libraries (/MDd).
After some debugging I've been able to reduce problem to this piece of code:

s1 = String.new() # make empty string
10.times { s1 << 'abc' } # make sure string is not embedded in RString
p "S1: ", s1
s2 = s1.dup # make s2 and s1 share same buffer
s1.gsub!("abc", "xxx"); # gsub! discards s1, making s2 point to non-valid
memory
p "S1: ", s1 # ok
p "S2: ", s2 # ouch, some garbage

If debug versions of malloc/free are used, last p "S2: "... shows garbage. This
is because in s2 RString.as.heap.ptr references to memory that has been already
freed.
It happens like that:
after s2 = s1.dup we have two RString objects, with s2 sharing a data buffer
with s1, meaning s2.as.heap.ptr == s1.as.heap.ptr and s2.as.heap.aux.shared =
s1.

In particular, the dup is only flagging the copy as shared
which means that the original does not know it should not
release the memory. (Except for rb_str_new4(), which does
the exact opposite.)

gsub! leads to following call sequence rb_str_gsub_bang -> str_gsub ->
rb_str_shared_replace -> str_discard
str_discard is called on s1 and calls xfree -> ruby_xfree -> vm_xfree -> free
on s1.as.heap.ptr buffer. If debug version of free is used, freed memory is
filled
with some constant, but we STILL have s2.as.heap.ptr pointing to this (already
freed) memory, which is obviously wrong.
There's some problem reproducing it in linux because linux memory allocator
does not touch freed memory and it can be used just fine (unless its allocated
for something else, which is a rare case).

Sorry for such long explanation, I'm messing with ruby internals for only few
days and can hardly believe such bad bugs exists in ruby, so there's really big
chance I'm overlooking something.

Indeed. At the very least, both strings should be flagged
as being shared. Depending on the GC, it might be needed
to implement something fancier if accurate collection of
the cstr memory is needed.

As post scriptum, with due respect, the string.c code is
pretty bad for anyone to follow, with questionable variable
naming and tons of similarly named functions (which may or
may not be unused, obsolete, or valid in some contexts
only.)

Eero

Magic is insufficiently advanced technology.

=end

[quetzal (Quet Zal)]

=begin
Here's even more simplified example how to reproduce this bug. (First one didnt work on linux).

s1 = String.new()
10.times { s1 << "abc" }
p s1
s2 = s1.dup
p s2
s1.replace("xxx");
p s1
p s2

Run with valgrind --free-fill=AA ./miniruby test.rb

=end

[wanabe (_ wanabe)]

• Status changed from Open to Closed

=begin
Thank you for the report and sorry for late response.
This issue seems to be fixed now, on linux at least.
By r23669, I guess.
=end