Feature #15349
closedUse a shared array for the `duparray` instruction
Description
In this example code:
def foo
[1, 2, 3, 4]
end
The array literal uses a duparray instruction. Before this patch,
rb_ary_resurrect would malloc and memcpy a new array buffer. This
patch changes rb_ary_resurrect to use ary_make_partial so that the
new array object shares the underlying buffer with the array stored in
the instruction sequences.
Before this patch, the new array object is not shared:
$ ruby -r objspace -e'p ObjectSpace.dump([1, 2, 3, 4])'
"{\"address\":\"0x7fa2718372d0\", \"type\":\"ARRAY\", \"class\":\"0x7fa26f8b0010\", \"length\":4, \"memsize\":72, \"flags\":{\"wb_protected\":true}}\n"
After this patch:
$ ./ruby -r objspace -e'p ObjectSpace.dump([1, 2, 3, 4])'
"{\"address\":\"0x7f9a76883638\", \"type\":\"ARRAY\", \"class\":\"0x7f9a758af900\", \"length\":4, \"shared\":true, \"references\":[\"0x7f9a768837c8\"], \"memsize\":40, \"flags\":{\"wb_protected\":true}}\n"
I wrote a test program:
def foo
[1, 2, 3, 4]
end
list = []
10000.times { list << foo }
GC.start
puts "ready #{$$}"
system "malloc_history #{$$} -allEvents > #{RUBY_VERSION}-#{ARGV[0]}.log" # MacOS specific
This test program uses a MacOS specific tool to get a list of all malloc / free calls (should be able to do the same with valgrind, I just don't know the command). I compared trunk, trunk + this patch, and Ruby 2.5.3. All tests disable RubyGems.
Here is a graph of the results:
The X axis is sample number, and the Y axis is total bytes the process is using at that sample. Each sample is a call to malloc, so the closer the line is to the origin (0, 0), the better (it means fewer samples and less live memory).
After this patch there are fewer calls to malloc than in either trunk or 2.5 for this test program.
If I modify the test program to be like this:
def foo
[
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
]
end
list = []
10000.times { list << foo }
GC.start
puts "ready #{$$}"
system "malloc_history #{$$} -allEvents > #{RUBY_VERSION}-#{ARGV[0]}.log"
The difference in memory usage becomes more pronounced:
I noticed that strings already use this technique:
$ ruby -v --disable-gems -robjspace -e'p ObjectSpace.dump("abcdefghijklmnopqrstuvwxyz")'
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
"{\"address\":\"0x00007fad18051528\", \"type\":\"STRING\", \"class\":\"0x00007fad180d3c08\", \"shared\":true, \"encoding\":\"UTF-8\", \"references\":[\"0x00007fad18051640\"], \"memsize\":40, \"flags\":{\"wb_protected\":true}}\n"
I was going to apply this patch, but it modifies rb_ary_resurrect
and I wanted to double check. rb_str_resurrect
will create shared strings, so it seems OK for rb_ary_resurrect
to create shared arrays, IMO.
Does anyone have opinions on this? If not, I'll just apply the patch.
(Also I noticed trunk seems to allocate a lot more memory at boot)
Thanks!
Files