Bug #8092

[patch] gc: improve accuracy of objspace_live_num() and allocated/freed counters

Added by Aman Gupta about 1 year ago. Updated about 1 year ago.

[ruby-core:53392]
Status:Closed
Priority:Normal
Assignee:Narihiro Nakamura
Category:core
Target version:-
ruby -v:ruby 2.1.0dev (2013-03-14 trunk 39748) [x86_64-darwin12.2.1] Backport:

Description

Test with large rails app:

ruby -e'
require "./config/environment"

stat, count = {}, {}
GC.start
GC.stat(stat)
ObjectSpace.count_objects(count)

printf "%d == %d\n", stat[:heap_live_num], count[:TOTAL]-count[:FREE]

'

Without patch:

632974 == 628506

With patch:

628506 == 628506

diff --git a/gc.c b/gc.c
index bd95073..48f9470 100644
--- a/gc.c
+++ b/gc.c
@@ -1432,10 +1432,8 @@ finalizelist(rbobjspacet *objspace, RVALUE *p)
run
final(objspace, (VALUE)p);
if (!FLTEST(p, FLSINGLETON)) { /* not freeing page */
addslotlocalfreelist(objspace, p);
- if (!is
lazysweeping(objspace)) {
- objspace->total
freedobjectnum++;
- objspace->heap.freenum++;
- }
+ objspace->total
freedobjectnum++;
+ objspace->heap.freenum++;
}
else {
struct heaps
slot *slot = (struct heapsslot *)(VALUE)RDATA(p)->dmark;
@@ -1939,9 +1937,9 @@ slot
sweep(rbobjspacet *objspace, struct heapsslot *sweepslot)
else {
sweepslot->freenext = NULL;
}
- objspace->totalfreedobjectnum += freednum;
objspace->heap.freenum += freednum + emptynum;
}
+ objspace->total
freedobjectnum += freednum;
objspace->heap.final
num += final_num;

 if (deferred_final_list && !finalizing) {

@@ -2965,11 +2963,11 @@ rbgcforcerecycle(VALUE p)
rb
objspacet *objspace = &rbobjspace;
struct heaps_slot *slot;

  • objspace->totalfreedobjectnum++; if (MARKEDINBITMAP(GETHEAPBITMAP(p), p)) { addslotlocalfreelist(objspace, (RVALUE *)p); } else {
  • objspace->totalfreedobjectnum++; objspace->heap.freenum++; slot = addslotlocalfreelist(objspace, (RVALUE *)p); if (slot->freenext == NULL) {

Related issues

Related to Backport200 - Backport #8147: Backport r39812 Closed 03/22/2013

Associated revisions

Revision 39812
Added by nari about 1 year ago

  • gc.c: Improve accuracy of objspacelivenum() and allocated/freed counters. patched by tmm1(Aman Gupta). [Bug #8092]

History

#1 Updated by Narihiro Nakamura about 1 year ago

  • Category set to core
  • Assignee set to Narihiro Nakamura

#2 Updated by Narihiro Nakamura about 1 year ago

Thank you for bug report.

But, the following test case is failure.

I think ko1-san is the implementer of GC.stat.
ko1-san, what do you think?

=== start ===
$ git diff test
diff --git a/test/ruby/testgc.rb b/test/ruby/testgc.rb
index bed58f3..8cb036a 100644
--- a/test/ruby/testgc.rb
+++ b/test/ruby/test
gc.rb
@@ -64,6 +64,15 @@ class TestGc < Test::Unit::TestCase
assertequal(arg, res)
assert
equal(false, res.empty?)
assertkindof(Integer, res[:count])
+
+ stat, count = {}, {}
+ GC.start
+ GC.stat(stat)
+ ObjectSpace.countobjects(count)
+ puts ""
+ p stat
+ p count
+ assert
equal(stat[:heaplivenum], count[:TOTAL]-count[:FREE])
end

def testsingletonmethod

$ % make test-all TESTS='ruby/testdefined.rb ruby/testinteger.rb ruby/testobjectspace.rb ruby/testsprintfcomb.rb ruby/testargf.rb ruby/testfiber.rb ruby/testgc.rb'
....

Running tests:

[ 91/131] TestGc#teststat
{:count=>351, :heap
used=>684, :heaplength=>1139, :heapincrement=>399, :heaplivenum=>46061, :heapfreenum=>152639, :heapfinalnum=>0, :totalallocatedobject=>402509, :totalfreedobject=>356448}
{:TOTAL=>198553, :FREE=>152639, :TOBJECT=>31312, :TCLASS=>655, :TMODULE=>56, :TFLOAT=>10, :TSTRING=>8425, :TREGEXP=>146, :TARRAY=>1966, :THASH=>49, :TSTRUCT=>10, :TBIGNUM=>25, :TFILE=>10, :TDATA=>2225, :TMATCH=>22, :TCOMPLEX=>1, :TNODE=>934, :TICLASS=>68}
= 0.01 s
1) Failure:
teststat(TestGc) [/home/nari/source/ruby/ruby-git/test/ruby/testgc.rb:75]:
expected but was
.

Finished tests in 6.628281s, 19.7638 tests/s, 657.9383 assertions/s.
131 tests, 4361 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 2.1.0dev (2013-03-14 trunk 38552) [x86_64-linux]
make: *** [yes-test-all] Error 1
=== end ===

#3 Updated by Aman Gupta about 1 year ago

This test is passing for me on trunk. Before r39811 it is failing.

[ 92/132] TestGc#teststat

{:count=>357, :heap
used=>421, :heaplength=>811, :heapincrement=>283, :heaplivenum=>14452, :heapfreenum=>156816, :heapfinalnum=>0, :totalallocatedobject=>389190, :totalfreedobject=>374738}
{:TOTAL=>171268, :FREE=>156816, :TOBJECT=>46, :TCLASS=>655, :TMODULE=>56, :TFLOAT=>10, :TSTRING=>8268, :TREGEXP=>145, :TARRAY=>1947, :THASH=>42, :TSTRUCT=>9, :TBIGNUM=>25, :TFILE=>3, :TDATA=>2221, :TMATCH=>21, :TCOMPLEX=>1, :TNODE=>935, :TICLASS=>68}
Finished tests in 3.297242s, 40.0335 tests/s, 1313.5220 assertions/s.

132 tests, 4331 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.1.0dev (2013-03-18 trunk 39811) [x86_64-linux]

#4 Updated by Aman Gupta about 1 year ago

With this updated patch the test is passing both before and after r39811. Problem was in finalize_list. After r39811 removing heap is less common, so the failure did not appear.

diff --git a/gc.c b/gc.c
index 2afd311..e72b198 100644
--- a/gc.c
+++ b/gc.c
@@ -1431,12 +1431,10 @@ finalizelist(rbobjspacet *objspace, RVALUE *p)
while (p) {
RVALUE *tmp = p->as.free.next;
run
final(objspace, (VALUE)p);
+ objspace->totalfreedobjectnum++;
if (!FL
TEST(p, FLSINGLETON)) { /* not freeing page */
add
slotlocalfreelist(objspace, p);
- if (!islazysweeping(objspace)) {
- objspace->totalfreedobjectnum++;
- objspace->heap.free
num++;
- }
+ objspace->heap.freenum++;
}
else {
struct heaps
slot *slot = (struct heapsslot *)(VALUE)RDATA(p)->dmark;
@@ -1940,9 +1938,9 @@ slot
sweep(rbobjspacet *objspace, struct heapsslot *sweepslot)
else {
sweepslot->freenext = NULL;
}
- objspace->totalfreedobjectnum += freednum;
objspace->heap.freenum += freednum + emptynum;
}
+ objspace->total
freedobjectnum += freednum;
objspace->heap.final
num += final_num;

 if (deferred_final_list && !finalizing) {

@@ -2969,11 +2967,11 @@ rbgcforcerecycle(VALUE p)
rb
objspacet *objspace = &rbobjspace;
struct heaps_slot *slot;

  • objspace->totalfreedobjectnum++; if (MARKEDINBITMAP(GETHEAPBITMAP(p), p)) { addslotlocalfreelist(objspace, (RVALUE *)p); } else {
  • objspace->totalfreedobjectnum++; objspace->heap.freenum++; slot = addslotlocalfreelist(objspace, (RVALUE *)p); if (slot->freenext == NULL) { diff --git a/test/ruby/testgc.rb b/test/ruby/testgc.rb index 90c4787..b1e52fc 100644 --- a/test/ruby/testgc.rb +++ b/test/ruby/testgc.rb @@ -64,6 +64,12 @@ class TestGc < Test::Unit::TestCase assertequal(arg, res) assertequal(false, res.empty?) assertkindof(Integer, res[:count]) +
  • stat, count = {}, {}
  • GC.start
  • GC.stat(stat)
  • ObjectSpace.count_objects(count)
  • assertequal(count[:TOTAL]-count[:FREE], stat[:heaplive_num])
    end

    def testsingletonmethod

#5 Updated by Narihiro Nakamura about 1 year ago

Wow, thanks!! I'll commit it soon.

#6 Updated by Narihiro Nakamura about 1 year ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r39812.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • gc.c: Improve accuracy of objspacelivenum() and allocated/freed counters. patched by tmm1(Aman Gupta). [Bug #8092]

Also available in: Atom PDF