Bug #8092

[patch] gc: improve accuracy of objspace_live_num() and allocated/freed counters

Added by Aman Gupta about 2 years ago. Updated about 2 years ago.

[ruby-core:53392]
Status:Closed
Priority:Normal
Assignee:Narihiro Nakamura
ruby -v:ruby 2.1.0dev (2013-03-14 trunk 39748) [x86_64-darwin12.2.1] Backport:

Description

Test with large rails app:

ruby -e'
require "./config/environment"

stat, count = {}, {}
GC.start
GC.stat(stat)
ObjectSpace.count_objects(count)

printf "%d == %d\n", stat[:heap_live_num], count[:TOTAL]-count[:FREE]

'

Without patch:

632974 == 628506

With patch:

628506 == 628506

diff --git a/gc.c b/gc.c
index bd95073..48f9470 100644
--- a/gc.c
+++ b/gc.c
@@ -1432,10 +1432,8 @@ finalize_list(rb_objspace_t objspace, RVALUE *p)
run_final(objspace, (VALUE)p);
if (!FL_TEST(p, FL_SINGLETON)) { /
not freeing page */
add_slot_local_freelist(objspace, p);
- if (!is_lazy_sweeping(objspace)) {
- objspace->total_freed_object_num++;
- objspace->heap.free_num++;
- }
+ objspace->total_freed_object_num++;
+ objspace->heap.free_num++;
}
else {
struct heaps_slot *slot = (struct heaps_slot *)(VALUE)RDATA(p)->dmark;
@@ -1939,9 +1937,9 @@ slot_sweep(rb_objspace_t *objspace, struct heaps_slot *sweep_slot)
else {
sweep_slot->free_next = NULL;
}
- objspace->total_freed_object_num += freed_num;
objspace->heap.free_num += freed_num + empty_num;
}
+ objspace->total_freed_object_num += freed_num;
objspace->heap.final_num += final_num;

 if (deferred_final_list && !finalizing) {

@@ -2965,11 +2963,11 @@ rb_gc_force_recycle(VALUE p)
rb_objspace_t *objspace = &rb_objspace;
struct heaps_slot *slot;

  • objspace->total_freed_object_num++; if (MARKED_IN_BITMAP(GET_HEAP_BITMAP(p), p)) { add_slot_local_freelist(objspace, (RVALUE *)p); } else {
  • objspace->total_freed_object_num++; objspace->heap.free_num++; slot = add_slot_local_freelist(objspace, (RVALUE *)p); if (slot->free_next == NULL) {

Related issues

Related to Backport200 - Backport #8147: Backport r39812 Closed 03/22/2013

Associated revisions

Revision 39812
Added by nari about 2 years ago

  • gc.c: Improve accuracy of objspace_live_num() and allocated/freed counters. patched by tmm1(Aman Gupta). [Bug #8092]

Revision 39812
Added by nari about 2 years ago

  • gc.c: Improve accuracy of objspace_live_num() and allocated/freed counters. patched by tmm1(Aman Gupta). [Bug #8092]

History

#1 Updated by Narihiro Nakamura about 2 years ago

  • Category set to core
  • Assignee set to Narihiro Nakamura

#2 Updated by Narihiro Nakamura about 2 years ago

Thank you for bug report.

But, the following test case is failure.

I think ko1-san is the implementer of GC.stat.
ko1-san, what do you think?

=== start ===
$ git diff test
diff --git a/test/ruby/test_gc.rb b/test/ruby/test_gc.rb
index bed58f3..8cb036a 100644
--- a/test/ruby/test_gc.rb
+++ b/test/ruby/test_gc.rb
@@ -64,6 +64,15 @@ class TestGc < Test::Unit::TestCase
assert_equal(arg, res)
assert_equal(false, res.empty?)
assert_kind_of(Integer, res[:count])
+
+ stat, count = {}, {}
+ GC.start
+ GC.stat(stat)
+ ObjectSpace.count_objects(count)
+ puts ""
+ p stat
+ p count
+ assert_equal(stat[:heap_live_num], count[:TOTAL]-count[:FREE])
end

def test_singleton_method

$ % make test-all TESTS='ruby/test_defined.rb ruby/test_integer.rb ruby/test_objectspace.rb ruby/test_sprintf_comb.rb ruby/test_argf.rb ruby/test_fiber.rb ruby/test_gc.rb'
....

Running tests:

[ 91/131] TestGc#test_stat
{:count=>351, :heap_used=>684, :heap_length=>1139, :heap_increment=>399, :heap_live_num=>46061, :heap_free_num=>152639, :heap_final_num=>0, :total_allocated_object=>402509, :total_freed_object=>356448}
{:TOTAL=>198553, :FREE=>152639, :T_OBJECT=>31312, :T_CLASS=>655, :T_MODULE=>56, :T_FLOAT=>10, :T_STRING=>8425, :T_REGEXP=>146, :T_ARRAY=>1966, :T_HASH=>49, :T_STRUCT=>10, :T_BIGNUM=>25, :T_FILE=>10, :T_DATA=>2225, :T_MATCH=>22, :T_COMPLEX=>1, :T_NODE=>934, :T_ICLASS=>68}
= 0.01 s
1) Failure:
test_stat(TestGc) [/home/nari/source/ruby/ruby-git/test/ruby/test_gc.rb:75]:
expected but was
.

Finished tests in 6.628281s, 19.7638 tests/s, 657.9383 assertions/s.
131 tests, 4361 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 2.1.0dev (2013-03-14 trunk 38552) [x86_64-linux]
make: *** [yes-test-all] Error 1
=== end ===

#3 Updated by Aman Gupta about 2 years ago

This test is passing for me on trunk. Before r39811 it is failing.

[ 92/132] TestGc#test_stat

{:count=>357, :heap_used=>421, :heap_length=>811, :heap_increment=>283, :heap_live_num=>14452, :heap_free_num=>156816, :heap_final_num=>0, :total_allocated_object=>389190, :total_freed_object=>374738}
{:TOTAL=>171268, :FREE=>156816, :T_OBJECT=>46, :T_CLASS=>655, :T_MODULE=>56, :T_FLOAT=>10, :T_STRING=>8268, :T_REGEXP=>145, :T_ARRAY=>1947, :T_HASH=>42, :T_STRUCT=>9, :T_BIGNUM=>25, :T_FILE=>3, :T_DATA=>2221, :T_MATCH=>21, :T_COMPLEX=>1, :T_NODE=>935, :T_ICLASS=>68}
Finished tests in 3.297242s, 40.0335 tests/s, 1313.5220 assertions/s.

132 tests, 4331 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.1.0dev (2013-03-18 trunk 39811) [x86_64-linux]

#4 Updated by Aman Gupta about 2 years ago

With this updated patch the test is passing both before and after r39811. Problem was in finalize_list. After r39811 removing heap is less common, so the failure did not appear.

diff --git a/gc.c b/gc.c
index 2afd311..e72b198 100644
--- a/gc.c
+++ b/gc.c
@@ -1431,12 +1431,10 @@ finalize_list(rb_objspace_t objspace, RVALUE *p)
while (p) {
RVALUE *tmp = p->as.free.next;
run_final(objspace, (VALUE)p);
+ objspace->total_freed_object_num++;
if (!FL_TEST(p, FL_SINGLETON)) { /
not freeing page */
add_slot_local_freelist(objspace, p);
- if (!is_lazy_sweeping(objspace)) {
- objspace->total_freed_object_num++;
- objspace->heap.free_num++;
- }
+ objspace->heap.free_num++;
}
else {
struct heaps_slot *slot = (struct heaps_slot *)(VALUE)RDATA(p)->dmark;
@@ -1940,9 +1938,9 @@ slot_sweep(rb_objspace_t *objspace, struct heaps_slot *sweep_slot)
else {
sweep_slot->free_next = NULL;
}
- objspace->total_freed_object_num += freed_num;
objspace->heap.free_num += freed_num + empty_num;
}
+ objspace->total_freed_object_num += freed_num;
objspace->heap.final_num += final_num;

 if (deferred_final_list && !finalizing) {

@@ -2969,11 +2967,11 @@ rb_gc_force_recycle(VALUE p)
rb_objspace_t *objspace = &rb_objspace;
struct heaps_slot *slot;

  • objspace->total_freed_object_num++; if (MARKED_IN_BITMAP(GET_HEAP_BITMAP(p), p)) { add_slot_local_freelist(objspace, (RVALUE *)p); } else {
  • objspace->total_freed_object_num++; objspace->heap.free_num++; slot = add_slot_local_freelist(objspace, (RVALUE *)p); if (slot->free_next == NULL) { diff --git a/test/ruby/test_gc.rb b/test/ruby/test_gc.rb index 90c4787..b1e52fc 100644 --- a/test/ruby/test_gc.rb +++ b/test/ruby/test_gc.rb @@ -64,6 +64,12 @@ class TestGc < Test::Unit::TestCase assert_equal(arg, res) assert_equal(false, res.empty?) assert_kind_of(Integer, res[:count]) +
  • stat, count = {}, {}
  • GC.start
  • GC.stat(stat)
  • ObjectSpace.count_objects(count)
  • assert_equal(count[:TOTAL]-count[:FREE], stat[:heap_live_num]) end

def test_singleton_method

#5 Updated by Narihiro Nakamura about 2 years ago

Wow, thanks!! I'll commit it soon.

#6 Updated by Narihiro Nakamura about 2 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r39812.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • gc.c: Improve accuracy of objspace_live_num() and allocated/freed counters. patched by tmm1(Aman Gupta). [Bug #8092]

Also available in: Atom PDF