Bug #8983

[PATCH] GC.stat[:heap_free_num] returns number of unused slots on heap

Added by Aman Gupta almost 2 years ago. Updated almost 2 years ago.

[ruby-core:57633]
Status:Closed
Priority:Normal
Assignee:Koichi Sasada
ruby -v:ruby 2.1.0dev (2013-10-03 trunk 43129) [x86_64-darwin12.4.1] Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN

Description

In ruby 1.9 and 2.0, gc_stat() used to call rest_sweep() so heap_free_num always equals the number of free slots (ObjectSpace.count_objects[:FREE]). In 2.1 the rest_sweep() was removed, so free_num is not always accurate. This patch fixes compatibility with the previous behavior.

With this patch, it is also easier to perform "out-of-band" GC in unicorn and passenger. For example: if your application requires 100k objects per request, then you can run the following in-between every request:

GC.start if GC.stat[:heap_free_num] < 100_000

I also added a objspace->heap.limit because HEAP_OBJ_LIMIT*heap_used is not accurate due to memory re-alignment in heap_assign_slot.

https://github.com/tmm1/ruby/compare/gc-stat-free-num
https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

Associated revisions

Revision 43138
Added by Aman Gupta almost 2 years ago

gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots  [Bug #8983]

Revision 43138
Added by Aman Gupta almost 2 years ago

gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots  [Bug #8983]

Revision 43139
Added by Aman Gupta almost 2 years ago

gc.c: add objspace_free_num and make GC.stat[:heap_free_num] use it

* gc.c (objspace_free_num): new method for available/free slots on
  heap.  [Bug #8983]
* gc.c (gc_stat): change heap_free_num definition to use new method.
* test/ruby/test_gc.rb: test for above.

Revision 43139
Added by Aman Gupta almost 2 years ago

gc.c: add objspace_free_num and make GC.stat[:heap_free_num] use it

* gc.c (objspace_free_num): new method for available/free slots on
  heap.  [Bug #8983]
* gc.c (gc_stat): change heap_free_num definition to use new method.
* test/ruby/test_gc.rb: test for above.

History

#1 Updated by Yui NARUSE almost 2 years ago

  • Status changed from Open to Assigned
  • Assignee changed from Aman Gupta to Koichi Sasada

#2 Updated by Aman Gupta almost 2 years ago

ko1-san, if you agree with this patch I can commit it.

#3 Updated by Koichi Sasada almost 2 years ago

(2013/10/03 18:29), naruse (Yui NARUSE) wrote:

https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

One point.

@@ -1787,6 +1791,7 @@ struct os_each_struct {
else {
struct heap_slot *slot = (struct heap_slot *)(VALUE)RDATA(p)->dmark;
slot->limit--;
+ heap_limit--;
}
p = tmp;
}

Should not reduce heap_limit here.

@@ -2343,6 +2348,7 @@ struct force_finalize_list {
RDATA(pp)->dmark = (void ()(void *))(VALUE)sweep_slot;
pp->as.free.flags |= FL_SINGLETON; /
freeing page mark */
}
+ heap_limit -= (freed_num + empty_num);
sweep_slot->limit = final_num;
unlink_heap_slot(objspace, sweep_slot);
}

Should set "heap_limit -= slot->limit".
This slot is already unavaialble from Ruby level.

--
// SASADA Koichi at atdot dot net

#4 Updated by Aman Gupta almost 2 years ago

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

#5 Updated by Koichi Sasada almost 2 years ago

(2013/10/04 10:09), tmm1 (Aman Gupta) wrote:

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

I got it.

objspace_live_num() should return the count including zombie (waiting to
finallization) because they are still living (living dead!!).

However, we can eliminate zombiees from live_num() because they are not
living normal objects.

Two approachs?

(1) eliminate zombies from live_num

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

--
// SASADA Koichi at atdot dot net

#6 Updated by Aman Gupta almost 2 years ago

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

I like this approach. This will be useful also for debugging finalizer leaks.

Once we replace heap_free_num with this formula, I would like to rename the existing heap.free_num counter. Do you have any suggestion for a better name? Maybe heap.reused_num or heap.swept_num

#7 Updated by Aman Gupta almost 2 years ago

Actually we already have heap.final_num. Is this the same as zombie_num?

#8 Updated by Aman Gupta almost 2 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r43138.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots  [Bug #8983]

Also available in: Atom PDF