Project

General

Profile

Actions

Bug #8983

closed

[PATCH] GC.stat[:heap_free_num] returns number of unused slots on heap

Added by tmm1 (Aman Karmani) over 10 years ago. Updated over 10 years ago.

Status:
Closed
Target version:
ruby -v:
ruby 2.1.0dev (2013-10-03 trunk 43129) [x86_64-darwin12.4.1]
[ruby-core:57633]

Description

In ruby 1.9 and 2.0, gc_stat() used to call rest_sweep() so heap_free_num always equals the number of free slots (ObjectSpace.count_objects[:FREE]). In 2.1 the rest_sweep() was removed, so free_num is not always accurate. This patch fixes compatibility with the previous behavior.

With this patch, it is also easier to perform "out-of-band" GC in unicorn and passenger. For example: if your application requires 100k objects per request, then you can run the following in-between every request:

GC.start if GC.stat[:heap_free_num] < 100_000

I also added a objspace->heap.limit because HEAP_OBJ_LIMIT*heap_used is not accurate due to memory re-alignment in heap_assign_slot.

https://github.com/tmm1/ruby/compare/gc-stat-free-num
https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

Updated by naruse (Yui NARUSE) over 10 years ago

  • Status changed from Open to Assigned
  • Assignee changed from tmm1 (Aman Karmani) to ko1 (Koichi Sasada)

Updated by tmm1 (Aman Karmani) over 10 years ago

ko1-san, if you agree with this patch I can commit it.

Updated by ko1 (Koichi Sasada) over 10 years ago

(2013/10/03 18:29), naruse (Yui NARUSE) wrote:

https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

One point.

@@ -1787,6 +1791,7 @@ struct os_each_struct {
else {
struct heap_slot *slot = (struct heap_slot *)(VALUE)RDATA(p)->dmark;
slot->limit--;

  •    heap_limit--;
    
    }
    p = tmp;
    }

Should not reduce heap_limit here.

@@ -2343,6 +2348,7 @@ struct force_finalize_list {
RDATA(pp)->dmark = (void (*)(void ))(VALUE)sweep_slot;
pp->as.free.flags |= FL_SINGLETON; /
freeing page mark */
}

  •    heap_limit -= (freed_num + empty_num);
       sweep_slot->limit = final_num;
       unlink_heap_slot(objspace, sweep_slot);
    
    }

Should set "heap_limit -= slot->limit".
This slot is already unavaialble from Ruby level.

--
// SASADA Koichi at atdot dot net

Updated by tmm1 (Aman Karmani) over 10 years ago

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

Updated by ko1 (Koichi Sasada) over 10 years ago

(2013/10/04 10:09), tmm1 (Aman Gupta) wrote:

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

I got it.

objspace_live_num() should return the count including zombie (waiting to
finallization) because they are still living (living dead!!).

However, we can eliminate zombiees from live_num() because they are not
living normal objects.

Two approachs?

(1) eliminate zombies from live_num

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

--
// SASADA Koichi at atdot dot net

Updated by tmm1 (Aman Karmani) over 10 years ago

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

I like this approach. This will be useful also for debugging finalizer leaks.

Once we replace heap_free_num with this formula, I would like to rename the existing heap.free_num counter. Do you have any suggestion for a better name? Maybe heap.reused_num or heap.swept_num

Updated by tmm1 (Aman Karmani) over 10 years ago

Actually we already have heap.final_num. Is this the same as zombie_num?

Actions #8

Updated by tmm1 (Aman Karmani) over 10 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r43138.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots [ruby-core:57633] [Bug #8983]
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0