Bug #8983

[PATCH] GC.stat[:heap_free_num] returns number of unused slots on heap

Added by Aman Gupta 7 months ago. Updated 7 months ago.

[ruby-core:57633]
Status:Closed
Priority:Normal
Assignee:Koichi Sasada
Category:-
Target version:2.1.0
ruby -v:ruby 2.1.0dev (2013-10-03 trunk 43129) [x86_64-darwin12.4.1] Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN

Description

In ruby 1.9 and 2.0, gcstat() used to call restsweep() so heapfreenum always equals the number of free slots (ObjectSpace.countobjects[:FREE]). In 2.1 the restsweep() was removed, so free_num is not always accurate. This patch fixes compatibility with the previous behavior.

With this patch, it is also easier to perform "out-of-band" GC in unicorn and passenger. For example: if your application requires 100k objects per request, then you can run the following in-between every request:

GC.start if GC.stat[:heapfreenum] < 100_000

I also added a objspace->heap.limit because HEAPOBJLIMIT*heapused is not accurate due to memory re-alignment in heapassign_slot.

https://github.com/tmm1/ruby/compare/gc-stat-free-num
https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

Associated revisions

Revision 43138
Added by Aman Gupta 7 months ago

gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots  [Bug #8983]

Revision 43139
Added by Aman Gupta 7 months ago

gc.c: add objspacefreenum and make GC.stat[:heapfreenum] use it

* gc.c (objspace_free_num): new method for available/free slots on
  heap.  [Bug #8983]
* gc.c (gc_stat): change heap_free_num definition to use new method.
* test/ruby/test_gc.rb: test for above.

History

#1 Updated by Yui NARUSE 7 months ago

  • Status changed from Open to Assigned
  • Assignee changed from Aman Gupta to Koichi Sasada

#2 Updated by Aman Gupta 7 months ago

ko1-san, if you agree with this patch I can commit it.

#3 Updated by Koichi Sasada 7 months ago

(2013/10/03 18:29), naruse (Yui NARUSE) wrote:

https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

One point.

@@ -1787,6 +1791,7 @@ struct oseachstruct {
else {
struct heapslot *slot = (struct heapslot *)(VALUE)RDATA(p)->dmark;
slot->limit--;
+ heap_limit--;
}
p = tmp;
}

Should not reduce heap_limit here.

@@ -2343,6 +2348,7 @@ struct forcefinalizelist {
RDATA(pp)->dmark = (void ()(void *))(VALUE)sweepslot;
pp->as.free.flags |= FL
SINGLETON; /
freeing page mark */
}
+ heaplimit -= (freednum + emptynum);
sweep
slot->limit = finalnum;
unlink
heapslot(objspace, sweepslot);
}

Should set "heap_limit -= slot->limit".
This slot is already unavaialble from Ruby level.

--
// SASADA Koichi at atdot dot net

#4 Updated by Aman Gupta 7 months ago

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

#5 Updated by Koichi Sasada 7 months ago

(2013/10/04 10:09), tmm1 (Aman Gupta) wrote:

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

I got it.

objspacelivenum() should return the count including zombie (waiting to
finallization) because they are still living (living dead!!).

However, we can eliminate zombiees from live_num() because they are not
living normal objects.

Two approachs?

(1) eliminate zombies from live_num

(2) count zombie_num

(objspace->heap.limit -
(objspacelivenum(objspace) - zombie_num)

--
// SASADA Koichi at atdot dot net

#6 Updated by Aman Gupta 7 months ago

(2) count zombie_num

(objspace->heap.limit -
(objspacelivenum(objspace) - zombie_num)

I like this approach. This will be useful also for debugging finalizer leaks.

Once we replace heapfreenum with this formula, I would like to rename the existing heap.freenum counter. Do you have any suggestion for a better name? Maybe heap.reusednum or heap.swept_num

#7 Updated by Aman Gupta 7 months ago

Actually we already have heap.finalnum. Is this the same as zombienum?

#8 Updated by Aman Gupta 7 months ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r43138.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots  [Bug #8983]

Also available in: Atom PDF