Project

General

Profile

Feature #12690

Improve GC with external library that may use large memory

Added by kou (Kouhei Sutou) over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
-
[ruby-core:76982]

Description

GC cares memory allocated by only Ruby. GC doesn't care memory allocated by external library. GC isn't run frequently when external library allocates large memory but Ruby allocates only small memory. In the case, process uses large memory. e.g.: Programs that handle large images may use large memory even when most images are garbage.

How about improving GC to care memory allocated by external library? It means that GC is run when external library allocates large memory.

I created a gem to show the concept: https://github.com/kou/gc-trigger

The gem provides an API that accepts diff of memory usage of external library. Bindings of external library notify diff of memory usage of external library. GC is run when total memory usage of external library overs threshold. Then memory of external library associated with garbage Ruby objects is freed.

I tried the concept with rcairo that is bindings of cairo 2D graphics library.

Yellow line is memory usage without the concept and blue line is memory usage with the concept:

Memory usage with/without the concept

Memory usage is increased without the concept but stable with the concept.

See also: https://github.com/kou/gc-trigger#effect

V8 has an API that is similar to the concept:

https://github.com/v8/v8/blob/5.4.469/include/v8.h#L5937-L5951

 /**
  * Adjusts the amount of registered external memory. Used to give V8 an
  * indication of the amount of externally allocated memory that is kept alive
  * by JavaScript objects. V8 uses this to decide when to perform global
  * garbage collections. Registering externally allocated memory will trigger
  * global garbage collections more often than it would otherwise in an attempt
  * to garbage collect the JavaScript objects that keep the externally
  * allocated memory alive.
  *
  * \param change_in_bytes the change in externally allocated memory that is
  *   kept alive by JavaScript objects.
  * \returns the adjusted value.
  */
 V8_INLINE int64_t
     AdjustAmountOfExternalAllocatedMemory(int64_t change_in_bytes);

Associated revisions

Revision 68aa1d81
Added by nobu (Nobuyoshi Nakada) over 2 years ago

gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56189 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 56189
Added by nobu (Nobuyoshi Nakada) over 2 years ago

gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

Revision 56189
Added by nobu (Nobuyoshi Nakada) over 2 years ago

gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

Revision 56189
Added by nobu (Nobuyoshi Nakada) over 2 years ago

gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

Revision 56189
Added by nobu (Nobuyoshi Nakada) over 2 years ago

gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

History

Updated by matz (Yukihiro Matsumoto) over 2 years ago

  • Assignee set to ko1 (Koichi Sasada)

LGTM. Iff Koichi agrees, it should be merged.

Matz.

Updated by kou (Kouhei Sutou) over 2 years ago

Koichi agreed:

https://twitter.com/_ko1/status/768278836882526208

I don't see this ticket details yet but implementing the concept is OK to me.

(Translated to English from Japanese by me.)

#4

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Status changed from Open to Closed

Applied in changeset r56189.


gc.c: rb_gc_adjust_memory_usage

  • gc.c (rb_gc_adjust_memory_usage): notify memory usage to the GC engine by extension libraries, to trigger GC. [Feature #12690]

Also available in: Atom PDF