Feature #12690: Improve GC with external library that may use large memory - Ruby master - Ruby Issue Tracking System

Actions

Copy link

Feature #12690

closed

Improve GC with external library that may use large memory

Added by kou (Kouhei Sutou) over 8 years ago. Updated over 8 years ago.

Status:

Closed

Assignee:

ko1 (Koichi Sasada)

Target version:

[ruby-core:76982]

Description

GC cares memory allocated by only Ruby. GC doesn't care memory allocated by external library. GC isn't run frequently when external library allocates large memory but Ruby allocates only small memory. In the case, process uses large memory. e.g.: Programs that handle large images may use large memory even when most images are garbage.

How about improving GC to care memory allocated by external library? It means that GC is run when external library allocates large memory.

I created a gem to show the concept: https://github.com/kou/gc-trigger

The gem provides an API that accepts diff of memory usage of external library. Bindings of external library notify diff of memory usage of external library. GC is run when total memory usage of external library overs threshold. Then memory of external library associated with garbage Ruby objects is freed.

I tried the concept with rcairo that is bindings of cairo 2D graphics library.

Yellow line is memory usage without the concept and blue line is memory usage with the concept:

Memory usage is increased without the concept but stable with the concept.

See also: https://github.com/kou/gc-trigger#effect

V8 has an API that is similar to the concept:

https://github.com/v8/v8/blob/5.4.469/include/v8.h#L5937-L5951

 /**
  * Adjusts the amount of registered external memory. Used to give V8 an
  * indication of the amount of externally allocated memory that is kept alive
  * by JavaScript objects. V8 uses this to decide when to perform global
  * garbage collections. Registering externally allocated memory will trigger
  * global garbage collections more often than it would otherwise in an attempt
  * to garbage collect the JavaScript objects that keep the externally
  * allocated memory alive.
  *
  * \param change_in_bytes the change in externally allocated memory that is
  *   kept alive by JavaScript objects.
  * \returns the adjusted value.
  */
 V8_INLINE int64_t
     AdjustAmountOfExternalAllocatedMemory(int64_t change_in_bytes);