Feature #20443: Allow Major GC's to be disabled

• Status: Open
• Assignee: -
• Target version: -

[eightbitraptor (Matthew Valentine-House)]

[Github PR #10598]

Background

Ruby's GC running during Rails requests can have negative impacts on currently
running requests, causing applications to have high tail-latency.

A technique to mitigate this high tail-latency is Out-of-band GC (OOBGC). This
is basically where the application is run with GC disabled, and then GC is
explicitly started after each request, or when no requests are in progress.

This can reduce the tail latency, but also introduces problems of its own. Long
GC pauses after each request reduce throughput. This is more pronounced on
threading servers like Puma because all the threads have to finish processing
user requests and be "paused" before OOBGC can be triggered.

This throughput decrease happens for a couple of reasons:

1. There are few heuristics available for users to determine when GC should run,
   this means that in OOBGC scenarios, it's possible that major GC's are being run
   more than necessary. 2. The lack of any GC during a request means that lots of
   garbage objects have been created and not cleaned up, so the process is using
   more memory than it should - requiring major GC's run as part of OOBGC to do
   more work and therefore take more time.

This ticket attempts to address these issues by:

1. Provide GC.disable_major and its antonym GC.enable_major to disable and
   enable only major GC 2. Provide GC.needs_major? as a basic heuristic allowing
   users to tell when Ruby should run a Major GC.

These ideas were originally proposed by @ko1 (Koichi Sasada) and @byroot (Jean Boussier) in this rails
issue

Disabling GC major's would still allow minor GC's to run during the request,
avoiding the ballooning memory usage caused by not running GC at all, and
reducing the time that a major takes when we do run it, because the nursery
objects have been cleaned up during the request already so there is less work
for a major GC to do.

This can be used in combination with GC.needs_major? to selectively run an
OOBGC only when necessary

Implementation

This PR adds 3 new methods to the GC module

• GC.disable_major This prevents major GC's from running automatically. It
  does not restrict minors. When objspace->rgengc.need_major_gc is set and a
  GC is run, instead of running a major, new heap pages will be allocated and a
  minor run instead. objspace->rgengc.need_major_gc will remain set until a
  major is manually run. If a major is not manually run then the process will
  eventually run out of memory.

  When major GC's are disabled, object promotion is disabled. That is, no
  objects will increment their ages during a minor GC. This is to attempt to
  minimise heap growth during the period between major GC's, by restricting the
  number of old-gen objects that will remain unconsidered by the GC until the
  next major.

  When GC.start is run, then major GC's will be enabled, a GC triggered with
  the options passed to GC.start, and then disable_major will be set to the
  state it was in before GC.start was called.

• GC.enable_major This simply unsets the bit preventing major GC's. This will
  revert the GC to normal generational behaviour. Everything behaves as default
  again.

• GC.needs_major? This exposes the value of objspace->rgengc.need_major_gc
  to the user level API. This is already exposed in
  GC.latest_gc_info[:need_major_by] but I felt that a simpler interface would
  make this easier to use and result in more readable code. eg.

out_of_band do 
  GC.start if GC.needs_major?  
end 

Because object aging is disabled when majors are disabled it is recommended to
use this in conjunction with Process.warmup, which will prepare the heap by
running a major GC, compacting the heap, and promoting every remaining object to
old-gen. This ensures that minor GC's are running over the smallets possible set
of young objects when GC.disable_major is true.

Benchmarks

We ran some tests in production on Shopify's core monolith over a weekend and
found that:

Mean time spent in GC, as well as p99.9 and p99.99 GC times are all
improved.

p99 GC time is slightly higher.

We're running far fewer OOBGC major GC's now that we have GC.needs_major? than
we were before, and we believe that this is contributing to a slightly increased
number of minor GC's. raising the p99 slightly.

App response times are all improved

We see a ~2% reduction in average response times when compared againststandard GC
(~7% p99, ~3% p99.9 and ~4% p99.99).

This drops slightly to an a ~1% reduction in average response times when compared
against our normal OOBGC approach (~6% p99, ~2% p99.9 and ~3% p99.99).

EDIT: to correct a formula error in the original Average charts, numbers updated.

[byroot (Jean Boussier)]

• File Capture d’écran 2024-04-22 à 18.41.52.png Capture d’écran 2024-04-22 à 18.41.52.png added

To add a little bit more context on @eightbitraptor's description. In some graph you see 3 groups:

• oobgc-off: which is workers without any Out of Band GC.
• oobgc-on: which is workers with our previous OOB GC implementation (once every 128 to 512 requests, 20% more every time)
• oobgc-disable-major: which is the new OOB GC implementation that only run GC when GC.need_major? returns true.

The new implementation not only improve latency in most case, it also reduce the capacity impact of having workers running major GC when it wasn't needed.

Here is the graph of GC.stat[:major_gc_count] over these 3 groups, and as you can see oobgc-disable-major runs major GC only about as often as no-OOBGC, whereas the previous implementation is triggering more often than actually needed, wasting server capacity.

[eightbitraptor (Matthew Valentine-House)]

• Description updated (diff)

[eightbitraptor (Matthew Valentine-House)]

• Description updated (diff)

[nateberkopec (Nate Berkopec)]

Regarding the interface:

GC.disable(major: true)
GC.disable(type: :major)

Should we consider these additional keyword arguments rather than adding a new method?

[eightbitraptor (Matthew Valentine-House)]

nateberkopec (Nate Berkopec) wrote in #note-4:

Regarding the interface:

GC.disable(major: true)
GC.disable(type: :major)

Should we consider these additional keyword arguments rather than adding a new method?

I slightly prefer having a new method pair for this, however I don't object to changing it.

I do have a slight concern that GC.disable(major: true) could be read either as disabling major GC's or keeping majors enabled and disabling minors

So if we decide to use the keyword approach I prefer GC.disable(type: major)

[byroot (Jean Boussier)]

I slightly prefer having a new method pair for this

Same. it makes it easy to test for existence with respond_to? and alternative implementations can make them undefined methods like for Process.fork etc.

[shan (Shannon Skipper)]

I wonder if "full_sweep" would be worth considering as an alternative to "major" to align with the existing GC.start(full_sweep: true) keyword argument? Or they could be aliased, but it seems nice to be consistent if I'm understanding it correctly that "major" and "full sweep" have equivalent meaning.

GC.start(full_sweep: true) # existing default
GC.enable_full_sweep
GC.disable_full_sweep

For checking if a full sweep or major is needed, would the addressing the Object convention mean GC.need_full_sweep? singular, Dir.exist? style? I wonder if "need" is the right word?

(edited to remove confusion)

[ko1 (Koichi Sasada)]

Basically I like this idea. Some points.

• should not use "major" as a "major gc", so GC.disable_major should be GC.disable_major_gc and so on.
• I don't have strong opinion about GC.disable(major_gc: true) or GC.disable_major_gc
• "When major GC's are disabled, object promotion is disabled" what happens on oldgen->younggen references? points from the remembers set? I think we can promote this case because it makes minor gc faster (the promoted objects can not be freed until major gc, so the number of living objects is same).

[ko1 (Koichi Sasada)]

• needs_major "s" should not be on method name (like File.exists -> File.exist)
• can you measure the memory consumption? It is a key compared with old OOBGC.

[duerst (Martin Dürst)]

ko1 (Koichi Sasada) wrote in #note-8:

Basically I like this idea. Some points.

• should not use "major" as a "major gc", so GC.disable_major should be GC.disable_major_gc and so on.

Isn't the gc already very obvious from the class GC?

[byroot (Jean Boussier)]

what happens on oldgen->younggen references? points from the remembers set?

Yes.

I think we can promote this case because it makes minor gc faster (the promoted objects can not be freed until major gc, so the number of living objects is same).

I understand your point, but I fear it could be counter-productive. We specifically stopped doing that in [Feature #19678] because there is many patterns in common Ruby code bases that are causing promotion.

I'd rather run minor GC out of band frequently, and major GC out of band very rarely, because the ratio of effectively permanent objects to ephemeral ones tend to be large in long running applications.