Project

General

Profile

Actions

Feature #18885

closed

End of boot advisory API for RubyVM

Added by byroot (Jean Boussier) almost 2 years ago. Updated 9 months ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:109081]

Description

Context

Many optimizations in the Ruby VM rely on lazily computed caches: Strings coderange, constant caches, method caches, etc etc.
As such even without JIT, some operations need a bit of a warm up, and might be flushed if new constants are defined, new code is loaded, or some objects are mutated.

Additionally these lazily computed caches can cause increased memory usage for applications relying on Copy-on-Write memory.
Whenever one of these caches is updated post fork, the entire memory page is invalidated. Precomputing these caches at the end of boot,
even if based on heuristic, could improve Copy-on-Write performance.

The classic example is the objects generation, young objects must be promoted to the old generation before forking, otherwise they'll get invalidated on the next GC run. That's what https://github.com/ko1/nakayoshi_fork addresses.

But there are other sources of CoW invalidation that could be addressed by MRI if it had a clear notification when it needs to be done.

Proposal

If applications had an API to notify the virtual machine that they're done loading code and are about to start processing user input,
it would give the VM a good point in time to perform optimizations on the existing code and objects.

e.g. could be something like RubyVM.prepare, or RubyVM.ready.

It's somewhat similar to Matz's static barrier idea from RubyConf 2020, except that it wouldn't disable any feature.

Potential optimizations

nakayoshi_fork already does the following:

  • Do a major GC run to get rid of as many dangling objects as possible.
  • Promote all surviving objects to the highest generation
  • Compact the heap.

But it would be much simpler to do this from inside the VM rather than do cryptic things such as 4.times { GC.start } from the Ruby side.

It's also not good to do this on every fork, once you fork the first long lived child, you shouldn't run it again. So decorating fork is not a good hook point.

Also after discussing with @jhawthorn (John Hawthorn), @tenderlovemaking (Aaron Patterson) and @alanwu (Alan Wu), we believe this would open the door to several other CoW optimizations:

Precompute inline caches

Even though we don't have hard data to prove it, we are convinced that a big source of CoW invalidation are inline caches. Most ISeq are never invoked during initialization, so child processed are forked with mostly cold caches. As a result the first time a method is executed in the child, many memory pages holding ISeq are invalidated as caches get updated.

We think MRI could try to precompute these caches before forking children. Constant cache particularly should be resolvable statically see https://github.com/ruby/ruby/pull/6187.

Method caches are harder to resolve statically, but we can probably apply some heuristics to at least reduce the cache misses.

Copy on Write aware GC

We could also keep some metadata about which memory pages are shared, or even introduce a "permanent" generation. The Instagram engineering team introduced something like that in Python (ticket, PR).

That makes the GC aware of which objects live on a shared page. With this information the GC can decide to no free dangling objects leaving on these pages, not to compact these pages, etc.

Scan the coderange of all strings

Strings have a lazily computed coderange attribute in their flags. So if a string is allocated at boot, but only used after fork, on first use its coderange will mayneed to be computed and the string mutated.

Using https://github.com/ruby/ruby/pull/6076, I noticed that 58% of the strings retained at the end of the boot sequence had an UNKNOWN coderange.

So eagerly scanning the coderange of all strings could also improve Copy on Write performance.

malloc_trim

This hook will also be a good point to release unused pages to the system with malloc_trim.


Related issues 1 (0 open1 closed)

Related to Ruby master - Feature #11164: Garbage collector in Ruby 2.2 provokes unexpected CoWRejectedauthorNari (Narihiro Nakamura)Actions
Actions

Also available in: Atom PDF

Like2
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like1Like0Like0Like0