Project

General

Profile

Feature #14235

Merge MJIT infrastructure with conservative JIT compiler

Added by k0kubun (Takashi Kokubun) 24 days ago. Updated 19 days ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
[ruby-core:84451]

Description

Background

In Feature#12589, Vladimir Makarov proposed to improve VM performance by replacing VM instructions
to RTL and introduce method JIT compiler based on those instructions.

While his approach for JIT (write C code to local file system, let C compiler executable
to compile it to shared object file and load it dynamically) was great and proven to work,
replacing all VM instructions may change the behavior of all Ruby programs and we can't turn off
such changes once it's released, even if we find a problem. So it's a little risky unlike optional JIT enablement.

Then I developed a JIT compiler called YARV-MJIT, which does not require any VM instruction changes.
After it, I heard Vladimir started to work on another approach to compile from current YARV instructions and
use RTL as IR for JIT compilation, but it's not published yet as far as I know.

Problems

  • We're developing the same JIT infrastructure independently, which can be shared for both implementations
    • it's definitely a waste of time, unlike seeking different optimization approaches
  • If we continue to develop JIT in a big feature branch,
    • affected places will be big too, and thus it'll be a dangerous release
    • all of us will continue to waste our time by day-to-day conflict resolution against trunk
    • many valuable commit logs will be lost when we maintain the branch for rebase or squash commits on merge

Solution

  • Proposal: Merge MJIT infrastructure from Vladimir's patch with a conservative JIT compiler in early 2.6 development.
    • MJIT infrastructure means: JIT worker thread, profiler, gcc/clang compiler support, loading function from shared object file, some hooks to ensure JIT does not cause SEGV, etc...

Patch: https://github.com/ruby/ruby/pull/1782

What's the "conservative JIT compiler"?

  • Based on my YARV-MJIT, but this drops some problematic optimizations and is slower
  • Pass make test, make test-all, make test-spec with and without JIT https://travis-ci.org/ruby/ruby/builds/321589821
  • Unlike MJIT on RTL, we can play optcarrot (not just for benchmark, but on GUI) and run Rails application stably (not tested on production yet though)

Notes

  • As YARV-MJIT implementation improved MJIT infrastructure too, pthread was already ported to Windows native threads and it can be compiled with Visual Studio.
    • That's exactly why we should develop this in cooperation
  • Visual Sudio is not supported as C compiler for JIT compilation. I did some experiments and had some ideas to support cl.exe, but I didn't want to add extra complexity to initial merge.
    • But it's perfectly working on MinGW and this will be available on Windows if a user uses RubyInstaller2.

Optcarrot

Benchmarked with: Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores, gcc 5.4.0

2.0.0 2.5.0 JIT off JIT on
optcarrot fps 35.05 46.75 46.05 63.06
vs 2.0.0 1.00x 1.33x 1.31x 1.80x
  • 2.0.0: Ruby 2.0.0-p0
  • 2.5.0: Ruby 2.5.0 (r61468)
  • JIT off: Patched Ruby (based on r61475), JIT disabled
  • JIT on: Patched Ruby (based on r61475), JIT enabled

Disclaimer: This JIT compiler performs better with gcc compared to clang for now, so it may be slow on macOS (clang).

Micro benchmarks

I used Vladimir's benchmark set which I modified for my convenience https://github.com/benchmark-driver/mjit-benchmarks.

2.0.0-p0 2.5.0 JIT off JIT on
aread 1.00 1.01 0.97 2.33
aref 1.00 0.96 0.96 3.01
aset 1.00 1.39 1.37 3.70
awrite 1.00 1.07 1.03 2.54
call 1.00 1.25 1.22 3.39
const 1.00 0.96 0.96 4.00
const2 1.00 0.96 0.96 3.97
fannk 1.00 0.98 1.02 1.00
fib 1.00 1.16 1.24 3.19
ivread 1.00 0.94 0.93 4.96
ivwrite 1.00 1.09 1.09 3.32
mandelbrot 1.00 0.98 0.98 1.27
meteor 1.00 3.02 2.85 3.16
nbody 1.00 1.02 0.99 1.47
nest-ntimes 1.00 1.05 1.01 1.31
nest-while 1.00 0.96 0.96 1.63
norm 1.00 1.06 1.07 1.26
nsvb 1.00 0.98 0.88 0.88
red-black 1.00 1.03 1.02 1.54
sieve 1.00 1.22 1.22 1.75
trees 1.00 1.07 1.08 1.32
while 1.00 0.96 0.96 5.13

Interface details

If the proposal is accepted, I'm going to add following CLI options:

  -j, --jit       use MJIT with default options
  -j:option, --jit:option
                  use MJIT with an option

MJIT options:
  c, cc           C compiler to generate native code (gcc, clang)
  s, save-temps   Save MJIT temporary files in $TMP or /tmp
  w, warnings     Enable printing MJIT warnings
  d, debug        Enable MJIT debugging (very slow)
  v=num, verbose=num
                  Print MJIT logs of level num or less to stderr
  n=num, num-cache=num
                  Maximum number of JIT codes in a cache

Note that -j:l/--jit:llvm are changed to -j:c/--jit:cc so that we can support cl.exe (Visual Studio) in the future.

Also, for testing, I would like to have following module and method.

MJIT.enabled? #=> true / false

See the commit log for details.


Related issues

Related to Ruby trunk - Feature #12589: VM performance improvement proposalOpen

History

#1 Updated by k0kubun (Takashi Kokubun) 24 days ago

  • Related to Feature #12589: VM performance improvement proposal added

#2 [ruby-core:84549] Updated by Eregon (Benoit Daloze) 21 days ago

Should the performance of MJIT be evaluated in more details before merging?

Merging the MJIT infrastructure means choosing MJIT as the MRI JIT, right?
If so, it would be good to have performance numbers on other larger benchmarks, not just optcarrot.

Many of the MJIT micro-benchmarks are just benchmarks to test optimizations.
IMHO, they should only be considered as showcases of possible speedups,
as they are not representative of real Ruby workloads.

#3 [ruby-core:84550] Updated by Eregon (Benoit Daloze) 21 days ago

Eregon (Benoit Daloze) wrote:

Merging the MJIT infrastructure means choosing MJIT as the MRI JIT, right?

Taking a closer look at the PR, it seems mostly infrastructure for a MJIT-like JIT.

On the other hand, mjit_compile.c is compiling YARV insns to C code, which is YARV-MJIT specific:
https://github.com/k0kubun/ruby/blob/ddb2ff7303982b19adb0a23707722e3ca4ece1d2/mjit_compile.c

So I guess this doesn't fix which variant of MJIT will be chosen,
but it does probably fix that the MRI JIT will be a variant of MJIT.

#4 [ruby-core:84553] Updated by k0kubun (Takashi Kokubun) 21 days ago

Should the performance of MJIT be evaluated in more details before merging?

Definitely true. This ticket was made in hurry for December Ruby Developers Meeting in Japan, so I did only benchmarks which are easy to run at that moment. At least I'm going to run all benchmarks in "/benchmark" directory in Ruby repository.

Merging the MJIT infrastructure means choosing MJIT as the MRI JIT, right?

First of all, I'm regarding "MRI JIT" (and MJIT) as 2 components: JIT infrastructure (Koichi called this as "JIT framework") and JIT compiler. JIT infrastructure includes JIT worker thread, profiler, gcc/clang compiler support, loading function from shared object file, some hooks to ensure JIT does not cause SEGV. And JIT compiler is Vladimir's RTL-based MJIT, insns -> RTL MJIT which is in development, or YARV-MJIT in the patch. Replacing mjit_compile.c allows to replace the component anytime. At least I'm not thinking it means "chosing (YARV-)MJIT as the (final) MRI JIT (compiler)".

In the context, "Merging the MJIT infrastructure" would also mean choosing YARV-MJIT as initial JIT compiler for MRI, but probably all of Ruby developers don't think it's going to be merged as final JIT compiler, as Vladimir is working on another JIT compiler whose first version is much faster than YARV-MJIT.

At the same time, as we can hear in some Ruby conferences, Matz seems to like the MJIT's JIT infrastructure and it might be the solution for a long term. At least LLVM or things that need tricky dependency won't be used. For inlining Ruby core's C code, I believe this part would work well for JIT compilers other than YARV-MJIT approach. As written in this ticket's Problems and Solution, the main goal of this ticket is merging and improving the JIT infrastructure in MJIT.

If so, it would be good to have performance numbers on other larger benchmarks, not just optcarrot.

Many of the MJIT micro-benchmarks are just benchmarks to test optimizations.
IMHO, they should only be considered as showcases of possible speedups,
as they are not representative of real Ruby workloads.

Even while I'm not going to merge it as the final solution, I agree we should have larger benchmarks because nobody will try the MJIT infrastructure if there is no performance gain in its JIT compiler on "real Ruby workloads".

In the initial plan, I was going to use https://github.com/noahgibbs/rails_ruby_bench and still want to use for it. But I get some errors on my environment even with existing Ruby. So it's not just ready. Then I roughly tried to run my simpler Rails application but unfortunately it had no performance gain for now.

I have some optimization ideas that are disabled for now or not implemented yet, and it'll be done in early 2018. If it ends up with no performance gain in "real Ruby workloads", I'll surely revert it. This kind of experiment should be done in early stage of new version, and this timing would be effective for Vladimir's insns -> RTL project (planned to be completed in February) too.

Taking a closer look at the PR, it seems mostly infrastructure for a MJIT-like JIT.

Yes. That's because Matz seems to be already in favor of the direction.

On the other hand, mjit_compile.c is compiling YARV insns to C code, which is YARV-MJIT specific:

Yes, this is intended to be replaced with things like https://github.com/vnmakarov/ruby/blob/21bbbd37b5d9f86910f7679a584bbbfb9dc9c9b1/mjit.c#L1231-L1588

So I guess this doesn't fix which variant of MJIT will be chosen,
but it does probably fix that the MRI JIT will be a variant of MJIT.

I understand "a variant of MJIT" means "method JIT compiler that generates C code from ISeq". If so, probably it's correct but still this is experimental and may be reverted later.

#5 [ruby-core:84556] Updated by Eregon (Benoit Daloze) 20 days ago

k0kubun (Takashi Kokubun) Thank you for the clarification!

I understand "a variant of MJIT" means "method JIT compiler that generates C code from ISeq".

Yes, that's exactly what I meant.

I think running https://github.com/noahgibbs/rails_ruby_bench would be quite interesting.
I think it's worth opening an issue if something doesn't work with Ruby 2.5.
Maybe noahgibbs (Noah Gibbs) can help?

#6 [ruby-core:84564] Updated by Eregon (Benoit Daloze) 19 days ago

I should have watched/read your great "Just-In-Time Compiler for MRI" presentation first:
https://www.youtube.com/watch?v=Ti4a7SXGWig
https://speakerdeck.com/k0kubun/just-in-time-compiler-for-mri

That clearly explains how LLRB, YARV-MJIT and MJIT relate.

Also available in: Atom PDF