Bug #18781: MJIT tests failing with Ubuntu focal with gcc-11 and some flags

• Status: Closed
• Assignee: jaruga (Jun Aruga)
• Target version: -
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

[jaruga (Jun Aruga)]

I find some MJIT tests failing in test/ruby/test_jit.rb and test/ruby/test_rubyvm_jit.rb on gcc 11 on Ubuntu focal on the latest master branch 708e839dee57bc5ef8f5b21fb4fa620e89caeb52 on CI. Here is the CI log on another PR on GitHub Actions - .github/workflows/compilers.yml on Ubuntu focal. The used configure options are ./configure --enable-shared --with-gcc="gcc -O2 -fcf-protection -Wl,-z,now". I was able to reproduce it on my local Fedora 35 too. Here is the steps. For convenience, I would just show one test failure.

$ gcc --version
gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ autoconf
$ ./configure --enable-shared --with-gcc="gcc -O2 -fcf-protection -Wl,-z,now"
$ make

$ make test-all TESTOPTS="-n TestJIT#test_compile_insn_setspecial" TESTS=test/ruby/test_jit.rb
config.status: creating ruby-runner.h
making mjit_build_dir.so
generating x86_64-linux-fake.rb
x86_64-linux-fake.rb updated
Run options: 
  --seed=24654
  "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems"
  --excludes-dir=./test/excludes
  --name=!/memory_leak/
  -n
  TestJIT#test_compile_insn_setspecial

# Running tests:

[1/0] TestJIT#test_compile_insn_setspecial = 0.27 s
  1) Failure:
TestJIT#test_compile_insn_setspecial [/home/jaruga/var/git/ruby/ruby/test/ruby/test_jit.rb:134]:
Expected 1 times of JIT success, but succeeded 0 times.

script:
"""
p proc {  
  true if nil.nil?..nil.nil?
}.call
"""


stderr:
"""
/bin/ld: no input files
collect2: error: ld returned 1 exit status
MJIT warning: Making precompiled header failed on compilation. Stopping MJIT worker...
MJIT warning: timed out to wait for JIT finish

"""

.
<1> expected but was
<0>.

Finished tests in 0.272793s, 3.6658 tests/s, 14.6632 assertions/s.
1 tests, 4 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 3.2.0dev (2022-05-13T22:27:55Z wip/test-annocheck b497531237) [x86_64-linux]
make: *** [uncommon.mk:823: yes-test-all] Error 1

[k0kubun (Takashi Kokubun)]

• Status changed from Open to Assigned
• Assignee set to k0kubun (Takashi Kokubun)

[jaruga (Jun Aruga)]

After fixing this issue, I hope you would delete the following part to test the fix. Thanks.

$ git diff
diff --git a/.github/workflows/compilers.yml b/.github/workflows/compilers.yml
index ddb53cadb8..23c4ce925c 100644
--- a/.github/workflows/compilers.yml
+++ b/.github/workflows/compilers.yml
@@ -248,13 +248,6 @@ jobs:
         if: ${{ matrix.entry.check }}
       - run: make test-tool
         if: ${{ matrix.entry.check }}
-      # FIXME: Skip MJIT tests failing in the annocheck case.
-      # https://bugs.ruby-lang.org/issues/18781
-      - run: |
-          rm test/ruby/test_jit.rb
-          rm test/ruby/test_rubyvm_jit.rb
-        if: ${{ endsWith(matrix.entry.name, 'annocheck') }}
-        working-directory: src
       - run: make test-all TESTS='-- ruby -ext-'
         if: ${{ matrix.entry.check }}
       - run: make test-spec

[k0kubun (Takashi Kokubun)]

• Assignee changed from k0kubun (Takashi Kokubun) to jaruga (Jun Aruga)

@jaruga (Jun Aruga) It seems like the problem is that you're passing a linker flag, -Wl,-z,now, through --with-gcc. To make MJIT's build work, it has to use the same linker flags as ones used for building the interpreter, and it fails that way when they are inconsistent. So I think it's not a problem of MJIT but CI's configuration. For example, if I configure it with --with-gcc="gcc -O2 -fcf-protection" ldflags=-Wl,-z,now, it just works fine. Could you take a look at fixing your configuration of GitHub Actions?

[jaruga (Jun Aruga)]

Could you take a look at fixing your configuration of GitHub Actions?

@k0kubun (Takashi Kokubun) Thanks for the investigation! Sure. I will fix the GitHub Actions configuration file to use your suggested case --with-gcc="gcc -O2 -fcf-protection" ldflags=-Wl,-z,now.

[jaruga (Jun Aruga)]

• File make_ldflags.log make_ldflags.log added

On the latest master 633608ebd4cbdeea562dc4c608495134965a19a9, I am testing.

$ ./configure --enable-shared --enable-mkmf-verbose --with-gcc="gcc -O2 -fcf-protection" ldflags=-Wl,-z,now
$ make V=1 2>&1 | tee make.log

Then -Wl,-z,now is not logged in the make.log to compile the code. For example I cannot find -Wl,-z,now for the gcc command log below. I cannot find -Wl,-z,now in the make.log. Is this intentional? I like a way to log the flag, but I don't know how to do. I would attach the log file as make_ldflags.log.

gcc -O2 -fcf-protection  -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Werror=deprecated-declarations -Werror=div-by-zero -Werror=duplicated-cond -Werror=implicit-function-declaration -Werror=implicit-int -Werror=misleading-indentation -Werror=pointer-arith -Werror=write-strings -Werror=old-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Werror=undef -std=gnu99  -fPIC  -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0    -o main.o -c ./main.c

[jaruga (Jun Aruga)]

I sent the PR, https://github.com/ruby/ruby/pull/5938 .

• Status changed from Assigned to Closed

Applied in changeset git|cfcf33f1270f78c5fb928d7a0927e6be02ec8900.

---

.github/workflows/compilers.yml: annocheck: Fix a linker flag to pass MJIT tests.

Set the linker flag -Wl,-z,now properly.

Co-authored-by: Takashi Kokubun [e-mail]

Fixes [Bug #18781]

[jaruga (Jun Aruga)]

I sent the PR, https://github.com/ruby/ruby/pull/5938 .

The PR was merged.

Then -Wl,-z,now is not logged in the make.log to compile the code. For example I cannot find -Wl,-z,now for the gcc command log below. I cannot find -Wl,-z,now in the make.log. Is this intentional? I like a way to log the flag, but I don't know how to do. I would attach the log file as make_ldflags.log.

The issue above is that the ./configure --enable-shared --enable-mkmf-verbose --with-gcc="gcc -O2 -fcf-protection" ldflags=-Wl,-z,now was wrong. The ./configure --enable-shared --with-gcc="gcc -O2 -fcf-protection" LDFLAGS=-Wl,-z,now is correct. Here is how I tested: https://github.com/ruby/ruby/pull/5938#issuecomment-1147833523 .