Bug #18073: test/ruby/test_jit.rb: "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: DONTNEED

[jaruga (Jun Aruga)]

I hit the following JIT test failures causing the following error on v2_7_4 tag (Ruby 2.7.4) with gcc 4.8.5 on RHEL 7. The tests work on Ruby 2.7.3 and also works on Ruby 3.0.2.

/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h: In function 'sprintf':
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h:442:3: error: invalid use of '__builtin_va_arg_pack ()'
   return __builtin___sprintf_chk (__s, 2 - 1, 
   ^

$ which gcc
/usr/bin/gcc

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) 
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ git branch
* (detached from v2_7_4)
... 

$ autoconf
$ ./configure --enable-shared 2>&1 | tee configure.log
$ make 2>&1 | tee make.log

Here is one of the failures. I would attach the full long files on this ticket.

$ make test-all TESTS="-v test/ruby/test_jit.rb" 2>&1 | tee make_test-all_test_jit.log
...
 30) Failure:
TestJIT#test_compile_insn_local [/mnt/git/ruby/ruby/test/ruby/test_jit.rb:62]:
Expected 1 times of JIT success, but succeeded 0 times.

script:
"""
p proc {  
  foo = 1
  foo  
}.call
"""


stderr:
"""
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h: In function 'sprintf':
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h:442:3: error: invalid use of '__builtin_va_arg_pack ()'
   return __builtin___sprintf_chk (__s, 2 - 1, 
   ^
compilation terminated due to -Wfatal-errors.
Successful MJIT finish

"""

.
<1> expected but was
<0>.
...
Finished tests in 29.837252s, 3.1504 tests/s, 15.7186 assertions/s.
94 tests, 469 assertions, 87 failures, 0 errors, 4 skips
...

[jaruga (Jun Aruga)]

• File configure.log configure.log added
• File make.log make.log added
• File make_test-all_test_jit.log make_test-all_test_jit.log added

[jaruga (Jun Aruga)]

• Subject changed from test/ruby/test_jit.rb: failures "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5 on RHEL7 to test/ruby/test_jit.rb: "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5

[jaruga (Jun Aruga)]

I confirmed this failures are reproduced at RubyCI - CentOS 7.6(1810) x86_64 - 2.7 case.
http://rubyci.s3.amazonaws.com/centos7/ruby-2.7/log/20210810T041843Z.fail.html.gz

[jaruga (Jun Aruga)]

I executed git bisect between v2_7_3 and v2_7_4. And I found the following commit causes these test failures.

Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5

---

I checked like this by git bisect

$ cat ~/work/test.sh
#!/bin/bash

set -ex

git clean -fdx
autoconf
./configure --enable-shared --prefix $(pwd)/dest
make
make test-all TESTS="-v test/ruby/test_jit.rb -n TestJIT#test_compile_insn_local"

Then run.

$ git bisect start v2_7_4 v2_7_3

$ git bisect run ~/work/test.sh
...
29bbad939939c6dceb804aac667ba372fdee4ef5 is the first bad commit
...
bisect run success

[jaruga (Jun Aruga)]

• Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN to 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: UNKNOWN

[jaruga (Jun Aruga)]

• Backport changed from 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: UNKNOWN to 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: DONTNEED

[nobu (Nobuyoshi Nakada)]

jaruga (Jun Aruga) wrote in #note-4:

I executed git bisect between v2_7_3 and v2_7_4. And I found the following commit causes these test failures.

Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5

This commit is for configure only.
How the results of configure (config.status and config.h files) differ before and after this commit?

[jaruga (Jun Aruga)]

This commit is for configure only.
How the results of configure (config.status and config.h files) differ before and after this commit?

I did ssh login to RubyCI CentOS7 server, and checked the files between commits fd95a18059 (previous commit for 29bbad9399) and 29bbad9399 (first failure),

config.status
./.ext/include/x86_64-linux/ruby/config.h

by running the following commands.

git clean -fdx
autoconf
./configure --enable-shared

For the config.status, here is the result of the difference. There are some differences by not sorted items.

$ \diff -u ok.fd95a18059.config.status err.29bbad9399.config.status
--- ok.fd95a18059.config.status	2021-08-10 10:38:05.000139654 +0000
+++ err.29bbad9399.config.status	2021-08-10 10:25:19.967182125 +0000
@@ -791,18 +791,18 @@
 S["GNU_LD"]="yes"
 S["LD"]="ld"
 S["GCC"]="yes"
-S["EGREP"]="/bin/grep -E"
-S["GREP"]="/bin/grep"
-S["CPP"]="$(CC) -E"
 S["ac_ct_CXX"]=""
 S["CXXFLAGS"]=""
 S["CXX"]="g++"
+S["EGREP"]="/bin/grep -E"
+S["GREP"]="/bin/grep"
+S["CPP"]="gcc -E"
 S["OBJEXT"]="o"
 S["EXEEXT"]=""
+S["ac_ct_CC"]="gcc"
 S["CPPFLAGS"]=" $(DEFS) ${cppflags}"
 S["LDFLAGS"]="-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic"
-S["CFLAGS"]="${cflags}  -fPIC"
-S["ac_ct_CC"]="gcc"
+S["CFLAGS"]="-g -O2 -fPIC"
 S["CC"]="gcc -std=gnu99"
 S["target_os"]="linux"
 S["target_vendor"]="pc"

However the essential difference is the following part.

-S["CFLAGS"]="${cflags}  -fPIC"
+S["CFLAGS"]="-g -O2 -fPIC"

For the config.h, here is the difference.

$ \diff -u ok.fd95a18059.config.h err.29bbad9399.config.h 
--- ok.fd95a18059.config.h	2021-08-10 10:38:04.799141259 +0000
+++ err.29bbad9399.config.h	2021-08-10 10:25:19.775183599 +0000
@@ -169,11 +169,8 @@
 #define PRI_SIZE_PREFIX "z"
 #define PRI_PTRDIFF_PREFIX "t"
 #define HAVE_STRUCT_STAT_ST_BLKSIZE 1
-#define HAVE_ST_BLKSIZE 1
 #define HAVE_STRUCT_STAT_ST_BLOCKS 1
-#define HAVE_ST_BLOCKS 1
 #define HAVE_STRUCT_STAT_ST_RDEV 1
-#define HAVE_ST_RDEV 1
 #define SIZEOF_STRUCT_STAT_ST_SIZE SIZEOF_OFF_T
 #define SIZEOF_STRUCT_STAT_ST_BLOCKS SIZEOF_OFF_T
 #define SIZEOF_STRUCT_STAT_ST_INO SIZEOF_LONG

[nobu (Nobuyoshi Nakada)]

jaruga (Jun Aruga) wrote in #note-8:

However the essential difference is the following part.

-S["CFLAGS"]="${cflags}  -fPIC"
+S["CFLAGS"]="-g -O2 -fPIC"

${cflags} should contain ${optflags} ${debugflags} ${warnflags} by the default.
That means the latter CFLAGS lacks these flags.
Possibly it might affect the precompiled header.

[jaruga (Jun Aruga)]

${cflags} should contain ${optflags} ${debugflags} ${warnflags} by the default.
That means the latter CFLAGS lacks these flags.
Possibly it might affect the precompiled header.

@nobu (Nobuyoshi Nakada) OK. Could you provide a patch to change only from S["CFLAGS"]="-g -O2 -fPIC" to S["CFLAGS"]="${cflags} -fPIC" in config.status? Then I am happy to test the patch or you can test it on the RubyCI CentOS7 server. Thanks.

[nobu (Nobuyoshi Nakada)]

jaruga (Jun Aruga) wrote in #note-10:

@nobu (Nobuyoshi Nakada) OK. Could you provide a patch to change only from S["CFLAGS"]="-g -O2 -fPIC" to S["CFLAGS"]="${cflags} -fPIC" in config.status? Then I am happy to test the patch or you can test it on the RubyCI CentOS7 server. Thanks.

It should be the default, unless you set CFLAGS at the command line or as the environment variable.

[jaruga (Jun Aruga)]

It should be the default, unless you set CFLAGS at the command line or as the environment variable.

As we chat, I confirmed the environment variable CFLAGS is not set before running autoconf, ./configure again. The CFLAGS: ${optflags} ${debugflags} ${warnflags} is printed as an output of the ./configure, but the actual value is S["CFLAGS"]="-g -O2 -fPIC" in config.status.

$ git checkout 29bbad939939c6dceb804aac667ba372fdee4ef5
$ git clean -fdx

$ echo "CFLAGS: $CFLAGS"
CFLAGS: 
$ env | grep CFLAGS
$ echo $?
1
$ unset CFLAGS
$ echo $?
0

$ autoconf
$ ./configure --enable-shared 2>&1 | tee configure.log
...
---
Configuration summary for ruby version 2.7.4
...
   * CFLAGS:              ${optflags} ${debugflags} ${warnflags}
...

$ grep '"CFLAGS"' config.status
S["CFLAGS"]="-g -O2 -fPIC"

I also compared the difference of the outputs of the ./configure --enable-shared, sorting the result by the sort command to only see the essential differences. And here is the result of the difference.

$ \diff -u ok.fd95a18059.configure_sorted.log err.29bbad9399.configure_sorted.log 
--- ok.fd95a18059.configure_sorted.log	2021-08-11 09:10:44.858148283 +0000
+++ err.29bbad9399.configure_sorted.log	2021-08-11 09:11:11.689938763 +0000
@@ -507,7 +507,7 @@
 checking grp.h presence... yes
 checking grp.h usability... yes
 checking host system type... x86_64-pc-linux-gnu
-checking how to run the C preprocessor... gcc -std=gnu99 -E
+checking how to run the C preprocessor... gcc -E
 checking ieeefp.h presence... no
 checking ieeefp.h usability... no
 checking if enum over int is allowed... yes
@@ -543,7 +543,6 @@
 checking pthread.h usability... yes
 checking pwd.h presence... yes
 checking pwd.h usability... yes
-checking return type of signal handlers... void
 checking sanitizer/asan_interface.h presence... no
 checking sanitizer/asan_interface.h usability... no
 checking sanitizer/msan_interface.h presence... no
@@ -694,7 +693,7 @@
 checking whether dtrace USDT is available... no
 checking whether finite is declared... yes
 checking whether g++ accepts -g... no
-checking whether gcc -std=gnu99 -E accepts -o... yes
+checking whether gcc -E accepts -o... yes
 checking whether gcc -std=gnu99 needs -traditional... no
 checking whether gcc accepts -g... yes
 checking whether getenv is declared... yes

[xtkoba (Tee KOBAYASHI)]

It seems that in configure.ac, AC_PROG_CC_C99 must come before AC_PREPROC_IFELSE to set CPP correctly. That is,

AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PROG_CC_C99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
echo "CC:" "$CC"
echo "CPP:" "$CPP"

works fine, while

AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
AC_PROG_CC_C99
echo "CC:" "$CC"
echo "CPP:" "$CPP"

does not.

The relevant part of configure.ac diff is shown below:

--- ruby-2.7.3/configure.ac
+++ ruby-2.7.4/configure.ac

@@ -177,12 +180,14 @@
     # clang version 1.0 (http://llvm.org/svn/llvm-project/cfe/tags/Apple/clang-23 exported)
     # Apple clang version 2.0 (tags/Apple/clang-137) (based on LLVM 2.9svn)
     # Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
-    AS_IF([! $CC -E -xc - <<SRC >/dev/null], [
-	@%:@if defined __APPLE_CC__ && defined __clang_major__ && __clang_major__ < 3
-	@%:@error premature clang
-	@%:@endif
-SRC
-	AC_MSG_ERROR([clang version 3.0 or later is required])
+    AC_PREPROC_IFELSE(
+	[AC_LANG_PROGRAM([
+	    @%:@if defined __APPLE_CC__ && defined __clang_major__ && __clang_major__ < 3
+	    @%:@error premature clang
+	    @%:@endif
+	])],
+	[],
+	[AC_MSG_ERROR([clang version 3.0 or later is required])
     ])],
 [openbsd*:openbsd*], [
     AC_CHECK_TOOLS(CC, [cc])
@@ -191,7 +196,9 @@
   AC_CHECK_TOOL(CC, gcc)
 ])
 
-AC_PROG_CC_C99
+dnl Seems necessarily in order to add -std=gnu99 option for gcc 4.9.
+m4_version_prereq([2.70], [], [AC_PROG_CC_C99])
+
 AS_CASE([$CC],
 [gcc-*], [
     gcc_prefix=gcc- gcc_suffix=`echo "$CC" | sed 's/^gcc//'`

[jaruga (Jun Aruga)]

@xtkoba (Tee KOBAYASHI) thanks for the info. I confirmed the value of CPP on RHEL 7 autoconf version 2.69.

The following configure.ac works fine.

$ cat configure.ac 
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PROG_CC_C99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
echo "CC:" "$CC"
echo "CPP:" "$CPP"

$ autoconf
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... (cached) -std=gnu99
checking how to run the C preprocessor... gcc -std=gnu99 -E
CC: gcc -std=gnu99
CPP: gcc -std=gnu99 -E

The following configure.ac doesn't.

$ cat configure.ac 
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
AC_PROG_CC_C99
echo "CC:" "$CC"
echo "CPP:" "$CPP"

$ autoconf
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes 
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for gcc option to accept ISO C99... (cached) -std=gnu99
CC: gcc -std=gnu99
CPP: gcc -E

[jaruga (Jun Aruga)]

jaruga (Jun Aruga) wrote in #note-4:

I executed git bisect between v2_7_3 and v2_7_4. And I found the following commit causes these test failures.

Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5

As a temporary workaround to build Ruby 2.7.4 on RHEL7, I am applying a patch to revert above commit, and I see the test failures in test/ruby/test_jit.rb are gone.

[ivoanjo (Ivo Anjo)]

I run into a similar issue (in https://github.com/DataDog/dd-trace-rb/issues/1799); in my case, I was running the Semaphore CI Ruby image for 2.7.5 and when trying to run with JIT I get the same failures

semaphore@semaphore-vm:~$ ruby -v
ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]

semaphore@semaphore-vm:~$ gcc -v 2>&1 | grep "gcc version"
gcc version 4.8.5 (Ubuntu 4.8.5-4ubuntu8)

semaphore@semaphore-vm:~$ ruby --jit --jit-warnings --jit-debug --jit-verbose=2 -e 'def foo; end; while true; foo; end'
MJIT: CC defaults to /usr/bin/gcc
MJIT: tmp_dir is /tmp
Creating precompiled header
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -nodefaultlibs -nostdlib -o /tmp/_ruby_mjit_hp4368u0.h.gch /home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h
start compilation: foo@-e:1 -> /tmp/_ruby_mjit_p4368u1.c
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -o /tmp/_ruby_mjit_p4368u1.o /tmp/_ruby_mjit_p4368u1.c -c -nostartfiles -nodefaultlibs -nostdlib
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h: In function 'sprintf':
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h:406:3: error: invalid use of '__builtin_va_arg_pack ()'
   return __builtin___sprintf_chk (__s, 2 - 1,
   ^
compilation terminated due to -Wfatal-errors.
compile_c_to_o: compile error: 1
Failed to generate so: /tmp/_ruby_mjit_p4368u1.so
start compilation: block in already_loaded?@/home/semaphore/.rbenv/versions/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems.rb:1290 -> /tmp/_ruby_mjit_p4368u0.c
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -o /tmp/_ruby_mjit_p4368u0.o /tmp/_ruby_mjit_p4368u0.c -c -nostartfiles -nodefaultlibs -nostdlib
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h: In function 'sprintf':
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h:406:3: error: invalid use of '__builtin_va_arg_pack ()'
   return __builtin___sprintf_chk (__s, 2 - 1,
   ^
compilation terminated due to -Wfatal-errors.
compile_c_to_o: compile error: 1
Failed to generate so: /tmp/_ruby_mjit_p4368u0.so

Would it make sense to perhaps validate that the mjit header is compilable after generating it?

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

These failures look related to MJIT, which has been removed.