Project

General

Profile

Actions

Misc #18691

closed

An option to build Ruby with build only flags not propagated to `rbconfig.rb`.

Added by jaruga (Jun Aruga) almost 2 years ago. Updated over 1 year ago.

Status:
Feedback
Assignee:
-
[ruby-core:108216]

Description

In a Fedora Ruby RPM packaging, I have a challenge for the current make rbconfig.rb. Currently when we build Ruby from source to create the Ruby RPM package, we set compiler flag CFLAGS including --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'. The file /usr/lib/rpm/redhat/redhat-hardened-cc1 is managed in redhat-rpm-config RPM package. This is a problem when end users run gem install <a_gem_with_native_extension>. Because the config.status created by configure script and rbconfig.rb created by make (make rbconfig) includes CFLAGS including --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'. And the gem install checks flags in rbconfig.rb. Then end users need to install the redhat-rpm-config RPM to run the gem install.

So, we want to create another rbconfig.rb file with flags not depending on files in the redhat-rpm-config RPM, and ship in the Ruby RPM, so that end users can run gem install without installing redhat-rpm-config RPM.

Here are ideal steps to create another rbconfig.rb when building Ruby to create the Ruby RPM package. I tested it on the current latest master 381475f02e6b44ae729f9403637b30c445b622e5.

$ autoconf
$ CFLAGS='-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' ./configure
$ make

$ mkdir -p build/rbconfig
$ cd build/rbconfig
$ CFLAGS='-O2' ../../configure
$ make rbconfig.rb

Then we can ship the build/rbconfig/rbconfig.rb.

But right now it seems that the make rbconfig.rb triggers the process to build miniruby, and I want to skip the process.

$ pwd
/home/jaruga/git/ruby/ruby/build/rbconfig

$ ls
config.log  config.status*  .ext/  GNUmakefile  Makefile  uncommon.mk

$ make rbconfig.rb
compiling ../../main.c
compiling ../../dmydln.c
...

I executed the following commands, to create miniruby on the current working directory and not to remake miniruby (make -o miniruby) to skip the process to build the miniruby. But ideally I want to execute only make rbconfig to create the rbconfig.rb in build/rbconfig directory. Do you have any idea about how to improve files such as configure.ac, common.mk or tool/mkconfig.rb to achieve this?

$ cp -p ../../miniruby .

$ make -o miniruby rbconfig.rb
/bin/sh ../../tool/ifchange "--timestamp=.rbconfig.time" rbconfig.rb rbconfig.tmp
rbconfig.rb updated

$ diff -u ../../rbconfig.rb rbconfig.rb
--- ../../rbconfig.rb	2022-04-12 18:03:45.484465916 +0200
+++ rbconfig.rb	2022-04-12 18:13:09.248334704 +0200
@@ -44,7 +44,7 @@
   CONFIG["RUBY_SEARCH_PATH"] = ""
   CONFIG["UNIVERSAL_INTS"] = ""
   CONFIG["UNIVERSAL_ARCHNAMES"] = ""
-  CONFIG["configure_args"] = " 'CFLAGS=-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'"
+  CONFIG["configure_args"] = " 'CFLAGS=-O2'"
   CONFIG["CONFIGURE"] = "configure"
   CONFIG["vendorarchdir"] = "$(vendorlibdir)/$(sitearch)"
   CONFIG["vendorlibdir"] = "$(vendordir)/$(ruby_version)"
@@ -173,7 +173,7 @@
   CONFIG["OBJEXT"] = "o"
   CONFIG["CPPFLAGS"] = " $(DEFS) $(cppflags)"
   CONFIG["LDFLAGS"] = "-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic"
-  CONFIG["CFLAGS"] = "-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1"
+  CONFIG["CFLAGS"] = "-O2"
   CONFIG["STRIP"] = "strip -S -x"
   CONFIG["RANLIB"] = "gcc-ranlib"
   CONFIG["OBJDUMP"] = "objdump"

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

I think you can pass build-time-only flags to make:

$ ./autogen.sh
$ CFLAGS='-O2' ./configure
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1

Then rbconfig.rb would not include that flag.

Actions #2

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

  • Status changed from Open to Feedback

Updated by jaruga (Jun Aruga) almost 2 years ago

  • Status changed from Feedback to Closed

Thanks for the info. I confirmed that the way you suggested works filling things that I want to do.

$ ./autogen.sh
$ CFLAGS='-O2' ./configure
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 Q= | tee make.log

I confirmed that the C source files are actually compiled with ARCH_FLAG (--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1)

$ cat main.log
...
compiling ./main.c
gcc  -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0    -o main.o -c ./main.c
compiling dmydln.c
gcc -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0    -o dmydln.o -c dmydln.c
...

I confirmed that the rbconfig.rb doesn't include ARCH_FLAG (--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1).

$ grep /usr/lib/rpm/redhat rbconfig.rb 
$ echo $?
1

Updated by jaruga (Jun Aruga) almost 2 years ago

  • Status changed from Closed to Feedback

I want to confirm that stdlibs such as "ruby/openssl" (ext/openssl) and bundled gems with native extension such as "ruby/debug" defined in gems/bundled_gems file are compiled with --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 in the process of make and make install.

I tried the following commands.

$ ./autogen.sh
$ CFLAGS='-O2' ./configure --prefix=$(pwd)/dest/ruby
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 Q= | tee make.log
$ make install V=1 ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 | tee make_install.log

I can see ruby/openssl is compiled with --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1.

$ cat make.log
...
./miniruby -I./lib -I. -I.ext/common  ./ext/extmk.rb --make='make' \
    --command-output=ext/openssl/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --extstatic  \
    -- configure ext/openssl
configuring openssl
...

I can see the ruby/debug is compiled with the --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1.

$ cat make_install.log
...
./miniruby -I./lib -I. -I.ext/common  ./ext/extmk.rb --make='make' \
    --command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
    -- configure .bundle/gems/debug-1.5.0
...
make -C ext/-test-/debug V=1 all
...
make -C .bundle/gems/debug-1.5.0/ext/debug V=1 all
...

But the ruby/openssl source files such as ext/openssl/openssl_missing.c and ruby/debug's source files such as ext/debug/debug.c are really compiled with the flags? How can I check it, logging those ext/openssl/openssl_missing.c and ext/debug/debug.c?

Updated by jaruga (Jun Aruga) almost 2 years ago

How can I check it, logging those ext/openssl/openssl_missing.c and ext/debug/debug.c?

Ah sorry I could find the following log for ext/openssl/openssl_missing.c. But I couldn't find the log for ext/debug/debug.c in make_install.log.

$ cat make.log
...
make -C ext/openssl V=1 all
make[2]: Entering directory '/home/jaruga/var/git/ruby/ruby/ext/openssl'
gcc -I. -I../../.ext/include/x86_64-linux -I../.././include -I../.././ext/openssl  -DRUBY_EXTCONF_H=\"extconf.h\"    -fPIC -O2  --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o openssl_missing.o -c openssl_missing.c
...

Updated by jaruga (Jun Aruga) almost 2 years ago

But I couldn't find the log for ext/debug/debug.c in make_install.log.

After changing like this, I could see the log to compile the ext/debug/debug.c. Maybe the V=1 in make install V=1 is not propagated to bundle gems Makefile created by extconf.rb.

diff --git a/lib/mkmf.rb b/lib/mkmf.rb
index 547a28577e..2ef15bd070 100644
--- a/lib/mkmf.rb
+++ b/lib/mkmf.rb
@@ -1970,7 +1970,7 @@ def configuration(srcdir)
 SHELL = /bin/sh

 # V=0 quiet, V=1 verbose.  other values don't work.
-V = 0
+V = 1
 V0 = $(V:0=)
 Q1 = $(V:1=)
 Q = $(Q1:0=@andirayo
$ make install V=1 ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 | tee make_install.log
$ cat make_install.log
...
gcc -I. -I../../../../../.ext/include/x86_64-linux -I../../../../.././include -I../../../../.././.bundle/gems/debug-1.5.0/ext/debug -DRUBY_EXTCONF_H=\"extconf.h\"    -fPIC -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o debug.o -c debug.c
gcc -I. -I../../../../../.ext/include/x86_64-linux -I../../../../.././include -I../../../../.././.bundle/gems/debug-1.5.0/ext/debug -DRUBY_EXTCONF_H=\"extconf.h\"    -fPIC -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o iseq_collector.o -c iseq_collector.c
...

Updated by jaruga (Jun Aruga) almost 2 years ago

A little related to this ticket, I sent PR to set V = 1 in mkmf.rb by configure --enable-mkmf-verbose.
https://github.com/ruby/ruby/pull/5879

Updated by jaruga (Jun Aruga) almost 2 years ago

https://github.com/ruby/ruby/pull/5879

Anyone, could you review this PR? The CI is passed. Thank you.

Updated by jaruga (Jun Aruga) almost 2 years ago

https://github.com/ruby/ruby/pull/5879
Anyone, could you review this PR? The CI is passed. Thank you.

As you know, the PR to add configure --enable-mkmf-verbose option above was merged.

Updated by jaruga (Jun Aruga) almost 2 years ago

nobu (Nobuyoshi Nakada) wrote in #note-1:

I think you can pass build-time-only flags to make:

$ ./autogen.sh
$ CFLAGS='-O2' ./configure
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1

Then rbconfig.rb would not include that flag.

I tested. And I see that the make install also needs the ARCH_FLAG to build some native extension gems.

$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1

Here is how I tested on the latest master commit 753da6deca34eb7d5d61a26cf66b014ad77ad51d.

$ ./autogen.sh
$ CFLAGS='-O2' ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 2>&1 | tee make_arch_flag.log

Then

$ make install 2>&1 | tee make_install.log
$ mv dest dest_make_install

$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 2>&1 | tee make_install_arch_flag.log
$ mv dest dest_make_install_arch_flag
$ diff -r dest_make_install/ dest_make_install_arch_flag/ > diff_make_install.txt

Especially for the ruby debug gem, here is the difference. You see the /usr/lib/rpm/redhat/redhat-hardened-cc1 in the ARCH_FLAG is used in the gcc command lines.

$ diff --color -r dest_make_install/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out dest_make_install_arch_flag/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out > diff_extentions_debug_gem_make.txt

$ cat diff_extentions_debug_gem_make.txt 
2c2
< /home/jaruga/git/ruby/ruby/dest/ruby/bin/ruby -I /home/jaruga/var/git/ruby/ruby/lib -r ./siteconf20220613-361867-pemm3y.rb extconf.rb
---
> /home/jaruga/git/ruby/ruby/dest/ruby/bin/ruby -I /home/jaruga/var/git/ruby/ruby/lib -r ./siteconf20220613-362313-zg4n9u.rb extconf.rb
18,19c18,19
< gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o debug.o -c debug.c
< gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o iseq_collector.o -c iseq_collector.c
---
> gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o debug.o -c debug.c
> gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o iseq_collector.o -c iseq_collector.c
21c21
< gcc -shared -o debug.so debug.o iseq_collector.o -L. -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed -Wl,--compress-debug-sections=zlib    -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -lruby  -lm -lpthread  -lc
---
> gcc -shared -o debug.so debug.o iseq_collector.o -L. -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed -Wl,--compress-debug-sections=zlib  --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -lruby  -lm -lpthread  -lc
27c27
< /bin/mkdir -p . ./.gem.20220613-361867-kmazjx/debug
---
> /bin/mkdir -p . ./.gem.20220613-362313-rsh8y1/debug
29c29
< /bin/install -c -m 0755 debug.so ./.gem.20220613-361867-kmazjx/debug
---
> /bin/install -c -m 0755 debug.so ./.gem.20220613-362313-rsh8y1/debug

So, I want to add the ARCH_FLAG to configure script environment variables for convenience.

$ ./configure --help
...
Some influential environment variables:
  cflags      additional CFLAGS (ignored when CFLAGS is given)
  cppflags    additional CPPFLAGS (ignored when CPPFLAGS is given)
  cxxflags    additional CXXFLAGS (ignored when CXXFLAGS is given)
  AR          Archiver command
  AS          Assembler command
  CC          C compiler command
  CXX         C++ compiler command
  LD          Linker command
  NM          Symbol list command
  OBJCOPY     Objcopy command
  OBJDUMP     Objdump command
  RANLIB      Ranlib command
  STRIP       Strip command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  LIBS        libraries to pass to the linker, e.g. -l<library>
  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
              you have headers in a nonstandard directory <include dir>
  CXXFLAGS    C++ compiler flags
  CPP         C preprocessor
...

Updated by jaruga (Jun Aruga) almost 2 years ago

So, I want to add the ARCH_FLAG to configure script environment variables for convenience.

Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)

diff --git a/configure.ac b/configure.ac
index aef679296d..5ab8b05453 100644
--- a/configure.ac
+++ b/configure.ac
@@ -49,6 +49,7 @@ m4_include([tool/m4/ruby_werror_flag.m4])dnl
 AC_ARG_VAR([cflags], [additional CFLAGS (ignored when CFLAGS is given)])dnl
 AC_ARG_VAR([cppflags], [additional CPPFLAGS (ignored when CPPFLAGS is given)])dnl
 AC_ARG_VAR([cxxflags], [additional CXXFLAGS (ignored when CXXFLAGS is given)])dnl
+AC_ARG_VAR([build_only_flags], [build only flags, not used in the rbconfig.rb])dnl
 
 : "environment section" && {
 HAVE_BASERUBY=yes
@@ -2894,6 +2895,7 @@ AC_ARG_WITH(mjit-tabs,
 AC_SUBST(MJIT_TABS)dnl
 AC_SUBST(DLDFLAGS)dnl
 AC_SUBST(ARCH_FLAG)dnl
+AC_SUBST(build_only_flags)dnl
 AC_SUBST(MJIT_HEADER_FLAGS)dnl
 AC_SUBST(MJIT_HEADER_INSTALL_DIR)dnl
 AC_SUBST(MJIT_CC)dnl
@@ -4411,6 +4413,7 @@ config_summary "DLDFLAGS"            "$DLDFLAGS"
 config_summary "optflags"            "$optflags"
 config_summary "debugflags"          "$debugflags"
 config_summary "warnflags"           "$warnflags"
+config_summary "build_only_flags"    "$build_only_flags"
 config_summary "strip command"       "$STRIP"
 config_summary "install doc"         "$DOCTARGETS"
 config_summary "JIT support"         "$MJIT_SUPPORT"
diff --git a/template/Makefile.in b/template/Makefile.in
index a8581260b9..9628fdac46 100644
--- a/template/Makefile.in
+++ b/template/Makefile.in
@@ -81,7 +81,7 @@ CC_VERSION = @CC_VERSION@
 OUTFLAG = @OUTFLAG@$(empty)
 COUTFLAG = @COUTFLAG@$(empty)
 CPPOUTFLAG = >
-ARCH_FLAG = @ARCH_FLAG@
+ARCH_FLAG = @ARCH_FLAG@ @build_only_flags@
 CFLAGS_NO_ARCH = @CFLAGS@
 CFLAGS = $(CFLAGS_NO_ARCH) $(ARCH_FLAG)
 cflags = @cflags@

Then I tested like this.

$ ./autogen.sh

$ CFLAGS='-O2' build_only_flags="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose | tee configure.log
...
   * build_only_flags:    --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
                          -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
...

$ vi build_only_flags config.status
...
S["build_only_flags"]="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
...

$ grep ^ARCH_FLAG Makefile
ARCH_FLAG =  --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

$ make V=1 2>&1 | tee make.log
$ make install 2>&1 | tee make_install.log

The command below works. The command with explicit ARCH_FLAG propagates the value to the bundled native extension gem's compile (gcc).

$ make install V=1 ARCH_FLAG="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" 2>&1 | tee make_install.log

$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common  ./ext/extmk.rb --make='make' \
    --command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1\ -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
    -- configure .bundle/gems/debug-1.5.0
...

$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -o debug.o -c debug.c
...

The command below doesn't work. It doesn't work without the make argument ARCH_FLAG. I expected the line ARCH_FLAG = --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 is used.

$ make install V=1 ECHO=echo Q= 2>&1 | tee make_install.log

$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common  ./ext/extmk.rb --make='make' \
    --command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ECHO=echo V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
    -- configure .bundle/gems/debug-1.5.0
...

$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o debug.o -c debug.c 
...

I added the ARCH_FLAG to the SCRIPT_ARGS used to compile bundled gems. However the command below also doesn't work. The ARCH_FLAG was not used in the dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out.

uncommon.mk

SCRIPT_ARGS   = --dest-dir="$(DESTDIR)" \
        --extout="$(EXTOUT)" \
        --ext-build-dir="./ext" \
        --mflags="$(MFLAGS)" \
        --make-flags="$(MAKEFLAGS) ARCH_FLAG=\"$(ARCH_FLAG)\""
$ make install V=1 ECHO=echo Q= 2>&1 | tee make_install.log

$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common  ./ext/extmk.rb --make='make' \
    --command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ECHO=echo V=1 ARCH_FLAG=\"--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1\"" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
    -- configure .bundle/gems/debug-1.5.0
...

$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o debug.o -c debug.c
...

Updated by nobu (Nobuyoshi Nakada) over 1 year ago

jaruga (Jun Aruga) wrote in #note-10:

I tested. And I see that the make install also needs the ARCH_FLAG to build some native extension gems.

$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1

That is because extension libraries in the bundled gems are deferred building until install right now.
It is a workaround for gem installation complexity and I recognize it as a bug.

jaruga (Jun Aruga) wrote in #note-11:

So, I want to add the ARCH_FLAG to configure script environment variables for convenience.

Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)

Now this flag will appear in rbconfig.rb file, I think.

   * build_only_flags:    --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
                          -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

BTW, why these two flags are need both, and --specs and -specs differ?

Updated by jaruga (Jun Aruga) over 1 year ago

nobu (Nobuyoshi Nakada) wrote in #note-12:

jaruga (Jun Aruga) wrote in #note-10:

I tested. And I see that the make install also needs the ARCH_FLAG to build some native extension gems.

$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1

That is because extension libraries in the bundled gems are deferred building until install right now.
It is a workaround for gem installation complexity and I recognize it as a bug.

I see. The ruby/ruby source 753da6deca34eb7d5d61a26cf66b014ad77ad51d I tested includes the commit https://github.com/ruby/ruby/commit/5c1b76a3a55afeb07116bbd3492303c6b6cd890d to fix the issue https://bugs.ruby-lang.org/issues/18373. However it seems that the issue is still not fixed. It's great if someone will fix it.

   * build_only_flags:    --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
                          -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

BTW, why these two flags are need both, and --specs and -specs differ?

Oh I didn't notice that. The gcc manual https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html says the -specs is correct. However --specs also works? I am not sure.

I executed the reproducer below with only the -specs options again on my forked repository's branch: https://github.com/junaruga/ruby/commits/wip/configure-arch-flag. And the -specs flags are not used in the ruby/debug's build. Here is the log.

$ ./autogen.sh

$ CFLAGS='-O2' build_only_flags="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose | tee configure.log
...
   * build_only_flags:    -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
                          -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
...
$ grep build_only_flags config.status
ac_cs_config='--prefix=/home/jaruga/git/ruby/ruby/dest/ruby --enable-shared --enable-mkmf-verbose '\''build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1'\'' CFLAGS=-O2'
  set X /bin/sh './configure'  '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2' $ac_configure_extra_args --no-create --no-recursion
S["configure_args"]=" '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-harde"\
S["build_only_flags"]="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

$ grep ^ARCH_FLAG Makefile
ARCH_FLAG =  -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

$ make V=1 2>&1 | tee make.log

$ grep main.c make.log
gcc  -O2 -fPIC -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0    -o main.o -c ./main.c
 91% [863/939]  main.c

$ grep redhat rbconfig.rb
  CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
  CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

$ make install V=1 ECHO=echo 2>&1 | tee make_install.log

$ vi make_install.log
...
echo Extracting bundle gem debug-1.5.0...
Extracting bundle gem debug-1.5.0...
/usr/local/ruby-3.1.1/bin/ruby --disable=gems -C "." \
    -Itool -rgem-unpack \
    -e 'Gem.unpack("gems/debug-1.5.0.gem", ".bundle/gems", ".bundle/specifications")'
Unpacked gems/debug-1.5.0.gem
rm -fr "./.bundle/gems/debug-1.5.0/".git*
./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems -r./x86_64-linux-fake ./tool/rbinstall.rb --make="make" --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ECHO=echo V=1" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc"  --gnumake --install=all --rdoc-output=".ext/rdoc" --html-output=".ext/html"
...

$ grep debug.c dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
echo compiling debug.c
compiling debug.c
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE    -fPIC -O2 -fPIC  -o debug.o -c debug.c

$ grep redhat dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
  => empty

jaruga (Jun Aruga) wrote in #note-11:

So, I want to add the ARCH_FLAG to configure script environment variables for convenience.

Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)

Now this flag will appear in rbconfig.rb file, I think.

Right. The new CONFIG["build_only_flags"] appears in the rbconfig.rb. Perhaps it might be better to remove the line in the rbconfig.rb? I am not sure.

$ grep redhat rbconfig.rb
  CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
  CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"

Updated by jaruga (Jun Aruga) over 1 year ago

  • Subject changed from An option to run `make rbconfig.rb` in a different directory to An option to build Ruby with build only flags not propagated to `rbconfig.rb`.

I just renamed this ticket's title from "An option to run make rbconfig.rb in a different directory" to "An option to build Ruby with build only flags not propagated to rbconfig.rb". It describes this ticket better.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0