Misc #18691
closedAn option to build Ruby with build only flags not propagated to `rbconfig.rb`.
Added by jaruga (Jun Aruga) almost 3 years ago. Updated over 2 years ago.
Description
In a Fedora Ruby RPM packaging, I have a challenge for the current make rbconfig.rb
. Currently when we build Ruby from source to create the Ruby RPM package, we set compiler flag CFLAGS
including --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'
. The file /usr/lib/rpm/redhat/redhat-hardened-cc1
is managed in redhat-rpm-config
RPM package. This is a problem when end users run gem install <a_gem_with_native_extension>
. Because the config.status
created by configure
script and rbconfig.rb
created by make
(make rbconfig
) includes CFLAGS
including --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'
. And the gem install
checks flags in rbconfig.rb
. Then end users need to install the redhat-rpm-config RPM to run the gem install
.
So, we want to create another rbconfig.rb
file with flags not depending on files in the redhat-rpm-config RPM, and ship in the Ruby RPM, so that end users can run gem install
without installing redhat-rpm-config
RPM.
Here are ideal steps to create another rbconfig.rb
when building Ruby to create the Ruby RPM package. I tested it on the current latest master 381475f02e6b44ae729f9403637b30c445b622e5
.
$ autoconf
$ CFLAGS='-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' ./configure
$ make
$ mkdir -p build/rbconfig
$ cd build/rbconfig
$ CFLAGS='-O2' ../../configure
$ make rbconfig.rb
Then we can ship the build/rbconfig/rbconfig.rb
.
But right now it seems that the make rbconfig.rb
triggers the process to build miniruby
, and I want to skip the process.
$ pwd
/home/jaruga/git/ruby/ruby/build/rbconfig
$ ls
config.log config.status* .ext/ GNUmakefile Makefile uncommon.mk
$ make rbconfig.rb
compiling ../../main.c
compiling ../../dmydln.c
...
I executed the following commands, to create miniruby
on the current working directory and not to remake miniruby
(make -o miniruby
) to skip the process to build the miniruby
. But ideally I want to execute only make rbconfig
to create the rbconfig.rb
in build/rbconfig
directory. Do you have any idea about how to improve files such as configure.ac
, common.mk
or tool/mkconfig.rb
to achieve this?
$ cp -p ../../miniruby .
$ make -o miniruby rbconfig.rb
/bin/sh ../../tool/ifchange "--timestamp=.rbconfig.time" rbconfig.rb rbconfig.tmp
rbconfig.rb updated
$ diff -u ../../rbconfig.rb rbconfig.rb
--- ../../rbconfig.rb 2022-04-12 18:03:45.484465916 +0200
+++ rbconfig.rb 2022-04-12 18:13:09.248334704 +0200
@@ -44,7 +44,7 @@
CONFIG["RUBY_SEARCH_PATH"] = ""
CONFIG["UNIVERSAL_INTS"] = ""
CONFIG["UNIVERSAL_ARCHNAMES"] = ""
- CONFIG["configure_args"] = " 'CFLAGS=-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1'"
+ CONFIG["configure_args"] = " 'CFLAGS=-O2'"
CONFIG["CONFIGURE"] = "configure"
CONFIG["vendorarchdir"] = "$(vendorlibdir)/$(sitearch)"
CONFIG["vendorlibdir"] = "$(vendordir)/$(ruby_version)"
@@ -173,7 +173,7 @@
CONFIG["OBJEXT"] = "o"
CONFIG["CPPFLAGS"] = " $(DEFS) $(cppflags)"
CONFIG["LDFLAGS"] = "-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic"
- CONFIG["CFLAGS"] = "-O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1"
+ CONFIG["CFLAGS"] = "-O2"
CONFIG["STRIP"] = "strip -S -x"
CONFIG["RANLIB"] = "gcc-ranlib"
CONFIG["OBJDUMP"] = "objdump"
Updated by nobu (Nobuyoshi Nakada) almost 3 years ago
I think you can pass build-time-only flags to make
:
$ ./autogen.sh
$ CFLAGS='-O2' ./configure
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
Then rbconfig.rb
would not include that flag.
Updated by nobu (Nobuyoshi Nakada) almost 3 years ago
- Status changed from Open to Feedback
Updated by jaruga (Jun Aruga) almost 3 years ago
- Status changed from Feedback to Closed
Thanks for the info. I confirmed that the way you suggested works filling things that I want to do.
$ ./autogen.sh
$ CFLAGS='-O2' ./configure
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 Q= | tee make.log
I confirmed that the C source files are actually compiled with ARCH_FLAG
(--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
)
$ cat main.log
...
compiling ./main.c
gcc -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0 -o main.o -c ./main.c
compiling dmydln.c
gcc -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0 -o dmydln.o -c dmydln.c
...
I confirmed that the rbconfig.rb
doesn't include ARCH_FLAG
(--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
).
$ grep /usr/lib/rpm/redhat rbconfig.rb
$ echo $?
1
Updated by jaruga (Jun Aruga) almost 3 years ago
- Status changed from Closed to Feedback
I want to confirm that stdlibs such as "ruby/openssl" (ext/openssl
) and bundled gems with native extension such as "ruby/debug" defined in gems/bundled_gems
file are compiled with --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
in the process of make
and make install
.
I tried the following commands.
$ ./autogen.sh
$ CFLAGS='-O2' ./configure --prefix=$(pwd)/dest/ruby
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 Q= | tee make.log
$ make install V=1 ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 | tee make_install.log
I can see ruby/openssl is compiled with --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
.
$ cat make.log
...
./miniruby -I./lib -I. -I.ext/common ./ext/extmk.rb --make='make' \
--command-output=ext/openssl/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --extstatic \
-- configure ext/openssl
configuring openssl
...
I can see the ruby/debug is compiled with the --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
.
$ cat make_install.log
...
./miniruby -I./lib -I. -I.ext/common ./ext/extmk.rb --make='make' \
--command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
-- configure .bundle/gems/debug-1.5.0
...
make -C ext/-test-/debug V=1 all
...
make -C .bundle/gems/debug-1.5.0/ext/debug V=1 all
...
But the ruby/openssl source files such as ext/openssl/openssl_missing.c
and ruby/debug's source files such as ext/debug/debug.c are really compiled with the flags? How can I check it, logging those ext/openssl/openssl_missing.c
and ext/debug/debug.c
?
Updated by jaruga (Jun Aruga) almost 3 years ago
How can I check it, logging those ext/openssl/openssl_missing.c and ext/debug/debug.c?
Ah sorry I could find the following log for ext/openssl/openssl_missing.c
. But I couldn't find the log for ext/debug/debug.c
in make_install.log
.
$ cat make.log
...
make -C ext/openssl V=1 all
make[2]: Entering directory '/home/jaruga/var/git/ruby/ruby/ext/openssl'
gcc -I. -I../../.ext/include/x86_64-linux -I../.././include -I../.././ext/openssl -DRUBY_EXTCONF_H=\"extconf.h\" -fPIC -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o openssl_missing.o -c openssl_missing.c
...
Updated by jaruga (Jun Aruga) almost 3 years ago
But I couldn't find the log for ext/debug/debug.c in make_install.log.
After changing like this, I could see the log to compile the ext/debug/debug.c
. Maybe the V=1
in make install V=1
is not propagated to bundle gems Makefile
created by extconf.rb
.
diff --git a/lib/mkmf.rb b/lib/mkmf.rb
index 547a28577e..2ef15bd070 100644
--- a/lib/mkmf.rb
+++ b/lib/mkmf.rb
@@ -1970,7 +1970,7 @@ def configuration(srcdir)
SHELL = /bin/sh
# V=0 quiet, V=1 verbose. other values don't work.
-V = 0
+V = 1
V0 = $(V:0=)
Q1 = $(V:1=)
Q = $(Q1:0=@andirayo
$ make install V=1 ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 | tee make_install.log
$ cat make_install.log
...
gcc -I. -I../../../../../.ext/include/x86_64-linux -I../../../../.././include -I../../../../.././.bundle/gems/debug-1.5.0/ext/debug -DRUBY_EXTCONF_H=\"extconf.h\" -fPIC -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o debug.o -c debug.c
gcc -I. -I../../../../../.ext/include/x86_64-linux -I../../../../.././include -I../../../../.././.bundle/gems/debug-1.5.0/ext/debug -DRUBY_EXTCONF_H=\"extconf.h\" -fPIC -O2 --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o iseq_collector.o -c iseq_collector.c
...
Updated by jaruga (Jun Aruga) almost 3 years ago
A little related to this ticket, I sent PR to set V = 1
in mkmf.rb
by configure --enable-mkmf-verbose
.
https://github.com/ruby/ruby/pull/5879
Updated by jaruga (Jun Aruga) almost 3 years ago
Anyone, could you review this PR? The CI is passed. Thank you.
Updated by jaruga (Jun Aruga) almost 3 years ago
https://github.com/ruby/ruby/pull/5879
Anyone, could you review this PR? The CI is passed. Thank you.
As you know, the PR to add configure --enable-mkmf-verbose
option above was merged.
Updated by jaruga (Jun Aruga) almost 3 years ago
nobu (Nobuyoshi Nakada) wrote in #note-1:
I think you can pass build-time-only flags to
make
:$ ./autogen.sh $ CFLAGS='-O2' ./configure $ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
Then
rbconfig.rb
would not include that flag.
I tested. And I see that the make install
also needs the ARCH_FLAG
to build some native extension gems.
$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
Here is how I tested on the latest master commit 753da6deca34eb7d5d61a26cf66b014ad77ad51d
.
$ ./autogen.sh
$ CFLAGS='-O2' ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose
$ make ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 2>&1 | tee make_arch_flag.log
Then
$ make install 2>&1 | tee make_install.log
$ mv dest dest_make_install
$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 2>&1 | tee make_install_arch_flag.log
$ mv dest dest_make_install_arch_flag
$ diff -r dest_make_install/ dest_make_install_arch_flag/ > diff_make_install.txt
Especially for the ruby debug gem, here is the difference. You see the /usr/lib/rpm/redhat/redhat-hardened-cc1
in the ARCH_FLAG
is used in the gcc command lines.
$ diff --color -r dest_make_install/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out dest_make_install_arch_flag/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out > diff_extentions_debug_gem_make.txt
$ cat diff_extentions_debug_gem_make.txt
2c2
< /home/jaruga/git/ruby/ruby/dest/ruby/bin/ruby -I /home/jaruga/var/git/ruby/ruby/lib -r ./siteconf20220613-361867-pemm3y.rb extconf.rb
---
> /home/jaruga/git/ruby/ruby/dest/ruby/bin/ruby -I /home/jaruga/var/git/ruby/ruby/lib -r ./siteconf20220613-362313-zg4n9u.rb extconf.rb
18,19c18,19
< gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC -o debug.o -c debug.c
< gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC -o iseq_collector.o -c iseq_collector.c
---
> gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o debug.o -c debug.c
> gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -o iseq_collector.o -c iseq_collector.c
21c21
< gcc -shared -o debug.so debug.o iseq_collector.o -L. -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed -Wl,--compress-debug-sections=zlib -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -lruby -lm -lpthread -lc
---
> gcc -shared -o debug.so debug.o iseq_collector.o -L. -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed -Wl,--compress-debug-sections=zlib --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -Wl,-rpath,/home/jaruga/git/ruby/ruby/dest/ruby/lib -L/home/jaruga/git/ruby/ruby/dest/ruby/lib -lruby -lm -lpthread -lc
27c27
< /bin/mkdir -p . ./.gem.20220613-361867-kmazjx/debug
---
> /bin/mkdir -p . ./.gem.20220613-362313-rsh8y1/debug
29c29
< /bin/install -c -m 0755 debug.so ./.gem.20220613-361867-kmazjx/debug
---
> /bin/install -c -m 0755 debug.so ./.gem.20220613-362313-rsh8y1/debug
So, I want to add the ARCH_FLAG
to configure script environment variables for convenience.
$ ./configure --help
...
Some influential environment variables:
cflags additional CFLAGS (ignored when CFLAGS is given)
cppflags additional CPPFLAGS (ignored when CPPFLAGS is given)
cxxflags additional CXXFLAGS (ignored when CXXFLAGS is given)
AR Archiver command
AS Assembler command
CC C compiler command
CXX C++ compiler command
LD Linker command
NM Symbol list command
OBJCOPY Objcopy command
OBJDUMP Objdump command
RANLIB Ranlib command
STRIP Strip command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
LIBS libraries to pass to the linker, e.g. -l<library>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
you have headers in a nonstandard directory <include dir>
CXXFLAGS C++ compiler flags
CPP C preprocessor
...
Updated by jaruga (Jun Aruga) over 2 years ago
So, I want to add the ARCH_FLAG to configure script environment variables for convenience.
Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)
diff --git a/configure.ac b/configure.ac
index aef679296d..5ab8b05453 100644
--- a/configure.ac
+++ b/configure.ac
@@ -49,6 +49,7 @@ m4_include([tool/m4/ruby_werror_flag.m4])dnl
AC_ARG_VAR([cflags], [additional CFLAGS (ignored when CFLAGS is given)])dnl
AC_ARG_VAR([cppflags], [additional CPPFLAGS (ignored when CPPFLAGS is given)])dnl
AC_ARG_VAR([cxxflags], [additional CXXFLAGS (ignored when CXXFLAGS is given)])dnl
+AC_ARG_VAR([build_only_flags], [build only flags, not used in the rbconfig.rb])dnl
: "environment section" && {
HAVE_BASERUBY=yes
@@ -2894,6 +2895,7 @@ AC_ARG_WITH(mjit-tabs,
AC_SUBST(MJIT_TABS)dnl
AC_SUBST(DLDFLAGS)dnl
AC_SUBST(ARCH_FLAG)dnl
+AC_SUBST(build_only_flags)dnl
AC_SUBST(MJIT_HEADER_FLAGS)dnl
AC_SUBST(MJIT_HEADER_INSTALL_DIR)dnl
AC_SUBST(MJIT_CC)dnl
@@ -4411,6 +4413,7 @@ config_summary "DLDFLAGS" "$DLDFLAGS"
config_summary "optflags" "$optflags"
config_summary "debugflags" "$debugflags"
config_summary "warnflags" "$warnflags"
+config_summary "build_only_flags" "$build_only_flags"
config_summary "strip command" "$STRIP"
config_summary "install doc" "$DOCTARGETS"
config_summary "JIT support" "$MJIT_SUPPORT"
diff --git a/template/Makefile.in b/template/Makefile.in
index a8581260b9..9628fdac46 100644
--- a/template/Makefile.in
+++ b/template/Makefile.in
@@ -81,7 +81,7 @@ CC_VERSION = @CC_VERSION@
OUTFLAG = @OUTFLAG@$(empty)
COUTFLAG = @COUTFLAG@$(empty)
CPPOUTFLAG = >
-ARCH_FLAG = @ARCH_FLAG@
+ARCH_FLAG = @ARCH_FLAG@ @build_only_flags@
CFLAGS_NO_ARCH = @CFLAGS@
CFLAGS = $(CFLAGS_NO_ARCH) $(ARCH_FLAG)
cflags = @cflags@
Then I tested like this.
$ ./autogen.sh
$ CFLAGS='-O2' build_only_flags="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose | tee configure.log
...
* build_only_flags: --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
...
$ vi build_only_flags config.status
...
S["build_only_flags"]="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
...
$ grep ^ARCH_FLAG Makefile
ARCH_FLAG = --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
$ make V=1 2>&1 | tee make.log
$ make install 2>&1 | tee make_install.log
The command below works. The command with explicit ARCH_FLAG
propagates the value to the bundled native extension gem's compile (gcc).
$ make install V=1 ARCH_FLAG="--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" 2>&1 | tee make_install.log
$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common ./ext/extmk.rb --make='make' \
--command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1\ -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
-- configure .bundle/gems/debug-1.5.0
...
$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -o debug.o -c debug.c
...
The command below doesn't work. It doesn't work without the make argument ARCH_FLAG
. I expected the line ARCH_FLAG = --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
is used.
$ make install V=1 ECHO=echo Q= 2>&1 | tee make_install.log
$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common ./ext/extmk.rb --make='make' \
--command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ECHO=echo V=1" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
-- configure .bundle/gems/debug-1.5.0
...
$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC -o debug.o -c debug.c
...
I added the ARCH_FLAG
to the SCRIPT_ARGS
used to compile bundled gems. However the command below also doesn't work. The ARCH_FLAG
was not used in the dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
.
uncommon.mk
SCRIPT_ARGS = --dest-dir="$(DESTDIR)" \
--extout="$(EXTOUT)" \
--ext-build-dir="./ext" \
--mflags="$(MFLAGS)" \
--make-flags="$(MAKEFLAGS) ARCH_FLAG=\"$(ARCH_FLAG)\""
$ make install V=1 ECHO=echo Q= 2>&1 | tee make_install.log
$ vi make_install.log
...
./miniruby -I./lib -I. -I.ext/common ./ext/extmk.rb --make='make' \
--command-output=.bundle/gems/debug-1.5.0/exts.mk --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- Q= ECHO=echo V=1 ARCH_FLAG=\"--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1\"" --gnumake=yes --extflags="" --make-flags="MINIRUBY='./miniruby -I./lib -I. -I.ext/common '" --no-extstatic \
-- configure .bundle/gems/debug-1.5.0
...
$ vi dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
...
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC -o debug.o -c debug.c
...
Updated by nobu (Nobuyoshi Nakada) over 2 years ago
jaruga (Jun Aruga) wrote in #note-10:
I tested. And I see that the
make install
also needs theARCH_FLAG
to build some native extension gems.$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
That is because extension libraries in the bundled gems are deferred building until install
right now.
It is a workaround for gem installation complexity and I recognize it as a bug.
jaruga (Jun Aruga) wrote in #note-11:
So, I want to add the ARCH_FLAG to configure script environment variables for convenience.
Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)
Now this flag will appear in rbconfig.rb file, I think.
* build_only_flags: --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \ -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
BTW, why these two flags are need both, and --specs
and -specs
differ?
Updated by jaruga (Jun Aruga) over 2 years ago
nobu (Nobuyoshi Nakada) wrote in #note-12:
jaruga (Jun Aruga) wrote in #note-10:
I tested. And I see that the
make install
also needs theARCH_FLAG
to build some native extension gems.$ make install ARCH_FLAG=--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
That is because extension libraries in the bundled gems are deferred building until
install
right now.
It is a workaround for gem installation complexity and I recognize it as a bug.
I see. The ruby/ruby source 753da6deca34eb7d5d61a26cf66b014ad77ad51d
I tested includes the commit https://github.com/ruby/ruby/commit/5c1b76a3a55afeb07116bbd3492303c6b6cd890d to fix the issue https://bugs.ruby-lang.org/issues/18373. However it seems that the issue is still not fixed. It's great if someone will fix it.
* build_only_flags: --specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \ -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
BTW, why these two flags are need both, and
--specs
and-specs
differ?
Oh I didn't notice that. The gcc manual https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html says the -specs
is correct. However --specs
also works? I am not sure.
I executed the reproducer below with only the -specs
options again on my forked repository's branch: https://github.com/junaruga/ruby/commits/wip/configure-arch-flag. And the -specs
flags are not used in the ruby/debug's build. Here is the log.
$ ./autogen.sh
$ CFLAGS='-O2' build_only_flags="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" ./configure --prefix=$(pwd)/dest/ruby --enable-shared --enable-mkmf-verbose | tee configure.log
...
* build_only_flags: -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 \
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
...
$ grep build_only_flags config.status
ac_cs_config='--prefix=/home/jaruga/git/ruby/ruby/dest/ruby --enable-shared --enable-mkmf-verbose '\''build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1'\'' CFLAGS=-O2'
set X /bin/sh './configure' '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2' $ac_configure_extra_args --no-create --no-recursion
S["configure_args"]=" '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-harde"\
S["build_only_flags"]="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
$ grep ^ARCH_FLAG Makefile
ARCH_FLAG = -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
$ make V=1 2>&1 | tee make.log
$ grep main.c make.log
gcc -O2 -fPIC -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/14.0.0 -o main.o -c ./main.c
91% [863/939] main.c
$ grep redhat rbconfig.rb
CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
$ make install V=1 ECHO=echo 2>&1 | tee make_install.log
$ vi make_install.log
...
echo Extracting bundle gem debug-1.5.0...
Extracting bundle gem debug-1.5.0...
/usr/local/ruby-3.1.1/bin/ruby --disable=gems -C "." \
-Itool -rgem-unpack \
-e 'Gem.unpack("gems/debug-1.5.0.gem", ".bundle/gems", ".bundle/specifications")'
Unpacked gems/debug-1.5.0.gem
rm -fr "./.bundle/gems/debug-1.5.0/".git*
./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems -r./x86_64-linux-fake ./tool/rbinstall.rb --make="make" --dest-dir="" --extout=".ext" --ext-build-dir="./ext" --mflags="" --make-flags=" -- ECHO=echo V=1" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc" --gnumake --install=all --rdoc-output=".ext/rdoc" --html-output=".ext/html"
...
$ grep debug.c dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
echo compiling debug.c
compiling debug.c
gcc -I. -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/x86_64-linux -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1/ruby/backward -I/home/jaruga/git/ruby/ruby/dest/ruby/include/ruby-3.2.0+1 -I. -DHAVE_RB_ISEQ_PARAMETERS -DHAVE_RB_ISEQ_CODE_LOCATION -DHAVE_RB_ISEQ_TYPE -fPIC -O2 -fPIC -o debug.o -c debug.c
$ grep redhat dest/ruby/lib/ruby/gems/3.2.0+1/extensions/x86_64-linux/3.2.0+1/debug-1.5.0/gem_make.out
=> empty
jaruga (Jun Aruga) wrote in #note-11:
So, I want to add the ARCH_FLAG to configure script environment variables for convenience.
Just note. I tried to implement build only flags to inject ARCH_FLAG in Makefile with the patch below (the last commit on https://github.com/junaruga/ruby/commits/wip/configure-arch-flag)
Now this flag will appear in rbconfig.rb file, I think.
Right. The new CONFIG["build_only_flags"]
appears in the rbconfig.rb
. Perhaps it might be better to remove the line in the rbconfig.rb
? I am not sure.
$ grep redhat rbconfig.rb
CONFIG["configure_args"] = " '--prefix=/home/jaruga/git/ruby/ruby/dest/ruby' '--enable-shared' '--enable-mkmf-verbose' 'build_only_flags=-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1' 'CFLAGS=-O2'"
CONFIG["build_only_flags"] = "-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1"
Updated by jaruga (Jun Aruga) over 2 years ago
- Subject changed from An option to run `make rbconfig.rb` in a different directory to An option to build Ruby with build only flags not propagated to `rbconfig.rb`.
I just renamed this ticket's title from "An option to run make rbconfig.rb
in a different directory" to "An option to build Ruby with build only flags not propagated to rbconfig.rb
". It describes this ticket better.