Bug #19683
closedruby-3.3.0-preview1 does not build with BSD make without --with-baseruby
Description
ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby
. However, it builds fine with GNU make and when using BSD make with --with-baseruby
. Previous Ruby tarball releases have built fine with BSD make without --with-baseruby
, so this issue has been introduced recently.
When building with BSD make without --with-baseruby
, building fails after generating RI format with:
/bin/sh: false: not found
*** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...)
*** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...)
Updated by jeremyevans0 (Jeremy Evans) about 1 year ago
This is still a problem in 3.3.0-preview2. I've been using the following bad hack to work around it:
Index: common.mk
--- common.mk.orig
+++ common.mk
@@ -378,10 +378,10 @@ ext/configure-ext.mk: $(PREP) all-incs $(MKFILES) $(RB
configure-ext: $(EXTS_MK)
build-ext: $(EXTS_MK)
- $(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
+ -$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
UPDATE_LIBRARIES=no $(EXTSTATIC)
- $(Q)$(MAKE) $(EXTS_NOTE)
+ -$(Q)$(MAKE) $(EXTS_NOTE)
exts-note: $(EXTS_MK)
$(Q)$(MAKE) $(EXTS_NOTE)
Updated by jeremyevans0 (Jeremy Evans) 11 months ago
- Target version set to 3.3
This was still a problem in 3.3.0-preview3, and is still a problem in 3.3.0-rc1. It should definitely be fixed before the final release of 3.3.0.
Updated by jeremyevans0 (Jeremy Evans) 11 months ago
I traced this issue to a1b01e7701f9fc370f8dff777aad6d39a2c5a3e3 (lrama), because that introduced the use of BASERUBY, which is what currently breaks this:
--- a/common.mk
+++ b/common.mk
@@ -298,7 +298,8 @@ configure-ext: $(EXTS_MK)
build-ext: $(EXTS_MK)
$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
- EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC)
+ EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
+ UPDATE_LIBRARIES=no $(EXTSTATIC)
$(Q)$(MAKE) $(EXTS_NOTE)
exts-note: $(EXTS_MK)
@@ -1208,8 +1209,8 @@ $(srcdir)/ext/ripper/ripper.c: $(srcdir)/ext/ripper/tools/preproc.rb $(srcdir)/p
$(Q) $(CHDIR) $(@D) && \
$(CAT_DEPEND) depend | \
$(exec) $(MAKE) -f - $(mflags) \
- Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" BISON="$(YACC)" top_srcdir=../.. srcdir=. VPATH=../.. \
- RUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C
+ Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" top_srcdir=../.. srcdir=. VPATH=../.. \
+ RUBY="$(BASERUBY)" BASERUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C
This isn't necessarily specific to BASERUBY, though. The underlying issue is that exts.mk
(created from templates/exts.mk.tmpl
) uses $(MFLAGS)
:
ruby:
$(Q)$(MAKE) $(MFLAGS) $(SUBMAKEOPTS) $@
This doesn't work as the ;
inside BASERUBY (defined as echo executable host ruby is required. use --with-baseruby option.; false
) is not quoted. I'm not sure how to get it quoted correctly (neither the Q
or QL
modifiers work), I'm checking with the OpenBSD make maintainer.
One possible way to workaround the issue would be to ship a tool/missing-baseruby
file that contained:
#!/bin/sh
echo "executable host ruby is required. use --with-baseruby option."
exit 1
And use that as BASERUBY
.
However, I think the best fix would be to not use BASERUBY
here. This is after miniruby
and ruby
are built, we should be able to use the built ruby instead of BASERUBY
when building extensions, unless I'm missing something.
Updated by jeremyevans0 (Jeremy Evans) 11 months ago
I tried the following patch and I was able to build master (which requires BASERUBY) and 3.3.0-rc1 (which does not) with it using OpenBSD make:
diff --git a/common.mk b/common.mk
index 0125645e37..6c637e0936 100644
--- a/common.mk
+++ b/common.mk
@@ -395,8 +395,7 @@ configure-ext: $(EXTS_MK)
build-ext: $(EXTS_MK)
$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
- EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
- UPDATE_LIBRARIES=no $(EXTSTATIC)
+ EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC)
$(Q)$(MAKE) $(EXTS_NOTE)
exts-note: $(EXTS_MK)
It passed on CI except on Windows when compiling with Visual Studio. It failed on Windows due to ripper
. ext/ripper/depend
uses BASERUBY
:
BISON = $(BASERUBY) $(top_srcdir)/tool/lrama/exe/lrama
This cannot be easily switched to RUBY
, because lrama
requires strscan
, and strscan
is currently not built until after ripper
. Even if you work around that to build strscan
before ripper
, RUBY
here does not allow loading the strscan
extension library:
../../miniruby -I'../..' -I'../.././lib' -I'../../.ext/x86_64-openbsd7.4' -I'../../.ext/common' ../.././tool/lrama/exe/lrama -t -v -oripper.c - ripper.y < ripper.y
/home/jeremy/tmp/ruby/tool/lrama/lib/lrama/lexer.rb:1:in `require': this executable file can't load extension libraries (LoadError)
For Ruby 3.3.0, the tool/missing-baseruby
approach I mentioned earlier appears to work. I submitted a pull request for that approach: https://github.com/ruby/ruby/pull/9299
Longer term alternative possible solutions to avoid need for BASERUBY
in build-ext
task for use by ripper
:
- Change
lrama
so it doesn't needstrscan
and can useRUBY
instead ofBASERUBY
- Create
ripper.y.c
whenparse.y.c
is created, at top level - Maybe
ripper
will be removed or replaced by prism and will not need lrama?
Updated by jeremyevans (Jeremy Evans) 11 months ago
- Status changed from Open to Closed
Applied in changeset git|7dca6b53a995a9782026f726c291344b5b8e0ab2.
Add tool/missing-baseruby.bat, used when BASERUBY not available
Previously, the embedded semicolon in BASERUBY if BASERUBY is
not available breaks tarball builds without BASERUBY when using
OpenBSD make, due to the inability to escape MFLAGS correctly.
This moves the same BASERUBY code into a separate file, avoiding
the MFLAGS quoting issue.
BASERUBY must be passed to build-ext because it is required
by ripper since the introduction of lrama.
Fixes [Bug #19683]
Co-authored-by: Nobuyoshi Nakada nobu@ruby-lang.org