https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112023-09-16T15:47:22ZRuby Issue Tracking SystemRuby master - Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baserubyhttps://bugs.ruby-lang.org/issues/19683?journal_id=1046342023-09-16T15:47:22Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>This is still a problem in 3.3.0-preview2. I've been using the following bad hack to work around it:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">Index: common.mk
</span><span class="gd">--- common.mk.orig
</span><span class="gi">+++ common.mk
</span><span class="p">@@ -378,10 +378,10 @@</span> ext/configure-ext.mk: $(PREP) all-incs $(MKFILES) $(RB
configure-ext: $(EXTS_MK)
<span class="err">
</span> build-ext: $(EXTS_MK)
<span class="gd">- $(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
</span><span class="gi">+ -$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
</span> EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
UPDATE_LIBRARIES=no $(EXTSTATIC)
<span class="gd">- $(Q)$(MAKE) $(EXTS_NOTE)
</span><span class="gi">+ -$(Q)$(MAKE) $(EXTS_NOTE)
</span><span class="err">
</span> exts-note: $(EXTS_MK)
$(Q)$(MAKE) $(EXTS_NOTE)
</code></pre> Ruby master - Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baserubyhttps://bugs.ruby-lang.org/issues/19683?journal_id=1056282023-12-11T20:01:45Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul><li><strong>Target version</strong> set to <i>3.3</i></li></ul><p>This was still a problem in 3.3.0-preview3, and is still a problem in 3.3.0-rc1. It should definitely be fixed before the final release of 3.3.0.</p> Ruby master - Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baserubyhttps://bugs.ruby-lang.org/issues/19683?journal_id=1057182023-12-19T02:33:52Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I traced this issue to <a class="changeset" title="Use Lrama LALR parser generator instead of Bison https://bugs.ruby-lang.org/issues/19637 Co-aut..." href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/a1b01e7701f9fc370f8dff777aad6d39a2c5a3e3">a1b01e7701f9fc370f8dff777aad6d39a2c5a3e3</a> (lrama), because that introduced the use of BASERUBY, which is what currently breaks this:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gd">--- a/common.mk
</span><span class="gi">+++ b/common.mk
</span><span class="p">@@ -298,7 +298,8 @@</span> configure-ext: $(EXTS_MK)
build-ext: $(EXTS_MK)
$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
<span class="gd">- EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC)
</span><span class="gi">+ EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
+ UPDATE_LIBRARIES=no $(EXTSTATIC)
</span> $(Q)$(MAKE) $(EXTS_NOTE)
exts-note: $(EXTS_MK)
<span class="p">@@ -1208,8 +1209,8 @@</span> $(srcdir)/ext/ripper/ripper.c: $(srcdir)/ext/ripper/tools/preproc.rb $(srcdir)/p
$(Q) $(CHDIR) $(@D) && \
$(CAT_DEPEND) depend | \
$(exec) $(MAKE) -f - $(mflags) \
<span class="gd">- Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" BISON="$(YACC)" top_srcdir=../.. srcdir=. VPATH=../.. \
- RUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C
</span><span class="gi">+ Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" top_srcdir=../.. srcdir=. VPATH=../.. \
+ RUBY="$(BASERUBY)" BASERUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C
</span><span class="err">
</span></code></pre>
<p>This isn't necessarily specific to BASERUBY, though. The underlying issue is that <code>exts.mk</code> (created from <code>templates/exts.mk.tmpl</code>) uses <code>$(MFLAGS)</code>:</p>
<pre><code>ruby:
$(Q)$(MAKE) $(MFLAGS) $(SUBMAKEOPTS) $@
</code></pre>
<p>This doesn't work as the <code>;</code> inside BASERUBY (defined as <code>echo executable host ruby is required. use --with-baseruby option.; false</code>) is not quoted. I'm not sure how to get it quoted correctly (neither the <code>Q</code> or <code>QL</code> modifiers work), I'm checking with the OpenBSD make maintainer.</p>
<p>One possible way to workaround the issue would be to ship a <code>tool/missing-baseruby</code> file that contained:</p>
<pre><code>#!/bin/sh
echo "executable host ruby is required. use --with-baseruby option."
exit 1
</code></pre>
<p>And use that as <code>BASERUBY</code>.</p>
<p>However, I think the best fix would be to not use <code>BASERUBY</code> here. This is after <code>miniruby</code> and <code>ruby</code> are built, we should be able to use the built ruby instead of <code>BASERUBY</code> when building extensions, unless I'm missing something.</p> Ruby master - Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baserubyhttps://bugs.ruby-lang.org/issues/19683?journal_id=1057422023-12-19T20:02:39Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I tried the following patch and I was able to build master (which requires BASERUBY) and 3.3.0-rc1 (which does not) with it using OpenBSD make:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/common.mk b/common.mk
index 0125645e37..6c637e0936 100644
</span><span class="gd">--- a/common.mk
</span><span class="gi">+++ b/common.mk
</span><span class="p">@@ -395,8 +395,7 @@</span> configure-ext: $(EXTS_MK)
<span class="err">
</span> build-ext: $(EXTS_MK)
$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \
<span class="gd">- EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \
- UPDATE_LIBRARIES=no $(EXTSTATIC)
</span><span class="gi">+ EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC)
</span> $(Q)$(MAKE) $(EXTS_NOTE)
<span class="err">
</span> exts-note: $(EXTS_MK)
</code></pre>
<p>It passed on CI except on Windows when compiling with Visual Studio. It failed on Windows due to <code>ripper</code>. <code>ext/ripper/depend</code> uses <code>BASERUBY</code>:</p>
<pre><code>BISON = $(BASERUBY) $(top_srcdir)/tool/lrama/exe/lrama
</code></pre>
<p>This cannot be easily switched to <code>RUBY</code>, because <code>lrama</code> requires <code>strscan</code>, and <code>strscan</code> is currently not built until after <code>ripper</code>. Even if you work around that to build <code>strscan</code> before <code>ripper</code>, <code>RUBY</code> here does not allow loading the <code>strscan</code> extension library:</p>
<pre><code>../../miniruby -I'../..' -I'../.././lib' -I'../../.ext/x86_64-openbsd7.4' -I'../../.ext/common' ../.././tool/lrama/exe/lrama -t -v -oripper.c - ripper.y < ripper.y
/home/jeremy/tmp/ruby/tool/lrama/lib/lrama/lexer.rb:1:in `require': this executable file can't load extension libraries (LoadError)
</code></pre>
<p>For Ruby 3.3.0, the <code>tool/missing-baseruby</code> approach I mentioned earlier appears to work. I submitted a pull request for that approach: <a href="https://github.com/ruby/ruby/pull/9299" class="external">https://github.com/ruby/ruby/pull/9299</a></p>
<p>Longer term alternative possible solutions to avoid need for <code>BASERUBY</code> in <code>build-ext</code> task for use by <code>ripper</code>:</p>
<ul>
<li>Change <code>lrama</code> so it doesn't need <code>strscan</code> and can use <code>RUBY</code> instead of <code>BASERUBY</code>
</li>
<li>Create <code>ripper.y.c</code> when <code>parse.y.c</code> is created, at top level</li>
<li>Maybe <code>ripper</code> will be removed or replaced by prism and will not need lrama?</li>
</ul> Ruby master - Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baserubyhttps://bugs.ruby-lang.org/issues/19683?journal_id=1057522023-12-20T05:36:42Zjeremyevans (Jeremy Evans)code@jeremyevans.net
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>Applied in changeset <a class="changeset" title="Add tool/missing-baseruby.bat, used when BASERUBY not available Previously, the embedded semicol..." href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/7dca6b53a995a9782026f726c291344b5b8e0ab2">git|7dca6b53a995a9782026f726c291344b5b8e0ab2</a>.</p>
<hr>
<p>Add tool/missing-baseruby.bat, used when BASERUBY not available</p>
<p>Previously, the embedded semicolon in BASERUBY if BASERUBY is<br>
not available breaks tarball builds without BASERUBY when using<br>
OpenBSD make, due to the inability to escape MFLAGS correctly.<br>
This moves the same BASERUBY code into a separate file, avoiding<br>
the MFLAGS quoting issue.</p>
<p>BASERUBY must be passed to build-ext because it is required<br>
by ripper since the introduction of lrama.</p>
<p>Fixes [Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby (Closed)" href="https://bugs.ruby-lang.org/issues/19683">#19683</a>]</p>
<p>Co-authored-by: Nobuyoshi Nakada <a href="mailto:nobu@ruby-lang.org" class="email">nobu@ruby-lang.org</a></p>