Bug #19462: MJIT not enabled with universal macOS x86_64 + arm64 build

• Status: Closed
• Assignee: k0kubun (Takashi Kokubun)
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-02-21T21:59:54Z master ae9e1aee59) [universal.x86_64-darwin22]
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[benhamilton (Ben Hamilton)]

I'm trying to build a universal macOS x86_64 + arm64 ruby from upstream @ ae9e1aee59b0db1e61aa0473556165f9fd719cde on x86_64 macOS Ventura 13.2, Xcode 14.2, Apple clang version 14.0.0 (clang-1400.0.29.202).

When I run autoconf && ./configure --with-arch=x86_64,arm64, MJIT support is yes and YJIT support is no (I assume YJIT is disabled because I'm building for arm64):

Configuration summary for ruby version 3.3.0

   * Installation prefix: /usr/local
   * exec prefix:         ${prefix}
   * arch:                universal-darwin22
   * site arch:           ${arch}
   * RUBY_BASE_NAME:      ruby
   * ruby lib prefix:     ${libdir}/${RUBY_BASE_NAME}
   * site libraries path: ${rubylibprefix}/${sitearch}
   * vendor path:         ${rubylibprefix}/vendor_ruby
   * target OS:           darwin22
   * compiler:            clang
   * with thread:         pthread
   * with coroutine:      universal
   * enable shared libs:  no
   * dynamic library ext: bundle
   * CFLAGS:              -fdeclspec ${optflags} ${debugflags} ${warnflags}
   * LDFLAGS:             -L. -fstack-protector-strong
   * DLDFLAGS:            -Wl,-multiply_defined,suppress
   * optflags:            -O3 -fno-fast-math
   * debugflags:          -ggdb3
   * warnflags:           -Wall -Wextra -Wextra-tokens \
                          -Wdeprecated-declarations -Wdivision-by-zero \
                          -Wdiv-by-zero -Wimplicit-function-declaration \
                          -Wimplicit-int -Wmisleading-indentation \
                          -Wpointer-arith -Wshorten-64-to-32 \
                          -Wwrite-strings -Wold-style-definition \
                          -Wmissing-noreturn -Wno-cast-function-type \
                          -Wno-constant-logical-operand -Wno-long-long \
                          -Wno-missing-field-initializers \
                          -Wno-overlength-strings -Wno-parentheses-equality \
                          -Wno-self-assign -Wno-tautological-compare \
                          -Wno-unused-parameter -Wno-unused-value \
                          -Wunused-variable -Wundef
   * strip command:       strip -A -n
   * install doc:         rdoc
   * MJIT support:        yes
   * YJIT support:        no
   * man page type:       doc
   * BASERUBY -v:         ruby 2.6.10p210 (2022-04-12 revision 67958) \
                          [universal.x86_64-darwin22]

However, when I compile with make -j2, I see that it's disabled:

% make -j2
	BASERUBY = /usr/bin/ruby --disable=gems
	CC = clang
	LD = ld
	LDSHARED = clang -dynamiclib
	CFLAGS = -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef  -pipe -arch x86_64 -arch arm64
	XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/universal-darwin22 -I./include -I. -I./enc/unicode/15.0.0 
	CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   
	DLDFLAGS = -Wl,-multiply_defined,suppress -fstack-protector-strong -Wl,-pie -framework CoreFoundation  -arch x86_64 -arch arm64
	SOLIBS = -ldl -lobjc -lpthread 
	LANG = en_US.UTF-8
	LC_ALL = 
	LC_CTYPE = 
	MFLAGS = - --jobserver-fds=3,4 -j
	RUSTC = rustc
	YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/Users/benhamilton/Developer/OpenSource/ruby/yjit/target/release/' ./yjit/src/lib.rs
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: x86_64-apple-darwin22.3.0
Thread model: posix
InstalledDir: /Applications/Xcode_14.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[snip]

building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-x86_64.h
building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-arm64.h
This compiler (clang -fdeclspec  -g  -pipe -arch arm64 -w) looks not supported for MJIT. Giving up to generate MJIT header.
This compiler (clang -fdeclspec  -g  -pipe -arch x86_64 -w) looks not supported for MJIT. Giving up to generate MJIT header.

The problem is here:

• https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/common.mk#L260
• https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/tool/transform_mjit_header.rb#L24
• https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/tool/transform_mjit_header.rb#L174

This logic runs vm.c through the C preprocessor and looks through the output for the __clang__ preprocessor macro.

However, clang's C preprocessor only prints macro values with the -dM flag. We can add this flag, but it won't work with the MSVC compiler, so we need to only add it for gcc/clang.

[k0kubun (Takashi Kokubun)]

• Status changed from Open to Assigned
• Assignee set to k0kubun (Takashi Kokubun)

[k0kubun (Takashi Kokubun)]

• Status changed from Assigned to Closed

The build process is replaced in [Feature #19420], so this issue should be irrelevant today.