Bug #16651: Extensions Do Not Compile on Mingw64 with mingw32-make

• Status: Closed
• Priority: Normal
• Assignee: -
• Target version: -
• ruby -v: ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x64-mingw32]
• Backport: 2.6: REQUIRED, 2.7: REQUIRED, 3.0: DONE

[cfis (Charlie Savage)]

When mkmf.rb creates a Makefile for an extension, it will generate something that looks like this:

srcdir = .
topdir = C/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/x64-mingw32

Notice the topdir path is c/ without the ":" Its only the topdir that does this, all other paths in the makefile use the "c:/" style.

mkmf.rb intentionally does that, see line 1098:

      def mkintpath(path)
        # mingw uses make from msys and it needs special care
        # converts from C:\some\path to /C/some/path
        path = path.dup
        path.tr!('\\', '/')
        path.sub!(/\A([A-Za-z]):(?=\/)/, '/\1') <-------- This line
        path
      end

But this is wrong, and causes errors like this (this is compiling the debase gem but it doesn't matter what c extension you use):

make: *** No rule to make target 'C/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/ruby.h', needed by 'breakpoint.o'.  Stop.

The fix is simple, just delete that line. The makefile should look like this:

srcdir = .
topdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/x64-mingw32

Note I'm not the first person to see this, but I've just been manually fixing it over the years. Would be good to really fix it.

https://github.com/oneclick/rubyinstaller2/issues/105
https://github.com/oneclick/rubyinstaller2/issues/47
https://github.com/tmm1/http_parser.rb/issues/55

Note some of those tickets put the blame on using mingw-make versus msys make. But on my system, neither work with the "c/" style path but both work with the "c:/" style path.

[cfis (Charlie Savage)]

Ping?

[cfis (Charlie Savage)]

This is still and issue with Ruby 2.7.2.

The offending line of code is now at line 1908 in mkmf.rb.

[nobu (Nobuyoshi Nakada)]

It would replace "C:/MSYS64/..." as "/C/MSYS64/...".
Where did a slash go?

[cfis (Charlie Savage)]

Sorry, missed your comment. The problem is /C/ is wrong.

This is still an issue with Ruby 3.0.0 and causes building extensions to fail.

Install msys2, install mingw64. Then try to build any extension. You'll get this error message:

* No rule to make target '/C/msys64/usr/local/ruby-3.0.0/include/ruby-3.0.0/ruby.h', needed by 'breakpoint.o'.  Stop.

If you comment out this line:

#path.sub!(/\A([A-Za-z]):(?=\/)/, '/\1')

Then the paths becomes:

C:/msys64/usr/local/ruby-3.0.0/include/ruby-3.0.0/ruby.h

And the problem is fixed. That is the correct path we want.

[cfis (Charlie Savage)]

Any thoughts?

[xtkoba (Tee KOBAYASHI)]

With RubyInstaller's I got the following result in the Command Prompt of Windows 10:

C:\Ruby30-x64\bin>irb
irb(main):001:0> Dir.pwd
=> "C:/Ruby30-x64/bin"
irb(main):002:0> File.exist?("C:/Ruby30-x64")
=> true
irb(main):003:0> File.exist?("C/Ruby30-x64")
=> false
irb(main):004:0> File.exist?("/C:/Ruby30-x64")
=> false
irb(main):005:0> File.exist?("/C/Ruby30-x64")
=> false
irb(main):006:0> File.exist?("c:/Ruby30-x64")
=> true
irb(main):007:0> File.exist?("c/Ruby30-x64")
=> false
irb(main):008:0> File.exist?("/c:/Ruby30-x64")
=> false
irb(main):009:0> File.exist?("/c/Ruby30-x64")
=> false
irb(main):010:0>

I suppose MSYS2 might be necessary for the notation like /c/ [1]. I might be talking nonsense because I don't know about MSYS2 in detail.

[1] https://www.msys2.org/wiki/How-does-MSYS2-differ-from-Cygwin/

[cfis (Charlie Savage)]

When running msys2 (ie, in an open console) the /c/ notation does work.

But it does not work in makefiles processed by mingw64. If you look at the makefiles that extconf.rb generates, they use paths like c:/abc/def except for this one case. And this one case breaks building extensions if you are using mingw make. Thus this should be changed (note that msys supports the c:/ format also).

[xtkoba (Tee KOBAYASHI)]

This issue is related to commit 389157d8cfab7b6e84cdcdb2863421bd2c93bd7f, which merged the patch posted as [ruby-core:21448]. The description of the patch reads that the character : in the path confuses make from MSYS because it is interpreted as a path separator. If this is still true, then we cannot simply leave : unremoved with a drive letter.

[cfis (Charlie Savage)]

Thanks for the background, that was very helpful. The difference is that I am using mingw64 make, not msys make. So:

• msys make supports /C/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0 and C:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0

• mingw make only supports c:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0

If you look at the generated makefiles (this is from gem install --platform ruby pg):

topdir = /C/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0/x64-mingw32

Notice the makefiles use both forms of the path.

Bottom line - the patch from 12 years ago is no longer useful. It is not needed for msys make and it breaks mingw-make. So can it be removed?

Thanks - Charlie

[cfis (Charlie Savage)]

• Subject changed from Extensions Do Not Compile on Mingw64 to Extensions Do Not Compile on Mingw64 with mingw64-make

[xtkoba (Tee KOBAYASHI)]

I suppose arch_hdrdir also should have been canonicalized by mkintpath, because otherwise it would cause trouble with VPATH in Makefile which is defined as:

VPATH = $(srcdir):$(arch_hdrdir)/ruby:$(hdrdir)/ruby

And I believe that the notation of VPATH as above does not work well with mingw32-make, because it allows only one path.

It seems clear that mingw32-make is not supported in the current build system of Ruby. I doubt if the maintainers have tried to support it ever.

One way of supporting both make from MSYS and mingw32-make would be to stop using VPATH, although I have no idea whether this is possible. FYI, VPATH seems to cause another trouble (Bug #12179), and it would be desirable if we could throw it away.

[cfis (Charlie Savage)]

VPATH works fine with arch_hdrdir in the form c:/msys64/etc - Ruby has been doing that on Windows for many years.

With my proposed fix both msys make and mingw64 make work correctly.

[nobu (Nobuyoshi Nakada)]

cfis (Charlie Savage) wrote in #note-12:

VPATH works fine with arch_hdrdir in the form c:/msys64/etc - Ruby has been doing that on Windows for many years.

Does it work with different drives too?

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)

[xtkoba (Tee KOBAYASHI)]

I suppose it is by luck that arch_hdrdir of the form C:/foo/bar is not causing problems with make from MSYS. In fact, in my environment (not Windows) gem install rails && rails new foo succeeds when VPATH is modified such that it does not contain $(arch_hdrdir)/ruby, or not even $(hdrdir)/ruby. I wonder if any extension in the world requires these paths contained in VPATH.

The proposed modification might be OK for make from MSYS if the definition of VPATH can be modified so that it never contains any drive letters. Note that some extensions (e.g. sassc) add additional paths to VPATH. And assuming that the modification is applied, I doubt if mingw32-make will successfully build an extension which really depends on VPATH containing multiple paths.

cfis (Charlie Savage) Could you please try to gem install sassc with mingw32-make? If it really does not work with multiple paths in VPATH, then it should fail to build that extension. Unfortunately I do not have access to Windows machines connected to the Internet.

[nobu (Nobuyoshi Nakada)]

Building io-console with 2.7.2 from RubyInstaller:

$ ruby -v ../src/ext/io/console/extconf.rb
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x64-mingw32]
checking for HAVE_RUBY_FIBER_SCHEDULER_H... no
checking for rb_scheduler_timeout()... no
creating Makefile

$ grep -e ^topdir -e ^arch_hdrdir Makefile
topdir = /C/Ruby27-x64/include/ruby-2.7.0
arch_hdrdir = C:/Ruby27-x64/include/ruby-2.7.0/x64-mingw32

$ make
compiling ../src/ext/io/console/console.c
linking shared-object io/console.so

It succeeds with no problem.

/C/ is not wrong in msys make, I'm not sure why the first description had the wrong path without the leading slash.

[nobu (Nobuyoshi Nakada)]

• Backport changed from 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN to 2.6: REQUIRED, 2.7: REQUIRED, 3.0: REQUIRED
• Subject changed from Extensions Do Not Compile on Mingw64 with mingw64-make to Extensions Do Not Compile on Mingw64 with mingw32-make

MinGW on MSys build is cross compilation.
No reason to use mingw32-make, which isn't for the build OS.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|799ea1d1540aca03542ee31647052343507a4b41.

---

mkmf.rb: convert also arch_hdrdir [Bug #16651]

[cfis (Charlie Savage)]

To answer the questions above:

• gem install sassc works fine
• using different drives, d:/ works fine

I don't understand the reason for this change - it just makes building on mingw64 with mingw-64 on windows more broken. If the format c:/ didn't work, then every native gem would have failed to compile because arch_hdrdir used that style. So clearly it does work.

Thus arch_hdrdir should not be changed, and instead topdir should have changed.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.6: REQUIRED, 2.7: REQUIRED, 3.0: REQUIRED to 2.6: REQUIRED, 2.7: REQUIRED, 3.0: DONE

ruby_3_0 5e21726cda22e3cb34127751aec7e9babb4308b3 merged revision(s) 799ea1d1540aca03542ee31647052343507a4b41.