Project

General

Profile

Actions

Feature #17845

open

Windows Ruby - ucrt build?

Added by MSP-Greg (Greg L) about 2 months ago. Updated 6 days ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
[ruby-core:103686]

Description

Currently, Windows Ruby is normally compiled two ways.

The first, mswin, is compiled using Microsoft's current Visual C compiler, and links to the universal runtime (ucrt).

The second, mingw, is compiled using MinGW gcc. This links to msvcrt, an older version of Microsoft's Visual C runtime.

Previously, all the MSYS2 MinGW packages linked to msvcrt. The MSYS2 project is now releasing build tools and packages linking to ucrt.

MSYS2 has provided ruby packages, and GitHub user @Biswa96 contributed https://github.com/msys2/MINGW-packages/pull/8518, allowing Ruby to compile with ucrt.

I tried the patch with ruby-loco, and it builds. There are some test issues, haven't had a chance to look at them yet.

Normally, using exe/dll/so files together that use different versions of the runtime is not a good idea. Building extension gems with ucrt may provide gems that can be used with mswin, but can't be compiled due to issues with Visual C (vs gcc). Also, packages from the vcpkg project may be compatible with Ruby ucrt.

The reason for this post is that Windows Ruby built with ucrt is essentially another platform, but the build 'looks like' a mingw build.

This is really a third platform choice. What should it be called? Things like CONFIG['RUBY_SO_NAME'], RUBY_PLATFORM, etc?


Files

ruby-mingw32-ucrt.patch (622 Bytes) ruby-mingw32-ucrt.patch xtkoba (Tee KOBAYASHI), 06/07/2021 02:15 PM
ruby-configure-mingw32-ucrt.patch (687 Bytes) ruby-configure-mingw32-ucrt.patch xtkoba (Tee KOBAYASHI), 06/09/2021 12:27 AM
ruby-configure-coroutine-mingw32.patch (351 Bytes) ruby-configure-coroutine-mingw32.patch xtkoba (Tee KOBAYASHI), 06/11/2021 01:27 AM
ruby-i386-mingw32-ucrt-undef.patch (398 Bytes) ruby-i386-mingw32-ucrt-undef.patch xtkoba (Tee KOBAYASHI), 06/13/2021 06:15 PM
ruby-date-mingw32-ucrt-timezone.patch (333 Bytes) ruby-date-mingw32-ucrt-timezone.patch xtkoba (Tee KOBAYASHI), 06/13/2021 06:22 PM

Updated by xtkoba (Tee KOBAYASHI) about 2 months ago

I wonder if there is a standard target triplet for MinGW + UCRT. I came up with x86_64-w64-mingw32-ucrt, but when googled it hits only one page. I suppose something like x64-mingw32-ucrt is acceptable for target_os in configure, as there is a preceding example for Android: aarch64-linux-android vs aarch64-linux (GNU/Linux), although I have no idea whether it is sufficient to just suffix target_os.

Updated by MSP-Greg (Greg L) about 2 months ago

MinGW msvcrt packages have a prefix of mingw-w64-x86_64,
MinGW  ucrt  packages have a prefix of mingw-w64-ucrt-x86_64.

Updated by larskanis (Lars Kanis) about 2 months ago

I wonder if there is a standard target triplet for MinGW + UCRT.

The compiler triplet for MINGW with UCRT is still 'x86_64-w64-mingw32'. This is defined in MSYS2 here. It is not distinct from MSVCRT based MINGW because it differs only in a library (msvcrt.dll vs. ucrtbase.dll) and the dependent libraries are not part of the compiler triplet.

I did some testing with the patch used in MSYS2. It doesn't establish any new platform strings, so that RbConfig is mostly the same with MSVCRT and UCRT. Extension libraries linked to MSVCRT usually load into a UCRT based ruby at the first attempt, but they fail later on. FFI fails when using long double values and Nokogiri fails in some test case while calling free() of msvcrt.dll. These are expected issues, when mixing MSVCRT and UCRT into one process.

So, I would propose a patch similar to this rather than the one in MSYS2. This sets

  • CONFIG['arch'] = 'x64-mingw32' # equal in MSVCRT and UCRT
  • CONFIG['sitearch'] = 'x64-ucrt' # vs. 'x64-msvcrt' in MSVCRT
  • CONFIG['RUBY_SO_NAME'] = 'x64-ucrt-ruby310' # vs. 'x64-msvcrt-ruby310' in MSVCRT
  • RUBY_PLATFORM = 'x64-mingw32' # equal in MSVCRT and UCRT

I'm uncertain if the use of UCRT should be reflected in RUBY_PLATFORM. In any case it should still contain "mingw" since I've seen a lot of code checking for Windows by RUBY_PLATFORM=~/mingw|mswin/.

Similarly Rubygems should use a different platform string for UCRT and MSVCRT based gems, since binary extensions are incompatible. This is still missing in the patch above. Maybe 'x64-mingw32-ucrt' as rubygems platform or 'x64-mingwucrt' or the platform of MSWIN?

My rough idea is to switch to UCRT with RubyInstaller-3.1. That means that all current releases keep using MSVCRT and that there will be no MSVCRT based RubyInstaller-3.1.

Updated by MSP-Greg (Greg L) about 2 months ago

larskanis (Lars Kanis)

Sorry, I meant to ping you immediately, but got distracted. Also, I saw your changes to ridk, but didn't see your patch file.

Anyway, I agree that there needs to be some way to distinguish between mingw and ucrt builds. And RubyGems/Bundler will need changes.

I haven't looked at runtime dependencies for a while. Question - wouldn't some (or most) extension gems compiled with ucrt be compatible with mswin builds, since there isn't a MSVC runtime mismatch?

I'll see what can be done to get ucrt usable with GH Actions...

Updated by xtkoba (Tee KOBAYASHI) 17 days ago

The test TestEnv#test_huge_value showed a difference between msvcrt and ucrt for x64-mingw32:

  1) Failure:
TestEnv#test_huge_value [C:/CIFS/ruby-trunk/test/ruby/test_env.rb:472]:
Exception raised:
<#<Errno::EINVAL: Invalid argument - ruby_setenv(foo)>>
Backtrace:
  C:/CIFS/ruby-trunk/test/ruby/test_env.rb:472:in `[]='

The test code is shown below. It seems that x64-mingw32 + ucrt behaves here as if /mswin/ =~ RUBY_PLATFORM and not like msvcrt. To make this test work correctly we need a way to distinguish between the two.

  def test_huge_value
    huge_value = "bar" * 40960
    ENV["foo"] = "bar"
    if /mswin/ =~ RUBY_PLATFORM
      assert_raise(Errno::EINVAL) { ENV["foo"] = huge_value }
      assert_equal("bar", ENV["foo"])
    else
      assert_nothing_raised { ENV["foo"] = huge_value }
      assert_equal(huge_value, ENV["foo"])
    end
  end
Actions #6

Updated by naruse (Yui NARUSE) 15 days ago

  • Target version set to 3.1
  • Tracker changed from Misc to Feature

Updated by xtkoba (Tee KOBAYASHI) 15 days ago

The attached patch is essentially a cleanup of that by @Biswa96 from GitHub. With this ./configure rb_cv_msvcrt=ucrt140 should work for Mingw-w64 with the default crt being UCRT [1].

[1] https://sourceforge.net/p/mingw-w64/mailman/message/36621319/

It would be desirable that the configure script is improved so that the checking for mingw32 runtime DLL can find the conftest executable linked against UCRT.

Updated by xtkoba (Tee KOBAYASHI) 13 days ago

A patch is attached for the configure script to set rb_cv_msvcrt=ucrt and RT_VER=140 when UCRT is used.

Updated by xtkoba (Tee KOBAYASHI) 11 days ago

Pattern matching for target_os in configure script should be permissive if we consider suffixing something onto "mingw32".

Updated by xtkoba (Tee KOBAYASHI) 9 days ago

_WIN64 is not defined on i386-mingw32 + ucrt.

Updated by xtkoba (Tee KOBAYASHI) 9 days ago

Or should extconf.rb be improved to notice deprecation?

../../../ext/date/date_core.c:7803:17: warning: 'timezone' is deprecated: Only provided for source compatibility; this variable might not always be accurate when linking to UCRT. [-Wdeprecated-declarations]
    of = (long)-timezone;
                ^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/time.h:257:32: note: 'timezone' has been explicitly marked deprecated here
  _CRTIMP extern long timezone __MINGW_ATTRIB_DEPRECATED_UCRT;
                               ^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/time.h:249:5: note: expanded from macro '__MINGW_ATTRIB_DEPRECATED_UCRT'
    __MINGW_ATTRIB_DEPRECATED_MSG( \
    ^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/_mingw.h:182:58: note: expanded from macro '__MINGW_ATTRIB_DEPRECATED_MSG'
#define __MINGW_ATTRIB_DEPRECATED_MSG(x) __attribute__ ((__deprecated__(x)))
                                                         ^

Updated by xtkoba (Tee KOBAYASHI) 9 days ago

A separate ticket for -Wparentheses warning issue: #17946

Updated by larskanis (Lars Kanis) 6 days ago

Thank you xtkoba (Tee KOBAYASHI) for working on this! I replaced my previous patches by the above patches in rubyinstaller2-packages.

One thing I noticed is that you disabled the call to _CrtSetReportMode(_CRT_ASSERT, 0) in MINGW-UCRT. Actually gcc provides the function, if crtdbg.h is included. However I don't know if it makes any functional difference to call it or not.

Or should extconf.rb be improved to notice deprecation?

IMHO it's OK to use defines and to keep extconf.rb untouched.

Updated by xtkoba (Tee KOBAYASHI) 6 days ago

On Mingw-w64 _CrtSetReportMode is defined in crtdbg.h as

#define _CrtSetReportMode(t,f) ((int)0)

thus I believe the definition in win32/win32.c makes no difference although it may cause a -Wmacro-redefined warning.

Updated by larskanis (Lars Kanis) 6 days ago

The rubyinstaller-head nightly build is compiled with MINGW-UCRT since a few days. So all projects that use ruby-setup with version "head" or Appveyor with rubyinstaller-head download run on it now. Rubyinstaller-head already triggered a small issue in ffi and fxruby but other projects seems to run smooth.

The experience with rubyinstaller-head strengthens my plan to switch to UCRT with RubyInstaller-3.1.

Regarding CI runs: ruby-setup with version "mingw" is still on MSVCRT and https://github.com/MSP-Greg/setup-ruby-pkgs doesn't support UCRT builds so far.

MSP-Greg (Greg L): Would you mind to add support to setup-ruby-pkgs? Do you want to switch your ruby-logo build to UCRT as well or add it as another architecture?

Updated by MSP-Greg (Greg L) 6 days ago

@larkanis

Do you want to switch your ruby-logo build to UCRT as well or add it as another architecture?

I think adding as another architecture would be best? Maybe tag it as ruby-ucrt?

Would you mind to add support to setup-ruby-pkgs?

I'll open an issue about adding the ucrt build tools to the Actions images so a full download isn't required.

Also, there's a lot of patches here, will they be added to ruby/ruby?

Re setup-ruby-pkgs, maybe the best solution is to install ucrt packages matching the listed mingw package when the build is ruby-head (or head)?

Not sure whether to discuss here or in a setup-ruby-pkgs issue. I think this affects setup-ruby, as it adds the tools paths to ENV['Path']... Eregon (Benoit Daloze)?

Sorry, I've been following this, but didn't expect you'd convert your build so quickly. I'm fine with it though...

Actions

Also available in: Atom PDF