Bug #13080: [mingw regression] broken openssl extension in 2.4.0p0

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
• Backport: 2.2: DONTNEED, 2.3: DONTNEED, 2.4: DONE

[jonforums (Jon Forums)]

When building using slightly modified rubyinstaller build recipes and a msys2/mingw64-6.2.0 (32bit) toolchain, the build fails to create the openssl extension when using openssl 1.0.2j or 1.1.0c.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- openssl (LoadError)
        from C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'

Attached are the failing openssl mkmf.log and Makefile files from the 2.4.0p0 build.

I can provide good versions of the files from a 2.3.3p224 x32 build using openssl 1.0.2j if needed.

[nobu (Nobuyoshi Nakada)]

From https://bugs.ruby-lang.org/attachments/6316/bad_mkmf_2.4.log#L229, your ssl library has references to GDI32.dll. I'm not sure why rand_win.c needs it though.
Probably you need --with-openssl-config option for a proper pkg-config file.

[jonforums (Jon Forums)]

Ok. I'll play with my openssl build script https://github.com/jonforums/buildlets/blob/master/build_openssl.ps1#L62 to see if I make it work.

Strange that the same openssl 1.0.2j library works fine with 2.3.3 but not 2.4.0.

[jonforums (Jon Forums)]

The bad news is that my custom openssl 1.0.2j library and the ones built by the rubyinstaller project (32 and 64bit) both have pkg-config info like the following. Openssl 1.1.0c (32 and 64bit) built with mingw64-6.2.0 also have the -lgdi32 dep.

#file: libcrypto.pc
...
Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 1.0.2j
Requires: 
Libs: -L${libdir} -lcrypto
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}

#file: libssl.pc
...
Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 1.0.2j
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}

petudio https://winitor.com/ show that gdi32.dll is used by my INSTALL_DIR/lib/ruby/2.3.0/i386-mingw32/openssl.so file for calling these Windows API's:

• CreateCompatibleBitmap
• DeleteObject
• GetDlBits
• GetDeviceCaps
• GetObjectA

I don't yet know why openssl 1.0.2j needs these gdi32.dll functions, and I haven't tweaked my openssl build config to see if the gdi32.dll dep can be removed. I also haven't diffed ruby's openssl 2.3.0 and 2.4.0 build config to see what changed.

That said, I doubt I can make the gdi32.dll dep go away.

Anything you see in 2.4.0's ruby openssl config that would cause both 1.0.2j and 1.1.0c to stop building the extension?

[rhenium (Kazuki Yamaguchi)]

Apparently my change broke it:

https://github.com/ruby/openssl/commit/5ba12a364fa5019f0e0f1e356087c001330097f1

I never thought OpenSSL would actually use gdi32.dll, but it does! RAND_screen() which used those gdi32.dll functions was removed in OpenSSL 1.1.0, but it now fails because crypt32.dll is missing.

Please apply this patch, or use pkg-config, or use shared libraries of OpenSSL (which I recommend).

--- a/ext/openssl/extconf.rb
+++ b/ext/openssl/extconf.rb
@@ -37,6 +37,12 @@
 Logging::message "=== Checking for required stuff... ===\n"
 result = pkg_config("openssl") && have_header("openssl/ssl.h")
 unless result
+  if $mswin || $mingw
+    # required for static OpenSSL libraries
+    have_library("gdi32") # OpenSSL <= 1.0.2 (for RAND_screen())
+    have_library("crypt32")
+  end
+
   result = have_header("openssl/ssl.h")
   result &&= %w[crypto libeay32].any? {|lib| have_library(lib, "CRYPTO_malloc")}
   result &&= %w[ssl ssleay32].any? {|lib| have_library(lib, "SSL_new")}

[jonforums (Jon Forums)]

Yes, that was very surprising to find windows openssl using gdi32.dll.

I've got more testing to do, but your patch appears to work for both 1.0.2j and 1.1.0c on 2.4.0p0:

Thank you.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.1.0c  10 Nov 2016
...
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j  26 Sep 2016
...
$ make test
...
gcc.exe (i686-win32-sjlj-rev1, Built by MinGW-W64 project) 6.2.0
...
PASS all 1010 tests
exec ./miniruby.exe -I../../../ruby-git/lib -I. -I.ext/common  ../../../ruby-git/tool/runruby.rb --extout=.ext  -- --disable-gems "../../../ruby-git/bootstraptest/runner.rb" --ruby="ruby.exe --disable-gems"   ../../../ruby-git/KNOWNBUGS.rb
2016-12-30 13:06:35 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]

KNOWNBUGS.rb  PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded

Are you planning to commit this patch to trunk and have it backported to ruby_2_4?

[jonforums (Jon Forums)]

Before you decide whether to commit your patch to support using static openssl libs, I'll double check that a 32bit build with the rubyinstaller-built openssl lib from https://bintray.com/oneclick/OpenKnapsack/openssl/1.0.2j#files also works.

Rubyinstaller builds shared openssl libs

https://github.com/oneclick/knapsack-recipes/blob/master/openssl/openssl-1.0.2j.knapfile#L31

while I'm using static openssl libs to mininize potential {libeay32,ssleay32}.dll windows PATH conflicts.

I think your current extconf.rb patch will work fine with both, but will double check.

No reason to create a New Year's surprise for the rubyinstaller team.

[jonforums (Jon Forums)]

Your patch also works when using the rubyinstaller-built shared openssl 1.0.2j lib.

make test-all segfaults, but test segfaults and other unresolved testing failures are typical when building ruby on windows.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j  26 Sep 2016
...
$ make test
...
PASS all 1010 tests
...
2016-12-30 21:50:05 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]

KNOWNBUGS.rb  PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded

Your patch gives the user the option of building whatever openssl ext setup works best for their environment, and doesn't appear to break the rubyinstaller builds.

Thanks.

• Status changed from Open to Closed

Applied in changeset r57482.

---

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3

---

Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

[rhenium (Kazuki Yamaguchi)]

• Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED

The patch is committed to ruby/openssl:

https://github.com/ruby/openssl/commit/a2dc925ac646f30e7d518158d7931ff422444ffe

and imported to Ruby tree at r57482.

2.4 branch maintainer: Could you backport r57482? It fixes this issue, #13170 and a GitHub issue ruby/openssl#96.

[naruse (Yui NARUSE)]

• Backport changed from 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: DONE

ruby_2_4 r57881 merged revision(s) 57482.

[rhenium (Kazuki Yamaguchi)]

• Related to Bug #9817: The extconf.rb for OpenSSL assumes MingW added