Project

General

Profile

Bug #13080

[mingw regression] broken openssl extension in 2.4.0p0

Added by jonforums (Jon Forums) almost 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
[ruby-core:78878]

Description

When building using slightly modified rubyinstaller build recipes and a msys2/mingw64-6.2.0 (32bit) toolchain, the build fails to create the openssl extension when using openssl 1.0.2j or 1.1.0c.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- openssl (LoadError)
        from C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'

Attached are the failing openssl mkmf.log and Makefile files from the 2.4.0p0 build.

I can provide good versions of the files from a 2.3.3p224 x32 build using openssl 1.0.2j if needed.


Files

bad_mkmf_2.4.log (21.9 KB) bad_mkmf_2.4.log jonforums (Jon Forums), 12/27/2016 07:46 PM
bad_Makefile_2.4 (6.22 KB) bad_Makefile_2.4 jonforums (Jon Forums), 12/27/2016 07:46 PM

Related issues

Related to Ruby master - Bug #9817: The extconf.rb for OpenSSL assumes MingWClosedActions

Associated revisions

Revision 8795838f
Added by rhe almost 3 years ago

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3


Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57482 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 57482
Added by rhenium (Kazuki Yamaguchi) almost 3 years ago

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3


Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

Revision 57482
Added by rhe almost 3 years ago

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3


Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

Revision 57482
Added by rhe almost 3 years ago

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3


Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

Revision c1478b66
Added by naruse (Yui NARUSE) over 2 years ago

merge revision(s) 57482: [Backport #13080] [Backport #13170]

    openssl: import v2.0.3

    Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
    2.0.2 (imported at r57146) can be found at:

      https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3

    ----------------------------------------------------------------
    Corey Bonnell (1):
          Fix for ASN1::Constructive 'each' implementation

    Kazuki Yamaguchi (10):
          Fix build with static OpenSSL libraries on Windows
           ([ruby-core:78878] [Bug #13080])
          Merge pull request #96 from CBonnell/master
          Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
          appveyor.yml: update OpenSSL version to 1.0.2j
          buffering: fix typo in doc
          test/envutil: fix assert_raise_with_message
          x509: fix OpenSSL::X509::Name#eql?
           ([ruby-core:79310] [Bug #13170])
          ruby-openssl-docker: update versions of Ruby and OpenSSL
          .travis.yml: test with Ruby 2.4
          Ruby/OpenSSL 2.0.3

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@57881 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 57881
Added by naruse (Yui NARUSE) over 2 years ago

merge revision(s) 57482: [Backport #13080] [Backport #13170]

openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

  https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3

----------------------------------------------------------------
Corey Bonnell (1):
      Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
      Fix build with static OpenSSL libraries on Windows
       ([ruby-core:78878] [Bug #13080])
      Merge pull request #96 from CBonnell/master
      Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
      appveyor.yml: update OpenSSL version to 1.0.2j
      buffering: fix typo in doc
      test/envutil: fix assert_raise_with_message
      x509: fix OpenSSL::X509::Name#eql?
       ([ruby-core:79310] [Bug #13170])
      ruby-openssl-docker: update versions of Ruby and OpenSSL
      .travis.yml: test with Ruby 2.4
      Ruby/OpenSSL 2.0.3

History

Updated by nobu (Nobuyoshi Nakada) almost 3 years ago

From https://bugs.ruby-lang.org/attachments/6316/bad_mkmf_2.4.log#L229, your ssl library has references to GDI32.dll. I'm not sure why rand_win.c needs it though.
Probably you need --with-openssl-config option for a proper pkg-config file.

Updated by jonforums (Jon Forums) almost 3 years ago

Ok. I'll play with my openssl build script https://github.com/jonforums/buildlets/blob/master/build_openssl.ps1#L62 to see if I make it work.

Strange that the same openssl 1.0.2j library works fine with 2.3.3 but not 2.4.0.

Updated by jonforums (Jon Forums) almost 3 years ago

The bad news is that my custom openssl 1.0.2j library and the ones built by the rubyinstaller project (32 and 64bit) both have pkg-config info like the following. Openssl 1.1.0c (32 and 64bit) built with mingw64-6.2.0 also have the -lgdi32 dep.

#file: libcrypto.pc
...
Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 1.0.2j
Requires: 
Libs: -L${libdir} -lcrypto
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}

#file: libssl.pc
...
Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 1.0.2j
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}

petudio https://winitor.com/ show that gdi32.dll is used by my INSTALL_DIR/lib/ruby/2.3.0/i386-mingw32/openssl.so file for calling these Windows API's:

  • CreateCompatibleBitmap
  • DeleteObject
  • GetDlBits
  • GetDeviceCaps
  • GetObjectA

I don't yet know why openssl 1.0.2j needs these gdi32.dll functions, and I haven't tweaked my openssl build config to see if the gdi32.dll dep can be removed. I also haven't diffed ruby's openssl 2.3.0 and 2.4.0 build config to see what changed.

That said, I doubt I can make the gdi32.dll dep go away.

Anything you see in 2.4.0's ruby openssl config that would cause both 1.0.2j and 1.1.0c to stop building the extension?

Updated by rhenium (Kazuki Yamaguchi) almost 3 years ago

Apparently my change broke it:

https://github.com/ruby/openssl/commit/5ba12a364fa5019f0e0f1e356087c001330097f1

I never thought OpenSSL would actually use gdi32.dll, but it does! RAND_screen() which used those gdi32.dll functions was removed in OpenSSL 1.1.0, but it now fails because crypt32.dll is missing.

Please apply this patch, or use pkg-config, or use shared libraries of OpenSSL (which I recommend).

--- a/ext/openssl/extconf.rb
+++ b/ext/openssl/extconf.rb
@@ -37,6 +37,12 @@
 Logging::message "=== Checking for required stuff... ===\n"
 result = pkg_config("openssl") && have_header("openssl/ssl.h")
 unless result
+  if $mswin || $mingw
+    # required for static OpenSSL libraries
+    have_library("gdi32") # OpenSSL <= 1.0.2 (for RAND_screen())
+    have_library("crypt32")
+  end
+
   result = have_header("openssl/ssl.h")
   result &&= %w[crypto libeay32].any? {|lib| have_library(lib, "CRYPTO_malloc")}
   result &&= %w[ssl ssleay32].any? {|lib| have_library(lib, "SSL_new")}

Updated by jonforums (Jon Forums) almost 3 years ago

Yes, that was very surprising to find windows openssl using gdi32.dll.

I've got more testing to do, but your patch appears to work for both 1.0.2j and 1.1.0c on 2.4.0p0:

Thank you.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.1.0c  10 Nov 2016
...
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j  26 Sep 2016
...
$ make test
...
gcc.exe (i686-win32-sjlj-rev1, Built by MinGW-W64 project) 6.2.0
...
PASS all 1010 tests
exec ./miniruby.exe -I../../../ruby-git/lib -I. -I.ext/common  ../../../ruby-git/tool/runruby.rb --extout=.ext  -- --disable-gems "../../../ruby-git/bootstraptest/runner.rb" --ruby="ruby.exe --disable-gems"   ../../../ruby-git/KNOWNBUGS.rb
2016-12-30 13:06:35 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]

KNOWNBUGS.rb  PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded

Are you planning to commit this patch to trunk and have it backported to ruby_2_4?

Updated by jonforums (Jon Forums) almost 3 years ago

Before you decide whether to commit your patch to support using static openssl libs, I'll double check that a 32bit build with the rubyinstaller-built openssl lib from https://bintray.com/oneclick/OpenKnapsack/openssl/1.0.2j#files also works.

Rubyinstaller builds shared openssl libs

https://github.com/oneclick/knapsack-recipes/blob/master/openssl/openssl-1.0.2j.knapfile#L31

while I'm using static openssl libs to mininize potential {libeay32,ssleay32}.dll windows PATH conflicts.

I think your current extconf.rb patch will work fine with both, but will double check.

No reason to create a New Year's surprise for the rubyinstaller team.

Updated by jonforums (Jon Forums) almost 3 years ago

Your patch also works when using the rubyinstaller-built shared openssl 1.0.2j lib.

make test-all segfaults, but test segfaults and other unresolved testing failures are typical when building ruby on windows.

$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j  26 Sep 2016
...
$ make test
...
PASS all 1010 tests
...
2016-12-30 21:50:05 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]

KNOWNBUGS.rb  PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded

Your patch gives the user the option of building whatever openssl ext setup works best for their environment, and doesn't appear to break the rubyinstaller builds.

Thanks.

#8

Updated by Anonymous almost 3 years ago

  • Status changed from Open to Closed

Applied in changeset r57482.


openssl: import v2.0.3

Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:

https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3


Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation

Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3

Updated by rhenium (Kazuki Yamaguchi) almost 3 years ago

  • Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED

The patch is committed to ruby/openssl:

https://github.com/ruby/openssl/commit/a2dc925ac646f30e7d518158d7931ff422444ffe

and imported to Ruby tree at r57482.

2.4 branch maintainer: Could you backport r57482? It fixes this issue, #13170 and a GitHub issue ruby/openssl#96.

Updated by naruse (Yui NARUSE) over 2 years ago

  • Backport changed from 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: DONE

ruby_2_4 r57881 merged revision(s) 57482.

#11

Updated by rhenium (Kazuki Yamaguchi) about 2 years ago

  • Related to Bug #9817: The extconf.rb for OpenSSL assumes MingW added

Also available in: Atom PDF