Bug #8690

Enciphering a key with AES-128-CBC-HMAC-SHA1 crashes in OpenSSL

Added by drbrain (Eric Hodel) over 7 years ago. Updated over 4 years ago.

Third Party's Issue
Target version:
ruby -v:
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0]


While I realize this is probably wrong, I was trying to find a cipher that JRuby 1.7.4 supported and encountered this crash.

My OpenSSL version is:

$ ruby -ropenssl -e 'p OpenSSL::OPENSSL_VERSION'
"OpenSSL 1.0.1e 11 Feb 2013"

The crash:

$ ruby crash.rb
crash.rb:10: [BUG] Segmentation fault
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0]

-- Crash Report log information --------------------------------------------
See Crash Report log file under the one of following:
* ~/Library/Logs/CrashReporter
* /Library/Logs/CrashReporter
* ~/Library/Logs/DiagnosticReports
* /Library/Logs/DiagnosticReports
the more detail of.

-- Control frame information -----------------------------------------------
c:0004 p:---- s:0015 e:000014 CFUNC :initialize
c:0003 p:---- s:0013 e:000012 CFUNC :new
c:0002 p:0083 s:0008 E:002448 EVAL crash.rb:10 [FINISH]
c:0001 p:0000 s:0002 E:000868 TOP [FINISH]

crash.rb:10:in <main>'
crash.rb:10:in `initialize'

-- C level backtrace information -------------------------------------------

-- Other runtime information -----------------------------------------------

  • Loaded script: crash.rb

  • Loaded features:

    1 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/enc/encdb.bundle
    2 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/enc/trans/transdb.bundle
    3 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/rbconfig.rb
    4 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/compatibility.rb
    5 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/defaults.rb
    6 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/deprecate.rb
    7 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/errors.rb
    8 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/version.rb
    9 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/requirement.rb
    10 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/platform.rb
    11 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/basic_specification.rb
    12 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/stub_specification.rb
    13 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/specification.rb
    14 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/exceptions.rb
    15 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_gem.rb
    16 /usr/local/lib/ruby/2.0.0/thread.rb
    17 /usr/local/lib/ruby/2.0.0/monitor.rb
    18 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb
    19 /usr/local/lib/ruby/site_ruby/2.0.0/rubygems.rb
    20 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/digest.bundle
    21 /usr/local/lib/ruby/2.0.0/digest.rb
    22 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/openssl.bundle
    23 /usr/local/lib/ruby/2.0.0/openssl/bn.rb
    24 /usr/local/lib/ruby/2.0.0/openssl/cipher.rb
    25 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/stringio.bundle
    26 /usr/local/lib/ruby/2.0.0/openssl/config.rb
    27 /usr/local/lib/ruby/2.0.0/openssl/digest.rb
    28 /usr/local/lib/ruby/2.0.0/openssl/x509.rb
    29 /usr/local/lib/ruby/2.0.0/openssl/buffering.rb
    30 /usr/local/lib/ruby/2.0.0/x86_64-darwin12.4.0/fcntl.bundle
    31 /usr/local/lib/ruby/2.0.0/openssl/ssl.rb
    32 /usr/local/lib/ruby/2.0.0/openssl.rb

You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details:

Abort trap: 6

Attached are the script I used to reproduce the crash and the OS X crash report.


crash.rb (240 Bytes) crash.rb script to reproduce drbrain (Eric Hodel), 07/26/2013 05:45 AM
ruby_2013-07-25-134113_kault.crash (9.16 KB) ruby_2013-07-25-134113_kault.crash OS X crash report drbrain (Eric Hodel), 07/26/2013 05:45 AM

Updated by drbrain (Eric Hodel) over 7 years ago

AES-256-CBC-HMAC-SHA1 also crashes with the same stack trace

Updated by MartinBosslet (Martin Bosslet) over 7 years ago

We just talked about it and found out that it's an issue only with Intel processors with the AES-NI instructions. The algorithm is only available for those CPUs, and I was able to reproduce the crash there. It happens in the very last line where the RSA key is instantiated. I'll have to debug further to find out if this is happening in Ruby OpenSSL or OpenSSL itself.


Updated by zzak (Zachary Scott) over 5 years ago

  • Assignee changed from MartinBosslet (Martin Bosslet) to openssl

Updated by rhenium (Kazuki Yamaguchi) over 4 years ago

  • Backport deleted (1.9.3: UNKNOWN, 2.0.0: UNKNOWN)
  • Status changed from Assigned to Third Party's Issue

I believe this is a bug of OpenSSL that is fixed in OpenSSL 1.0.2.

Since it happens in PEM_read_bio_RSAPrivateKey(), I think there is nothing we can do.

key.pem: generated with"AES-128-CBC-HMAC-SHA1"), "a"*16)

Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC-HMAC-SHA1,DD14AA4A3742E7F40460B71157CF90EC


and openssl rsa explodes:

bash$ openssl version
OpenSSL 1.0.1u-dev  xx XXX xxxx
bash$ openssl rsa -in key.pem -passin pass:abc
Segmentation fault (core dumped)

Also available in: Atom PDF