Project

General

Profile

Actions

Bug #15841

closed

SegFault in OpenSSL::PKey::RSA#private_encrypt

Added by thekuwayama (tomoya kuwayama) almost 5 years ago. Updated over 4 years ago.

Status:
Closed
Target version:
-
ruby -v:
ruby-trunk
[ruby-core:92614]

Description

Hi.

I am writing code that gets rsa private_key using OpenSSL::PKey::RSA#set_key.
As a test, I tried to run following code, and got a crash report.

require 'openssl'

MODULUS = OpenSSL::BN.new('126914039353434453831661971268647447269232081862082764501010934367441434199199964254884893447916776634375786528636229937728173623541291144426274921409848997181513107190580453415730826852070626720125773687471242\
611642649234390348699947633571205684722799950579951120477619298143923772148965919919195784168283711', 10)
PUBLIC_EXPONENT = OpenSSL::BN.new('65537', 10)
PRIVATE_EXPONENT = OpenSSL::BN.new('341964495821065129936072986248372022243660770187105326365541869938588248782459643985676392231199635777382326886137241414828657902188760530546426203854726301121665061632837569847323878241274517300277489\
6102686920500059152100016165854694372963975060765003171003826784408362498480661236694500218201182323054913', 10)
PRIME1 = OpenSSL::BN.new('11952373024606947105152469897150254148042322654516052874548960228374163164391052864033557517269946782417764389875359650595699633451962690417812447456789781', 10)
PRIME2 = OpenSSL::BN.new('10618313124276675806272072098863521356129998721878748974728637357066521302704987846522920724710466419737573058767973827707394086143442677100153976677110531', 10)

rsa = OpenSSL::PKey::RSA.new
rsa.set_key(MODULUS, PUBLIC_EXPONENT, nil)# PRIVATE_EXPONENT)
rsa.set_factors(PRIME1, PRIME2)
puts rsa.private_encrypt('plaintext') if rsa.private?

I expected that

  1. rsa.private_encrypt should not crash, raise RSAError.
  2. if rsa.set_key had called without d argument, rsa.private? should return false.

This is the execute environment.

$ ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin18]
$ ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION'
OpenSSL 1.1.1b  26 Feb 2019
$ gem list openssl

*** LOCAL GEMS ***

openssl (default: 2.1.2)

with 2.7.0-dev too

$ ruby --version
ruby 2.7.0dev (2019-05-09 trunk 025206d0dd) [x86_64-darwin18]
$ ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION'
OpenSSL 1.1.1b  26 Feb 2019
$ gem list openssl

*** LOCAL GEMS ***

openssl (default: 2.1.2)

Thanks.


Files

ruby_2019-05-09-191920_MacBookPro.crash (39.1 KB) ruby_2019-05-09-191920_MacBookPro.crash using 2.6.3p62 thekuwayama (tomoya kuwayama), 05/09/2019 10:20 AM
ruby_2019-05-09-192040_MacBookPro.crash (39.1 KB) ruby_2019-05-09-192040_MacBookPro.crash using 2.7.0dev thekuwayama (tomoya kuwayama), 05/09/2019 10:20 AM

Updated by MSP-Greg (Greg L) almost 5 years ago

Interesting. I've meant to create a repo that allowed one to write a test, and run it against MinGW, Linux, & OSX builds, using current Ruby builds (2.4 thru trunk).

All darwin18 builds passed, Linux Xenial builds failed, and MinGW builds passed. Some use OpenSSL 1.1.1, others 1.0.2.

The test is here:
https://github.com/MSP-Greg/ruby-test/blob/15841/test/test_15841.rb

Travis:
https://travis-ci.org/MSP-Greg/ruby-test/builds/531454106

Appveyor:
https://ci.appveyor.com/project/MSP-Greg/ruby-test

Re this issue, when and where SEGV's are considered 'improper' is not something I'll weigh in on...

Updated by jeremyevans0 (Jeremy Evans) almost 5 years ago

  • Assignee set to rhenium (Kazuki Yamaguchi)

I can confirm that the bug also happens on OpenBSD 6.5 with LibreSSL 2.9.1.

I created a pull request with a fix: https://github.com/ruby/openssl/pull/255

Updated by thekuwayama (tomoya kuwayama) almost 5 years ago

I creaated a PR, https://github.com/ruby/openssl/pull/258 , about remaining issues.

  1. if rsa.set_key had called without d argument, rsa.private? should return false.

Updated by thekuwayama (tomoya kuwayama) over 4 years ago

Both of PR is merged and this issue is resolved, so this can be closed.

Thanks!

Actions #5

Updated by jeremyevans0 (Jeremy Evans) over 4 years ago

  • Status changed from Open to Closed
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0