Bug #15760: OpenSSL::BN.new(nil, 2) dumps core

• Status: Closed
• Assignee: rhenium (Kazuki Yamaguchi)
• Target version: -
• ruby -v: ruby 2.7.0dev (2019-04-10 trunk 67496) [x86_64-linux]
• Backport: 2.4: REQUIRED, 2.5: REQUIRED, 2.6: DONE

[mame (Yusuke Endoh)]

ext/openssl/ossl_bn.c

 221         if (!BN_bin2bn((unsigned char *)StringValuePtr(str), RSTRING_LENINT(str), bn)) {
 222             ossl_raise(eBNError, NULL);
 223         }
 224         break;

This code depends on the evaluation order of function arguments. It works if StringValuePtr(str) is evaluated earlier than RSTRING_LENINT(str), but it doesn't if the order is reversed, which causes core dump.

[mame (Yusuke Endoh)]

• Status changed from Open to Closed

Applied in changeset trunk|r67497.

---

ext/openssl/ossl_bn.c (ossl_bn_initialize): get rid of SEGV

OpenSSL::BN.new(nil, 2) dumped core.

[ruby-core:92231] [Bug #15760]

[naruse (Yui NARUSE)]

• Backport changed from 2.4: REQUIRED, 2.5: REQUIRED, 2.6: REQUIRED to 2.4: REQUIRED, 2.5: REQUIRED, 2.6: DONE

ruby_2_6 r67524 merged revision(s) 67497,67498.