Project

General

Profile

Bug #9569

SecureRandom should try /dev/urandom first

Added by Corey Csuhta almost 2 years ago. Updated about 1 month ago.

Status:
Rejected
Priority:
Normal
Assignee:
ruby-core
ruby -v:
Backport:
[ruby-core:61094]

Description

Right now, SecureRandom.random_bytes tries to detect an OpenSSL to use before it tries to detect /dev/urandom. I think it should be the other way around. In both cases, you just need random bytes to unpack, so SecureRandom could skip the middleman (and second point of failure) and just talk to /dev/urandom directly if it's available.

Is this a case of just re-ordering the two code chunks so that /dev/urandom is tried first?

Relevant lines: https://github.com/ruby/ruby/blob/trunk/lib/securerandom.rb#L59-L90

History

#1 [ruby-core:61096] Updated by Akira Tanaka almost 2 years ago

  • Status changed from Open to Rejected

/dev/urandom is not suitable to be used to generate directly session keys
and other application level random data which is generated frequently.

random(4) on GNU/Linux:

   The kernel random-number generator  is  designed  to  produce  a  small
   amount  of  high-quality  seed material to seed a cryptographic pseudo-
   random number generator (CPRNG).  It  is  designed  for  security,  not
   speed, and is poorly suited to generating large amounts of random data.
   Users should be very economical in the amount  of  seed  material  that
   they  read  from  /dev/urandom (and /dev/random); unnecessarily reading
   large quantities of data from this device will have a  negative  impact
   on other users of the device.

/dev/urandom should be used as "seed" for CPRNG.
OpenSSL do it.

/dev/urandom usage in securerandom.rb is not a good way.
So OpenSSL should be used at first.

#2 [ruby-core:61110] Updated by Corey Csuhta almost 2 years ago

The random(4) manpage on Linux isn't accurate in this reguard. You can use it as more than just a seed source, and you can use it as frequently as you want.

On modern Linux, both /dev/random and /dev/urandom are CSPRNGs, and can be used safely (after system boot, see references). The only difference is that /dev/random attempts to keep some kind of measure of its available entropy, and will sometimes block if if feels unsatisfied about that. On FreeBSD, Unix, and OS X, there is no difference between /dev/random and /dev/urandom anymore, and the manpages on OS X at least don't include this "rate-limit" warning about /dev/urandom.

Two additional points:

OpenSSL seeds itself from /dev/urandom as you stated, but you could run a lot of OpenSSL processes on your system at one time and none of them would complain that your /dev/urandom is not currently to be trusted because you used it too much.

SecureRandom in Ruby will use /dev/urandom if OpenSSL is not available, based on the code snippet I linked in the original post. This is contrary to your statement that /dev/urandom is not safe for sessions, or frequent access. As currently implemented, SecureRandom will access /dev/urandom frequently if OpenSSL is not available.

References:
http://blog.cr.yp.to/20140205-entropy.html
http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man4/random.4.html
http://security.stackexchange.com/questions/3936/is-a-rand-from-dev-urandom-secure-for-a-login-key

#3 [ruby-core:61111] Updated by Akira Tanaka almost 2 years ago

If you think a manpage is not accurate, please fix the manpage at first.

#4 [ruby-core:61112] Updated by Corey Csuhta almost 2 years ago

Akira, can you address this point?

SecureRandom in Ruby will use /dev/urandom if OpenSSL is not available, based on the code snippet I linked in the original post. This is contrary to your statement that /dev/urandom is not safe for sessions, or frequent access. As currently implemented, SecureRandom will access /dev/urandom frequently if OpenSSL is not available.

#5 [ruby-core:61114] Updated by Akira Tanaka almost 2 years ago

I said "/dev/urandom usage in securerandom.rb is not a good way." already.
It means securerandom.rb will consume too much entoropy if /dev/urandom is used directry.

It is not a big problem because most users use OpenSSL.
Also, we can say "Please install OpenSSL" if someone complain about the entoropy consumption.

Your proposal breaks this strategy.

#6 [ruby-core:72661] Updated by Aaron Zauner about 1 month ago

  • Tracker changed from Feature to Bug
  • Assignee set to ruby-core

Hi,

This still seems to be the case according to the code available on GitHub.

I urge the core team to move to /dev/urandom. It is an urban-legend (as Thomas Ptacek notes in the sockpuppet.org blog-post referenced two years ago) that one should use /dev/random or even fiddle nor interfere with seeding or entropy on Linux. The kernel does this for you. The only instance where you actually might wanna add entropy is during boot-strap or first boot of embedded devices or (cloned) virtual machines. Ask any cryptographer about this and they'll tell you that random(4) is just plainly wrong and nobody cared to update it yet.

The issuer before me has linked to various cryptographers telling you in their blogs to use /dev/urandom. Why would you rather listen to an out-dated man page? Even how this works (CSPRNG and entropy) in the kernel has changed significantly since the man page was last updated.

An OS does not "run out of entropy". This is not how a CSPRNG works [0]. Think of these constructions like you'd with entropy when considering the 2nd law of thermodynamics. :)

Thanks,
Aaron

[0] Here a cryptographer explains it to you: http://crypto.stackexchange.com/a/12441

#7 [ruby-core:72662] Updated by Nobuyoshi Nakada about 1 month ago

SecureRandom without OpenSSL (or compatible alternatives) is nonsense.

#8 [ruby-core:72667] Updated by Akira Tanaka about 1 month ago

Aaron Zauner wrote:

The issuer before me has linked to various cryptographers telling you in their blogs to use /dev/urandom. Why would you rather listen to an out-dated man page? Even how this works (CSPRNG and entropy) in the kernel has changed significantly since the man page was last updated.

Please update the man page first, if it is really out-dated.

#9 [ruby-core:72668] Updated by Aaron Zauner about 1 month ago

Nobuyoshi Nakada wrote:

SecureRandom without OpenSSL (or compatible alternatives) is nonsense.

You evidently have no idea what you are talking about. Why would you want to use the PRNG that's local to OpenSSL? It can fail in many ways, is slower than the kernel and may introduce vulnerabilities. The OpenSSL PRNG is not even fork save!

https://wiki.openssl.org/index.php/Random_fork-safety
https://emboss.github.io/blog/2013/08/21/openssl-prng-is-not-really-fork-safe/

I'm not part of the Linux documentation team, why do you insist on updating the man page before you will fix a critical vulnerability? this is laughable. You are potentially harming many users of SecureRandom in Ruby.

#10 [ruby-core:72670] Updated by Motohiro KOSAKI about 1 month ago

You evidently have no idea what you are talking about. Why would you want to use the PRNG that's local to OpenSSL? It can fail in many ways, is >slower than the kernel and may introduce vulnerabilities. The OpenSSL PRNG is not even fork save!

https://wiki.openssl.org/index.php/Random_fork-safety
https://emboss.github.io/blog/2013/08/21/openssl-prng-is-not-really-fork-safe/

Your pointed issue was mainly investigated Ruby core team and, of course, it has been solved, at least, on Ruby.
Please don't rude. You just said you don't understand the issue.

#11 [ruby-core:72671] Updated by Aaron Zauner about 1 month ago

The following is the output of your 'SecureRandom' construction vs. the Linux /dev/urandom facility:

http://nopaste.narf.at/show/EPVj9ETuMIcrCXKErsS6/
http://nopaste.narf.at/show/i0EJbkQrL3SXurfQZ524/

As you can see your RNG is less secure than the one my Debian installation serves via the kernel.

Sorry, I did not mean to be rude. Please reconsider on this issue, it's security relevant. Also you added an extra dependency: a bug in OpenSSL will now hurt your random number generation. And there is really no reason not to use /dev/urandom on Linux, BSD and Mac OS X these days - take a look at the kernel implementation:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/char/random.c
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/char/hw_random

#12 [ruby-core:72672] Updated by Motohiro KOSAKI about 1 month ago

First off, thank you for providing number. Then we can discuss scientific way.
blog pages are not considered a formal document.

However I can't reproduce your conclusion. In my result, securerandom has one
weak and /dev/urandom has two weaks.

https://gist.github.com/kosaki/3a9a9126cb39e601be2d

Of cource, this result doesn't mean urandom is crap. This just mean dieharder's output is unstable, I think.

#13 [ruby-core:72673] Updated by Filippo Valsorda about 1 month ago

Hi!

I happen to have just given a talk on urandom internals at 32C3: https://media.ccc.de/v/32c3-7441-the_plain_simple_reality_of_entropy

The main takeaway about urandom is that it's a CSPRNG, just like the OpenSSL one. (Just better, because the kernel can protect the entropy pool memory space better, the kernel one is continuously reseeded, and has a better security track record--see Debian.) So there is no security advantage, or actually any theoretical difference in using OpenSSL. In practice, urandom is significantly more secure.

What Ruby currently does is chaining two CSPRNGs, urandom and OpenSSL's. This makes problems twice as likely (not half), because now if any of the two fail, the entire system is compromised. If you just read from urandom, there's only one thing that can go wrong.

Another bad consequence of this scheme is that now there is no reliable failure detection. Take the case of exhausted file descriptors, where opening urandom is impossible, and let's say there's no getrandom support. An exception should be raised, as there's no recovery. But the current code will first ignore the failure of raw_seed, then call into OpenSSL which will hide the failure (which I argue is a OpenSSL API shortcoming, but I digress). If you just read from urandom, it's very clear when the system failed. (By the way, why adding the nanotime and pid at all? Nanotime and pid alone are not enough, and hide brokenness Debian-style, while if you have a real 16 byte of seed, that's enough and you don't need anything else.)

Finally, complexity is toxic. I believe that pre-7104a473, on Windows, when OpenSSL is installed, securerandom was only seeded with the time. (Because OpenSSL would take priority over the syscall, and OpenSSL is documented to seed itself automatically only on platforms that offer urandom. http://linux.die.net/man/3/rand_add I haven't checked the source.) This would be spectacularly bad (and I would actually be dropping a vulnerability right now considering that the patch is only one month old), but I'm not even sure that's the case after staring at code for a good hour. It's significantly bad that a reasonably experienced developer can't be sure of securerandom correctness easily. If you just read from urandom, it takes less than 30 minutes to audit the code.

I realize that we are some Internet strangers arguing against the Linux man page (which for the record was already recently amended to be FUD'y on urandom), but since this is all Open Source, we can thankfully discuss on the actual implementation instead of on what is written about it. If instead authority is what you are looking for, this slide in my 32C3 presentation has plenty, academic and not: https://speakerdeck.com/filosottile/the-plain-simple-reality-of-entropy-at-32c3?slide=36

Finally, I invite you to check how other libraries like BoringSSL, Python's stdlib and Go's stdlib solve this problem. They all read from /dev/urandom.

Also available in: Atom PDF