Feature #16468: Switch to Miller-Rabin for Prime.prime?

• Status: Closed
• Assignee: -
• Target version: -

[steveb3210 (Stephen Blackstone)]

The miller-rabin algorithm is a non-deterministic primality test, however it is known that below 2**64, you can always get a deterministic answer by only checking a=[2,3,5,7,11,13,17,19,23, 29, 31, 37]

Given that Prime.prime? would never respond in a reasonable amount of time for larger numbers, we can gain much more utility and performance by switching..

                                               user     system      total        real
miller_rabin: random set                   0.150000   0.000000   0.150000 (  0.152212)
Prime.prime?: random set                   0.270000   0.000000   0.270000 (  0.281257)

                                               user     system      total        real
miller_rabin: 16 digits                    0.010000   0.000000   0.010000 (  0.000300)
Prime.prime? 16 digits                     2.200000   0.020000   2.220000 (  2.368247)

                                               user     system      total        real
miller_rabin: 2-10000                      0.030000   0.000000   0.030000 (  0.035752)
Prime.prime? 2-10000                       0.020000   0.000000   0.020000 (  0.022948)

[steveb3210 (Stephen Blackstone)]

• Description updated (diff)

[steveb3210 (Stephen Blackstone)]

• File patch.diff added

Attached is an implementation against master....

[marcandre (Marc-Andre Lafortune)]

Interesting. We might as well always return the correct result, i.e. apply the fast algorithm for integers < 318,665,857,834,031,151,167,461 and the slow algorithm for larger ones. Would you care to modify your patch?

[steveb3210 (Stephen Blackstone)]

marcandre (Marc-Andre Lafortune) wrote:

Interesting. We might as well always return the correct result, i.e. apply the fast algorithm for integers < 318,665,857,834,031,151,167,461 and the slow algorithm for larger ones. Would you care to modify your patch?

If you were to use the existing algorithm on 318,665,857,834,031,151,167,461 you'd need Math.sqrt(318665857834031151167461 ) / 30.0 = 18_816_832_235 loop iterations.. Its not worth falling back given how inefficient the existing function is at that magnitude.

[mame (Yusuke Endoh)]

It can fall back to APR-CL primality test when Miller-Rabin does not work. In my personal opinion, it would be best to keep prime.rb simple because it is a hobby library.

You may be interested in my gem: https://github.com/mame/faster_prime

[steveb3210 (Stephen Blackstone)]

On second thought, I think Marc is right, we can't ruin someones day with a composite without a warning that theres a non-zero probability of it being incorrect so I will update......

[steveb3210 (Stephen Blackstone)]

• File patch_with_argument_error.diff added

[steveb3210 (Stephen Blackstone)]

• File deleted (patch.diff)

[steveb3210 (Stephen Blackstone)]

• File patch_with_argument_error.diff added

[steveb3210 (Stephen Blackstone)]

• File deleted (patch_with_argument_error.diff)

[steveb3210 (Stephen Blackstone)]

• File patch_with_argument_error_and_tests.diff added

[steveb3210 (Stephen Blackstone)]

• File deleted (patch_with_argument_error.diff)

[steveb3210 (Stephen Blackstone)]

• File deleted (patch_with_argument_error_and_tests.diff)

[steveb3210 (Stephen Blackstone)]

• File patch_with_argument_error_and_tests.diff added

• Add bounds check
• Add test

[Dan0042 (Daniel DeLorme)]

I think it would be interesting to expose the algorithm for larger numbers. So you could have miller_rabin which allows any integer, and prime? which checks the value and either call miller_rabin or raise ArgumentError.

[steveb3210 (Stephen Blackstone)]

Dan0042 (Daniel DeLorme) wrote in #note-15:

I think it would be interesting to expose the algorithm for larger numbers. So you could have miller_rabin which allows any integer, and prime? which checks the value and either call miller_rabin or raise ArgumentError.

Unforunately Miller-Rabin is not a deterministic test for arbitrarily large n - its only the work in the paper https://arxiv.org/pdf/1509.00864.pdf that allows us to provide functionality up to a bound.

[steveb3210 (Stephen Blackstone)]

• File deleted (patch_with_argument_error_and_tests.diff)

[steveb3210 (Stephen Blackstone)]

• File prime_patch.diff prime_patch.diff added

Attached is the latest diff.

[Dan0042 (Daniel DeLorme)]

steveb3210 (Stephen Blackstone) wrote in #note-16:

Unforunately Miller-Rabin is not a deterministic test for arbitrarily large n - its only the work in the paper https://arxiv.org/pdf/1509.00864.pdf that allows us to provide functionality up to a bound.

Yes, I know, that was my point. Maybe I want to know if arbitrarily large N is prime while accepting a probability of error of 4**(-k); then I could use miller_rabin(k). And the prime? method uses a bounds check and miller_rabin(13) to produce a deterministic result.

BTW your implementation appears to be off by one. It should be: if self >= 3317044064679887385961981 then raise ArgumentError. You could also slightly reduce the runtime for small numbers by using return list.include?(self) if self <= list.last before the loop, instead of checking inside the loop.

• Status changed from Open to Closed

Applied in changeset git|1866d483dce614a02c5186bd0588b48a5041e55e.

---

[ruby/prime] Optimize Integer#prime?

Miller Rabin algorithm can be used to test primality for integers smaller than a max value "MaxMR" (~3e24)

It can be much faster than previous implementation: ~100x faster for numbers with 13 digits, at least 5 orders of magnitude for even larger numbers (previous implementation is so slow that it requires more patience than I have for more precise estimate).

Miller Rabin test becomes faster than previous implementation at somewhere in the range 1e5-1e6. It seems that the range 62000..66000 is where Miller Rabin starts being always faster, so I picked 0xffff arbitrarily; before that, or above "MaxMR", the previous implementation remains.

I compared the faster_prime gem too. It is slower than previous implementation up to ~1e4. After that it becomes faster and faster compared to previous implementation, but is still slower than Miller Rabin starting at ~1e5 and up to MaxMR. Thus, after this commit, builtin Integer#prime? will be similar or faster than faster_prime up to "MaxMR".

Adapted from patch of Stephen Blackstone [Feature #16468]

Benchmark results and code: https://gist.github.com/marcandre/b263bdae488e76dabdda84daf73733b9

Co-authored-by: Stephen Blackstone [e-mail]