Float#round(n) returns a wrong result when n is big
First of all, don't confuse that this is a usual floating-point error issue.
The following looks inconsistent:
3.0e-31 #=> 3.0e-31 3.0e-31.round(31) #=> 3.0000000000000003e-31
What it should be¶
A Float value is actually a range.
3.0e-31 represents a range of
0.300000000000000003105637e-30 (the bounds are approximate). I call this range "A".
3.0000000000000003e-31 represents a range of
0.300000000000000046896214e-30. I call this range "B".
x.round(31) should (1) multiple x with
10**31, (2) round it as an integer, and (3) divide it with
In this case:
3.0e-31 * 10**31 is a range of
(2) The rounded result is 3, whichever value is chosen from the range above.
3.0 / 10**31 is within the range "A", not within the range "B", so the result should be
How the bug occurs¶
The reason why
3.0000000000000003e-31, is the implementation issue of
Float#round. It does the following:
f = pow(10, b)
x = round(x * f) as an integer
x / f
However, a double variable
f cannot represent
pow(10, 31) precisely. In other words, the
10**31 must be handled as an integer, but the implementation handles it as an inexact floating-point value. This is the issue.
How to fix¶
The issue is simple, but it might be very difficult to fix.
strtod handles a string
"3.0e-31" correctly. So, by doing the same as
strtod, this issue would be fixed. However, the strtod implementation looks very difficult, at least to me. Contribution from mathematician is welcome.
(Honestly, I don't want to see such a complication in the source code. Another simpler approach would be more preferable.)
This issue has been already reported in #5273 by marcandre. But the status of the ticket looks unclear; I cannot see how many issues remains. So, I created this ticket for just one bug that I could confirm.