Bug #5273: Float#round returns the wrong floats for higher precision

• Status: Closed
• Assignee: marcandre (Marc-Andre Lafortune)
• Target version: -
• ruby -v: r33186
• Backport: 2.3: UNKNOWN, 2.4: UNKNOWN

[marcandre (Marc-Andre Lafortune)]

Float#round returns the wrong floats for higher precision.

Rounding can be off:
2.5e-22.round(22) # => 2.0e-22

It can also return values that have extra decimals:
2.5e-31.round(31) # => 3.0000000000000003e-31
2.5e-36.round(36) # => 2.9999999999999998e-36

Both errors can occur at the same time too:
2.5e-39.round(39) # => 2.0000000000000002e-39

I believe these errors occur only for ndigits >= 22. For ndigits > 22, 10**(-ndigits) is not an exact Float (as 5 ** 23 won't fit in the 53 bits of a double's mantissa). For ndigits < 22, there should be enough bits left to avoid rounding errors.

For ndigits >= 22, the algorithm to use should be similar to the one used to convert doubles to string.

Hopefully, this is the last bug for #round with a given precision.

[naruse (Yui NARUSE)]

This seems not a bug:
printf "%.40f\n", 2.5e-22 #=> 0.0000000000000000000002499999999999999769

[naruse (Yui NARUSE)]

• Status changed from Open to Rejected

Additional to say, 2.5e-22 express the range
from 0.0000000000000000000002499999999999999534
to__ 0.0000000000000000000002500000000000000003

The center of them is 0.0000000000000000000002499999999999999769.
So of course 0.0000000000000000000002_499999999999999769.round(22) will be 2.0e-22.

[naruse (Yui NARUSE)]

One more additional to say, those arguments depend on IEEE 754.

[marcandre (Marc-Andre Lafortune)]

• Status changed from Rejected to Open

Hi,

Yui NARUSE wrote:

Additional to say, 2.5e-22 express the range
from 0.0000000000000000000002499999999999999534
to__ 0.0000000000000000000002500000000000000003

The center of them is 0.0000000000000000000002499999999999999769.
So of course 0.0000000000000000000002_499999999999999769.round(22) will be 2.0e-22.

That's a good point.

This doesn't make the other examples right though (e.g. 3.0e-31.round(31) should be equal to itself, not to 3.0000000000000003e-31)

[naruse (Yui NARUSE)]

• ruby -v changed from r33186 to -

(2011/09/10 10:00), Marc-Andre Lafortune wrote:

Issue #5273 has been updated by Marc-Andre Lafortune.

Status changed from Rejected to Open

Hi,

Yui NARUSE wrote:

Additional to say, 2.5e-22 express the range
from 0.0000000000000000000002499999999999999534
to__ 0.0000000000000000000002500000000000000003

The center of them is 0.0000000000000000000002499999999999999769.
So of course 0.0000000000000000000002_499999999999999769.round(22) will be 2.0e-22.

That's a good point.

This doesn't make the other examples right though (e.g. 3.0e-31.round(31) should be equal to itself, not to 3.0000000000000003e-31)

irb(main):017:0> 3.0/10**31
=> 3.0000000000000003e-31

This is in short, use BigDecimal.

--
NARUSE, Yui [e-mail]

[naruse (Yui NARUSE)]

• Status changed from Open to Rejected

r33186

[marcandre (Marc-Andre Lafortune)]

• Status changed from Rejected to Open
• Target version changed from 1.9.3 to 1.9.4
• ruby -v changed from - to r33186

On Mon, Sep 12, 2011 at 8:30 PM, Yui NARUSE [e-mail] wrote:

Issue #5273 has been updated by Yui NARUSE.

Status changed from Open to Rejected

r33186

I'm not sure, are you trying to be rude or irritating by rejecting for a second time this issue in this way?

If so, please stop.

If not, would you be kind enough to explain the following:

• What makes you believe r33186/issue #5272 is related with this issue #5273?
• Why do I have to repeat [ruby-core:39296] that issues #5272 and #5273 are not related?
• Why do you think I opened two different issues in the first place (and fixed only one of them)?
• Is there anything in the examples given in the two issues, or in the specs I wrote that makes it possible to confuse both issues?
• Why do you feel it is appropriate to reject this issue by simply referring to one of my commits, without any explanation whatsoever?
• Don't you feel that even if the commit addressed the issue, the issue should have been Closed, not Rejected?

In addition, may I ask that you do not reject this issue a third time? I will set it myself to rejected if a compelling reason is given. Note that "Use BigDecimal" is not such a reason. Nor is 3.0/10**31 == 3.0000000000000003e-31. Indeed, why stop there, why not have Float("3.0e-31") == 3.0000000000000003e-31 and invoque the same arguments?

[naruse (Yui NARUSE)]

• Status changed from Open to Assigned
• Assignee set to marcandre (Marc-Andre Lafortune)

Marc-Andre Lafortune wrote:

I'm not sure, are you trying to be rude or irritating by rejecting for a second time this issue in this way?

If so, please stop.

Ah sorry, I intended to close this ticket without comment.
"r33186" is wrong comment.
(though r33186 is not valid ruby -v, you must copy and paste full ruby -v output)

If not, would you be kind enough to explain the following:

• What makes you believe r33186/issue #5272 is related with this issue #5273?
• Why do I have to repeat [ruby-core:39296] that issues #5272 and #5273 are not related?
• Why do you think I opened two different issues in the first place (and fixed only one of them)?
• Is there anything in the examples given in the two issues, or in the specs I wrote that makes it possible to confuse both issues?
• Why do you feel it is appropriate to reject this issue by simply referring to one of my commits, without any explanation whatsoever?
• Don't you feel that even if the commit addressed the issue, the issue should have been Closed, not Rejected?

Yeah, #5272 and #5273 are different.
#5272 is truely a bug of ruby, but #5273 is not.

In addition, may I ask that you do not reject this issue a third time? I will set it myself to rejected if a compelling reason is given. Note that "Use BigDecimal" is not such a reason. Nor is 3.0/10**31 == 3.0000000000000003e-31. Indeed, why stop there, why not have Float("3.0e-31") == 3.0000000000000003e-31 and invoque the same arguments?

I can't your English of this line, I think you are saying you can't understand [ruby-core:39443].
I add little more explanation.

First of all, ISO C doesn't specify the detail of floating point numbers.
So my discussion of this comment follows IEEE 754 non x87 floating point numbers.

IEEE 754 double (binary64) is sign 1bit, exponent 11bit, and fraction 52bit,
and its internal reprensentation is binary (not decimal).
When a number is converted to float, the number's precision is limited to this and
lost information over decimal/binary conversion.

For example, 3.0 / 10**31 is considered as 3.0 * 1.0e-31 (this is not accurate).
This is 3.0 * 0.000000000000000000000000000000100000000000000008333642060759
It become 0.000000000000000000000000000000300000000000000025000926182276.
If you show this by inspect, it is rounded and showed as 3.0000000000000003e-31.
And it is a different number from 3.0e-31 in Float.

If you use BigDecimal, such binary/decimal conversion is not done and you can specify its precision.
So you can get expected result:
irb(main):092:0> BigDecimal("2.5e-31").round(31).to_s
=> "0.3E-30"

If you are still hard to understand my explaration, I doubt you don't understand floating point numbers.
If so, please read following documents.
http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html
http://wiki.github.com/rdp/ruby_tutorials_core/ruby-talk-faq#floats_imprecise
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

I assign this ticket to you, please close this after you understand.

[marcandre (Marc-Andre Lafortune)]

Yui NARUSE wrote:

I can't your English of this line, I think you are saying you can't understand [ruby-core:39443].

No, I understood what you are saying. I meant the tour argument is not valid justification for the current behavior.

You say that 3.0e-31.round(31) should be 3.0/10**31 == 3.0000000000000003e-31.

I say it should be 3.0e-31.

If we follow your argument, then it would be ok if Float("3.0e-31") == 3.0000000000000003e-31 but that is clearly not so.

I say that any_float.round(prec) == Float("#{some_int}.0e-#{prec}"), by definition (for positive prec).

Do you have any reason why you think that 3.0000000000000003e-31 is preferable to 3.0e-31? Do you truly think it is the expected result and the most useful result?

Note that in addition to commonsense and mathematics, javascript also agrees with me:

3.0e-31.toPrecision(1) == 3.0000000000000003e-31.toPrecision(1) == "3e-31"

[naruse (Yui NARUSE)]

• Target version changed from 1.9.4 to 2.6

[marcandre (Marc-Andre Lafortune)]

• Assignee changed from marcandre (Marc-Andre Lafortune) to naruse (Yui NARUSE)

[naruse (Yui NARUSE)]

• Status changed from Assigned to Open
• Assignee deleted (naruse (Yui NARUSE))

[naruse (Yui NARUSE)]

• Target version deleted (2.6)

[mame (Yusuke Endoh)]

• Related to Bug #14635: Float#round(n) returns a wrong result when n is big added

[mame (Yusuke Endoh)]

• Status changed from Open to Closed
• Assignee set to marcandre (Marc-Andre Lafortune)

I think there are two issues in this one ticket: 3.0e-31.round(31) #=> 3.0000000000000003e-31 and 2.5e-22.round(22) #=> 2.0e-22.

I created another ticket (#14635) about the former issue with some investigation.

I think that we cannot fix the latter issue for the reason that Naruse-san said. 2.5e-22 represents the range of 0.249999999999999953374475e-21 and 0.250000000000000000394250e-21, so the implementation cannot determine which 0.2e-38 and 0.3e-38 are preferable.

So unfortunately, 2.5e-39.round(39) would be 2.0e-39. Anyway, 2.0000000000000002e-39 is a wrong result, I think.

I'm closing this ticket because there are no other issue in this ticket. Marc-Andre, let me know if I am wrong.

[marcandre (Marc-Andre Lafortune)]

mame (Yusuke Endoh) wrote:

Marc-Andre, let me know if I am wrong.

I agree with everything you wrote :-)