Project

General

Profile

Actions

Bug #13210

closed

Wrong float substraction result

Added by dexter3 (Alex V) about 7 years ago. Updated about 7 years ago.

Status:
Rejected
Assignee:
-
Target version:
-
ruby -v:
2.4.0, 2.3.1
[ruby-core:79516]

Description

0.1 - 0.0001 => 0.0999
0.1 - 0.001  => 0.099
0.1 - 0.01   => 0.09000000000000001

0.01 - 0.0001 => 0.00999
0.01 - 0.001 => 0.0099
0.01 - 0.01 =>  0.009000000000000001

There is something strange going on.
As you can see, the subtraction of A-B with B being ten times smaller as A returns a wrong result. But this is somehow not always the case:

2.4.0 :001 > for i in 1..21 do p "#{0.1*i} - #{0.01*i} = #{0.1*i - 0.01*i}" end
"0.1 - 0.01 = 0.09000000000000001"
"0.2 - 0.02 = 0.18000000000000002"
"0.30000000000000004 - 0.03 = 0.27"
"0.4 - 0.04 = 0.36000000000000004"
"0.5 - 0.05 = 0.45"
"0.6000000000000001 - 0.06 = 0.54"
"0.7000000000000001 - 0.07 = 0.6300000000000001"
"0.8 - 0.08 = 0.7200000000000001"
"0.9 - 0.09 = 0.81"
"1.0 - 0.1 = 0.9"
"1.1 - 0.11 = 0.9900000000000001"
"1.2000000000000002 - 0.12 = 1.08"
"1.3 - 0.13 = 1.17"
"1.4000000000000001 - 0.14 = 1.2600000000000002"
"1.5 - 0.15 = 1.35"
"1.6 - 0.16 = 1.4400000000000002"
"1.7000000000000002 - 0.17 = 1.5300000000000002"
"1.8 - 0.18 = 1.62"
"1.9000000000000001 - 0.19 = 1.7100000000000002"
"2.0 - 0.2 = 1.8"
"2.1 - 0.21 = 1.8900000000000001"
 => 1..21

Even 0.1*x seems sometimes to fail:

2.4.0 :002 > for i in 1..21 do p "0.1*#{i} = #{0.1*i}" end
"0.1*1 = 0.1"
"0.1*2 = 0.2"
"0.1*3 = 0.30000000000000004"
"0.1*4 = 0.4"
"0.1*5 = 0.5"
"0.1*6 = 0.6000000000000001"
"0.1*7 = 0.7000000000000001"
"0.1*8 = 0.8"
"0.1*9 = 0.9"
"0.1*10 = 1.0"
"0.1*11 = 1.1"
"0.1*12 = 1.2000000000000002"
"0.1*13 = 1.3"
"0.1*14 = 1.4000000000000001"
"0.1*15 = 1.5"
"0.1*16 = 1.6"
"0.1*17 = 1.7000000000000002"
"0.1*18 = 1.8"
"0.1*19 = 1.9000000000000001"
"0.1*20 = 2.0"
"0.1*21 = 2.1"
 => 1..21

And to be sure that this can lead to problems:

2.4.0 :003 > 0.1 * 3 == 0.3
 => false

I successfully reproduced the wrong results on another device. Both devices are running on Ubuntu. I also reproduced it with ruby 2.3.1.

The wrong results somehow have a pattern, but I'm not sure about their origin.

I suggest ranking this bug as critical, as ruby should be able to calculate correctly.

BR Alex

Edit: typo

Updated by dexter3 (Alex V) about 7 years ago

  • Description updated (diff)

Updated by duerst (Martin Dürst) about 7 years ago

  • Status changed from Open to Rejected

Floating point calculations have limited precision!

Please read http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html.
It's linked from the top page of this site (https://bugs.ruby-lang.org/).

Actions

Also available in: Atom PDF

Like0
Like0Like0