Bug #7458

Depending on value of BigDecimal::limit, addition can give inaccurate results

Added by Bradford Powell over 1 year ago. Updated 4 months ago.

[ruby-core:50269]
Status:Closed
Priority:Normal
Assignee:Kenta Murata
Category:ext
Target version:next minor
ruby -v:2.0.0dev Backport:

Description

=begin
This bug is present in ruby-trunk and also in 1.8.7 and 1.9.3p125.

I apologize that I cannot figure out what exactly is going wrong and how to patch, but at least I have a small reproducible test case.

When adding very small numbers (like 7E-19) to 1, the result should be very close to 1. However, depending on the limit to BigDecimal's precision, this is not always what is resulted, as this script demonstrates:

require 'bigdecimal'

one = BigDecimal('1')
smallnum = BigDecimal('7E-19')
0.upto(20) do |lim|
BigDecimal::limit(lim)
result = one + smallnum
puts "result is #{result}, as float is #{result.to_f} with limit #{lim}"
end

When run in trunk (or 1.8.7), the results it gives are:

result is 0.10000000000000000007E1, as float is 1.0 with limit 0
result is 0.1E1, as float is 1.0 with limit 1
result is 0.1E1, as float is 1.0 with limit 2
result is 0.1E1, as float is 1.0 with limit 3
result is 0.1E1, as float is 1.0 with limit 4
result is 0.1E1, as float is 1.0 with limit 5
result is 0.1E1, as float is 1.0 with limit 6
result is 0.1E1, as float is 1.0 with limit 7
result is 0.1E1, as float is 1.0 with limit 8
result is 0.1E1, as float is 1.0 with limit 9
result is 0.2E1, as float is 2.0 with limit 10
result is 0.2E1, as float is 2.0 with limit 11
result is 0.2E1, as float is 2.0 with limit 12
result is 0.2E1, as float is 2.0 with limit 13
result is 0.2E1, as float is 2.0 with limit 14
result is 0.2E1, as float is 2.0 with limit 15
result is 0.2E1, as float is 2.0 with limit 16
result is 0.2E1, as float is 2.0 with limit 17
result is 0.2E1, as float is 2.0 with limit 18
result is 0.1000000000000000001E1, as float is 1.0 with limit 19
result is 0.10000000000000000007E1, as float is 1.0 with limit 20

So the results with limit <=9 or >=19 are as expected, but in between the result returned is equal to 2.0

A test that demonstrates this (suitable for inclusion in (({test/bigdecimal/test_bigdecimal.rb})) is:

def testlimitedoneplussmall
one = BigDecimal('1')
smallnum = BigDecimal('0.7E-18')
BigDecimal::savelimit do
1.upto(20) do |lim|
BigDecimal::limit(lim)
result = (one + smallnum).to
f.tos
assert
equal("1.0", result, "Error adding a small number to one with BigDecimal::limit(#{lim})")
end
end
end

I think this is different from #7404 (from what I can tell by google translate), but I apologize for the duplicate if not.
=end

Associated revisions

Revision 44155
Added by Kenta Murata 4 months ago

Append [Bug #7458] in the previous Changelog entry

History

#1 Updated by Usaku NAKAMURA over 1 year ago

  • Category set to ext
  • Status changed from Open to Assigned
  • Assignee set to Kenta Murata
  • Target version set to 2.0.0

#2 Updated by Yusuke Endoh about 1 year ago

  • Target version changed from 2.0.0 to next minor

#3 Updated by Bradford Powell 6 months ago

=begin
Still present in
(({ruby 2.1.0dev (2013-09-22 trunk 43011) [x86_64-linux]}))
=end

#4 Updated by Kenta Murata 4 months ago

  • Status changed from Assigned to Closed

I've fix this issue in r44153.

Also available in: Atom PDF