Project

General

Profile

Bug #13331

Inconsistent conversion of Float into Decimal

Added by msatkiewicz (Maciej Satkiewicz) over 2 years ago. Updated over 2 years ago.

Status:
Assigned
Priority:
Normal
Target version:
-
ruby -v:
ruby 2.4.0
[ruby-core:80234]

Description

One would reasonably expect the equality 1.to_d * f == f.t_d to hold for every Float value f, i.e. multiplying by DigDecimal's unit can be viewed as an alternative way of casting Float into Decimal; or simply one could expect that BigDecimal#* should always invoke Float#to_d on it's argument. Yet for some Floats (e.g. 64.4) the results differ, as showed in the provided script. This can cause a bug in some application contexts.

Originally I asked about that on Stackoverflow:
http://stackoverflow.com/questions/40472933/inconsistent-conversion-of-float-into-decimal-in-ruby/40473007

See the comprehensive answer by Stefan who points out that it is due to different precisions being used in Float#to_d and BigDecimal#*.


Files

multiplication_by_unit.rb (1.9 KB) multiplication_by_unit.rb msatkiewicz (Maciej Satkiewicz), 03/19/2017 06:51 PM

History

#1

Updated by msatkiewicz (Maciej Satkiewicz) over 2 years ago

  • Description updated (diff)

Updated by mrkn (Kenta Murata) over 2 years ago

  • Assignee set to mrkn (Kenta Murata)
  • Status changed from Open to Assigned

Also available in: Atom PDF