Project

General

Profile

Actions

Bug #13331

closed

Inconsistent conversion of Float into Decimal

Added by msatkiewicz (Maciej Satkiewicz) about 7 years ago. Updated over 3 years ago.

Status:
Closed
Target version:
-
ruby -v:
ruby 2.4.0
[ruby-core:80234]
Tags:

Description

One would reasonably expect the equality 1.to_d * f == f.t_d to hold for every Float value f, i.e. multiplying by DigDecimal's unit can be viewed as an alternative way of casting Float into Decimal; or simply one could expect that BigDecimal#* should always invoke Float#to_d on it's argument. Yet for some Floats (e.g. 64.4) the results differ, as showed in the provided script. This can cause a bug in some application contexts.

Originally I asked about that on Stackoverflow:
http://stackoverflow.com/questions/40472933/inconsistent-conversion-of-float-into-decimal-in-ruby/40473007

See the comprehensive answer by Stefan who points out that it is due to different precisions being used in Float#to_d and BigDecimal#*.


Files

multiplication_by_unit.rb (1.9 KB) multiplication_by_unit.rb msatkiewicz (Maciej Satkiewicz), 03/19/2017 06:51 PM
Actions #1

Updated by msatkiewicz (Maciej Satkiewicz) about 7 years ago

  • Description updated (diff)

Updated by mrkn (Kenta Murata) almost 7 years ago

  • Status changed from Open to Assigned
  • Assignee set to mrkn (Kenta Murata)
Actions #3

Updated by Anonymous over 3 years ago

  • Status changed from Assigned to Closed
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0