Project

General

Profile

Bug #13078

with Integer a and b, a.fdiv(b) sometimes inaccurate than Rational(a, b).to_f

Added by metanest (Makoto Kishimoto) almost 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:78855]

Description

For example,

a = 3
b = 0x20_0000_0000_0001
p a.fdiv(b)
p Rational(a, b).to_f

#=>
3.3306690738754696e-16
3.330669073875469e-16

I'm hacking to solve this problem, see diff view.
https://github.com/ruby/ruby/compare/trunk...metanest:fdiv_spike

Also available in: Atom PDF