Project

General

Profile

Actions

Bug #22607

open

Rational#to_f and Integer#fdiv are not correctly rounded for large operands

Bug #22607: Rational#to_f and Integer#fdiv are not correctly rounded for large operands

Added by sferik (Erik Berlin) about 3 hours ago. Updated about 3 hours ago.

Status:
Open
Assignee:
-
Target version:
-
ruby -v:
ruby 4.1.0dev (2026-11-19T13:09:31Z master 1e71e987c2) +PRISM [arm64-darwin27]
[ruby-core:127625]

Description

Rational#to_f, and Integer#fdiv with an Integer argument, don't always return the Float nearest to the quotient when the numerator or the denominator is too large to be exact as a double.

r = Rational(742515609648393158335, 10**14)
r.to_f                              #=> 7425156.096483931
742515609648393158335.fdiv(10**14)  #=> 7425156.096483931

The nearest Float is 7425156.096483932:

(r - 7425156.096483932.to_r).abs < (r - 7425156.096483931.to_r).abs  #=> true

I expect to_f to return the nearest Float, the way String#to_f does for the same number.

This isn't limited to contrived numbers. Converting a nanosecond timestamp to seconds is a common case:

Rational(1733983396674757576, 10**9).to_f  #=> 1733983396.6747577, but 1733983396.6747575 is nearer

In random samples of 10,000, the result was one unit in the last place off for about a quarter of nanosecond timestamps between 2001 and 2286 divided by 10**9, and for about a fifth of 25-digit numerators divided by a power of ten. Ruby 3.2 and 3.3 give the same results as master.

The cause is in rb_int_fdiv_double in numeric.c. When either operand isn't exact as a double, it divides both by their GCD, then converts each of them to double and divides. The two conversions and the division each round, so the result can end up on the wrong side of the halfway point. Dividing by the GCD doesn't help when the reduced operands are still too large, as they are in both examples above.

Actions

Also available in: PDF Atom