Float round working weirdly
I encountered this issue when casting a specific float with a especific round argument.
0.5.round(15) => 0.5 0.5.round(16) => 0.5000000000000001 0.5.round(17) => 0.5
Updated by jeremyevans (Jeremy Evans) over 1 year ago
- Status changed from Open to Closed
Applied in changeset git|d16b68cb2204eeb5af8bd39149202b630374c67f.
Use Rational for Float#round with ndigits > 14
ndigits higher than 14 can result in values that are slightly too
large due to floating point limitations. Converting to rational
for the calculation and then back to float fixes these issues.
Co-authored by: Yusuke Endoh email@example.com