to_f function / multiplication bug
This is posted on stack overflow but I thought I should report it here
Hi I just ran into an issue where ruby's to_f function is giving me inconsistent results.
ruby-1.9.2-head :026 > 8.45.to_f * 100 => 844.9999999999999
ruby-1.9.2-head :027 > 4.45.to_f * 100 => 445.0
ruby-1.9.2-head :028 > 4.35.to_f * 100 => 434.99999999999994
My workaround is to simply round the result this way
ruby-1.9.2-head :029 > (4.35.to_f * 100).round => 435
After more playing around I realised that the issue might be with the multiplication operator "* 100"
Updated by notomodungwa (Noto Modungwa) over 10 years ago
I have been educated that this is more a floating number calculation issue than an issue with ruby itself by guys on stackoverflow see http://stackoverflow.com/questions/7879606/ruby-to-f-bug-or-multiplication-operator-bug
This document Floating-Point Arithmetic's section on "Rounding Error" mentions that you should round the result of a floating point calculation. =>"...Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation..."
Updated by drbrain (Eric Hodel) over 10 years ago
- Status changed from Open to Rejected