Actions
Bug #9691
closedWrong value on float
Description
Getting strange value on multiplying a float
149.95 * 100
=> 14994.999999999998
Updated by shyouhei (Shyouhei Urabe) about 10 years ago
- Status changed from Open to Rejected
This is how it works. Learn floating point numbers.
"What Every Computer Scientist Should Know About Floating-Point Arithmetic"
http://docs.sun.com/source/806-3568/ncg_goldberg.html
Updated by avit (Andrew Vit) about 10 years ago
Use Fixnum (14995 * 100) or BigDecimal:
require 'bigdecimal'
d = BigDecimal.new('149.95')
d * 100
# => 0.14995E5
More info on floating point vs. decimal numbers:
http://www.youtube.com/watch?v=PZRI1IfStY0
Actions
Like0
Like0Like0