Bug #7037

float formatting inconsistently rounds half to even

Added by Charles Nutter over 1 year ago. Updated about 1 year ago.

[ruby-core:47582]
Status:Closed
Priority:Normal
Assignee:Yukihiro Matsumoto
Category:core
Target version:-
ruby -v:2.0.0dev Backport:

Description

MRI does not appear to consistently round half to even. I'm not sure what rounding strategy this is, but it rounds xx05 and xx15 to odd for xxx, and other values to even:

irb(main):001:0> "%1.1f" % 1.05
=> "1.1"
irb(main):002:0> "%1.1f" % 1.15
=> "1.1"
irb(main):003:0> "%1.1f" % 1.25
=> "1.2"
irb(main):004:0> "%1.1f" % 1.35
=> "1.4"
irb(main):005:0> "%1.1f" % 1.45
=> "1.4"
irb(main):006:0> "%1.1f" % 1.55
=> "1.6"

None of the tie-breaking strategies I could find (http://en.wikipedia.org/wiki/Rounding#Tie-breaking) seem to support MRI's model.

If MRI is indeed using "half even", xx05 should round to xx0 and xx15 should round to xx2. An example with Java's BigDecimal appears to support this:

irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALFEVEN)).tos
=> "1.0"
irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALFEVEN)).tos
=> "1.2"
irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALFEVEN)).tos
=> "1.2"
irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALFEVEN)).tos
=> "1.4"

We would like clarification about the proper rounding tie-breaker strategy to use so we can fix this JRuby issue properly: http://jira.codehaus.org/browse/JRUBY-6889

History

#1 Updated by Nobuyoshi Nakada over 1 year ago

  • Status changed from Open to Closed

=begin
Just formats the value with full precision and "rounds half up" the next char.

(gdb) printf "%.17f\n", 1.05
1.05000000000000004
(gdb) printf "%.17f\n", 1.15
1.14999999999999991
(gdb) printf "%.17f\n", 1.25
1.25000000000000000
(gdb) printf "%.17f\n", 1.35
1.35000000000000009
(gdb) printf "%.17f\n", 1.45
1.44999999999999996
(gdb) printf "%.17f\n", 1.55
1.55000000000000004
(gdb) printf "%.17f\n", 1.65
1.64999999999999991
(gdb) printf "%.17f\n", 1.75
1.75000000000000000
(gdb) printf "%.17f\n", 1.85
1.85000000000000009
(gdb) printf "%.17f\n", 1.95
1.94999999999999996
=end

#2 Updated by Shyouhei Urabe over 1 year ago

I can't under stand what @nobu says so I did this myself.

@headius mixed two points.

  • 1.15 for instance is not exactly representable in floating point number. So tie-breaking is not the case.
  • 1.25 for instance is exact. This, and only this one in above example, is the tie-breaking, and is rounded to 1.2. Rounding 1.25 to 1.2 is not inconsistent regarding "half to even".

am I correct?

#3 Updated by Nobuyoshi Nakada over 1 year ago

Sorry, it's my bad.
Indeed, "half even" not "half up".

#4 Updated by Charles Nutter over 1 year ago

Ok, I can buy the precision argument, and it fits if I expand my example to all values of 1.x5:

irb(main):002:0> "%1.1f" % 1.05
=> "1.1"
irb(main):003:0> "%1.1f" % 1.15
=> "1.1"
irb(main):004:0> "%1.1f" % 1.25
=> "1.2"
irb(main):005:0> "%1.1f" % 1.35
=> "1.4"
irb(main):006:0> "%1.1f" % 1.45
=> "1.4"
irb(main):007:0> "%1.1f" % 1.55
=> "1.6"
irb(main):008:0> "%1.1f" % 1.65
=> "1.6"
irb(main):009:0> "%1.1f" % 1.75
=> "1.8"
irb(main):010:0> "%1.1f" % 1.85
=> "1.9"
irb(main):011:0> "%1.1f" % 1.95
=> "1.9"

My next question is if this is actually desirable or not. In JRuby and Java/JDK, it appears float formatting rounds half up, and treats imprecise halves as precise.

JRuby:

irb(main):010:0> "%1.1f" % 1.05
=> "1.1"
irb(main):011:0> "%1.1f" % 1.15
=> "1.2"
irb(main):012:0> "%1.1f" % 1.25
=> "1.3"
irb(main):013:0> "%1.1f" % 1.35
=> "1.4"
irb(main):014:0> "%1.1f" % 1.45
=> "1.5"
irb(main):015:0> "%1.1f" % 1.55
=> "1.6"
irb(main):016:0> "%1.1f" % 1.65
=> "1.7"
irb(main):017:0> "%1.1f" % 1.75
=> "1.8"
irb(main):018:0> "%1.1f" % 1.85
=> "1.9"
irb(main):019:0> "%1.1f" % 1.95
=> "2.0"

Even given arguments that we should round half to even, we'd likely be consistent with human expectations here.

Here's Java's String.format in action:

public class FormatFloat {
public static void main(String[] args) {
System.out.println(String.format("%1.1f",1.05));
System.out.println(String.format("%1.1f",1.15));
System.out.println(String.format("%1.1f",1.25));
System.out.println(String.format("%1.1f",1.35));
System.out.println(String.format("%1.1f",1.45));
System.out.println(String.format("%1.1f",1.55));
System.out.println(String.format("%1.1f",1.65));
System.out.println(String.format("%1.1f",1.75));
System.out.println(String.format("%1.1f",1.85));
System.out.println(String.format("%1.1f",1.95));
}
}

Output:

system ~/projects/jruby $ java FormatFloat
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0

These are Java doubles. JRuby uses the same representation internally, so ignoring the half-up-versus-half-even issue, this may be a platform-specific float representation question.

The JVM does not enforce strict IEEE 754 floating point math since Java 1.2 for performance reasons; instead, it may (not required) use the most efficient and accurate representation of floating point values on the given platform. For x86_64, that is not a 64-bit value but is instead 80-bit extended precision. This means (correct me if I'm wrong) that the JVM can accurately represent 1.05 and friends exactly as the equivalent of 105x10-2 since it can fully and accurately represent a 64-bit signed integer and with an extra 16 bits for exponentiation.

In this light, ignoring the rounding strategy for the moment, the JRuby and JVM results make sense. Halves can all be represented accurately up to 64 bit signed integer values + exponent.

So this leaves me with two questions about how to handle this issue in JRuby:

  • Is strict IEEE 754 floating-point precision a specified behavior of Ruby?

If yes, then all builds of MRI on all platforms must exhibit the same behavior regardless of platform representation or compiler optimizations. If no, then JRuby's use of JVM floating-point representation and behavior are ok. We may be able to force IEEE 754 floating point behavior in JRuby, but my quick tests seemed to show it's not as simple as just turning on strictfp. I'd prefer to not have to go that direction.

  • Is rounding half even a specified behavior of Ruby?

If yes, I question whether it's a significant distinction to make, since only two "half" values can ever be represented accurately in IEEE 754 anyway: xx25 and xx75. If no, then JRuby's current behavior is acceptable as a platform-specific or implementation-specific detail. In JRuby's case, where we're relying on JVM floating-point representation, we are always using the "round half away from zero" strategy, which has bias, but we're consistent for all input values.

#5 Updated by Yui NARUSE over 1 year ago

headius (Charles Nutter) wrote:

  • Is strict IEEE 754 floating-point precision a specified behavior of Ruby?

ISO Ruby says floating point should follow ISO/IEC/IEEE 60559 (IEEE 754).

Anyway, did you see https://github.com/jruby/jruby/blob/master/src/org/jruby/util/Sprintf.java#L1434 ?
It looks like almost broken.

#6 Updated by Charles Nutter over 1 year ago

If we want to go by the letter of the spec (and I may have an older copy...please confirm), it says "if the underlying platform of a conforming processor supports IEC 60559:1989" the representation shall be that representation. I need to check the document for definitions of "platform" and "processor", but an argument can be made that our platform is the JVM...in which case, the JVM's representation of floating-point would be on with the spec.

The spec also says rounding due to arithmentic operations is implementation-defined.

The spec also says nothing about String#% or Kernel#sprintf.

#7 Updated by Charles Nutter over 1 year ago

The only definition I could find is a sideways use of "processor" to mean "Ruby processor", as in a Ruby language processor, i.e. a Ruby implementation. So... "the underlying platform of a conforming processor" would be the JVM, so it's probably ok for us to dodge the IEEE 754 issue.

Given the wording of the arithmetic rounding text in Float's description and the fact that formatting-related methods are not in the spec, we can probably dodge that issue too.

Of course, that's only if the ISO spec is all we care about :)

At this point I don't think I'm going to try to explore forcing strict IEEE 754, since even on the JVM it is intended as a workaround for specific, localized cases. So that leaves the rounding issue.

Given that there's no help from the spec on whether we should round half to even or half away from zero...what should we, as gentleman programmers, do?

If we agree to "half to even" then JRuby will round 1.05 to 1.0, 1.15 to 1.2, 1.85 to 1.8, and 1.95 to 2.0, which doesn't match MRI.

If we agree to "half away from zero" then MRI's rounding results of 1.25 to 1.2 would not match (the only other tiebreaking case in MRI's current system is 1.75, which already rounds to 1.8 as it would with "half away from zero").

And yes, I've seen that rounding code. I wish I could unsee it.

#8 Updated by Charles Nutter over 1 year ago

Oh, and if we agree that rounding for float formatting is implementation-defined (like rounding for arithmetic, according to the spec), then JRuby and MRI differ on the rounding results of 1.15, 1.25, 1.45, 1.65, and 1.95 would fail to match MRI solely due to precision.

#9 Updated by Shyouhei Urabe over 1 year ago

It seems Ruby is just following C here.

zsh % cat tmp.c
#include
#include
int main(void)
{
printf("%1.1f\n", 1.05);
printf("%1.1f\n", 1.15);
printf("%1.1f\n", 1.25);
printf("%1.1f\n", 1.35);
printf("%1.1f\n", 1.45);
printf("%1.1f\n", 1.55);
printf("%1.1f\n", 1.65);
printf("%1.1f\n", 1.75);
printf("%1.1f\n", 1.85);
printf("%1.1f\n", 1.95);
return EXIT_SUCCESS;
}
zsh % gcc tmp.c
zsh % ./a.out
1.1
1.1
1.2
1.4
1.4
1.6
1.6
1.8
1.9
1.9

so I think, no this is not a part of our spec. This is just machine-dependent. Strictly specifying this behaviour would not make both C/JRuby people happy.

#10 Updated by Shyouhei Urabe over 1 year ago

  • Status changed from Closed to Assigned
  • Assignee set to Yukihiro Matsumoto

Anyway I'm assigning this to matz, as it turned out to be a spec issue. How do you feel matz?

#11 Updated by Charles Nutter over 1 year ago

I would agree with leaving this behavior unspecified. Our behavior also matches underlying platform.

#12 Updated by Charles Nutter over 1 year ago

If there's nothing further to do here and we all agree that the details of rounding logic are implementation-dependent, this can be closed.

#13 Updated by Motohiro KOSAKI about 1 year ago

  • Status changed from Assigned to Closed

7037: float formatting inconsistently rounds half to even
* https://bugs.ruby-lang.org/issues/7037
* I believe we agree that float half-rounding behavior is
impl-specific, so this could be closed.

Also available in: Atom PDF