Project

General

Profile

Bug #15552

app_aobench.rb is broken

Added by Eregon (Benoit Daloze) 26 days ago. Updated 26 days ago.

Status:
Closed
Priority:
Normal
Target version:
-
ruby -v:
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
[ruby-core:91194]

Description

Specifically, the image generated is incorrect.
When modifying the footer of the benchmark like this:

alias printf_orig printf
def printf *args
  $fp.printf(*args)
end

File.open("ao.ppm", "w") do |fp|
  $fp = fp
  printf("P6\n")
  printf("%d %d\n", IMAGE_WIDTH, IMAGE_HEIGHT)
  printf("255\n")
  Scene.new.render(IMAGE_WIDTH, IMAGE_HEIGHT, NSUBSAMPLES)
end

undef printf
alias printf printf_orig

Here is the expected image:
ao_ref
I get this image with MRI 2.6.0:
ao_mri_2_6_0
And interestingly, TruffleRuby 1.0.0-rc11 renders an image closer to the expected one:
ao_tr_rc11

I guess this might be both an interpreter bug, and possibly also a bug in the benchmark code.

I think every benchmark should have some validation, otherwise it's prone to measure something unexpected like this.

Associated revisions

Revision be3a0c70
Added by mame (Yusuke Endoh) 26 days ago

benchmark/app_aobench.rb: add a magic comment

To support the change of default encoding.
It had not worked correctly since 2.0 :-)
[Bug #15552]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 66892
Added by mame (Yusuke Endoh) 26 days ago

benchmark/app_aobench.rb: add a magic comment

To support the change of default encoding.
It had not worked correctly since 2.0 :-)
[Bug #15552]

Revision 8a294e0f
Added by mame (Yusuke Endoh) 26 days ago

benchmark/app_aobench.rb: add srand(0)

To prevent noise for benchmark result. Just for the case.
[Bug #15552]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66893 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 66893
Added by mame (Yusuke Endoh) 26 days ago

benchmark/app_aobench.rb: add srand(0)

To prevent noise for benchmark result. Just for the case.
[Bug #15552]

History

#1

Updated by Eregon (Benoit Daloze) 26 days ago

  • Description updated (diff)

Updated by mame (Yusuke Endoh) 26 days ago

  • Assignee set to mame (Yusuke Endoh)
  • Status changed from Open to Assigned

And interestingly, TruffleRuby 1.0.0-rc11 renders an image closer to the expected one:

Closer :-)

This is caused by change of default encoding. The following fixes the issue. It is fortunate that the issue was not caused by wrong optimization.

diff --git a/benchmark/app_aobench.rb b/benchmark/app_aobench.rb
index 2bd6acfaf8..13cbc1657d 100644
--- a/benchmark/app_aobench.rb
+++ b/benchmark/app_aobench.rb
@@ -1,3 +1,5 @@
+# coding: US-ASCII
+
 # AO render benchmark
 # Original program (C) Syoyo Fujita in Javascript (and other languages)
 #      https://code.google.com/p/aobench/

By the way, the program uses a random number. The effect would be negligible, but I'd like to add srand(0) after the above fix.

#3

Updated by mame (Yusuke Endoh) 26 days ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r66892.


benchmark/app_aobench.rb: add a magic comment

To support the change of default encoding.
It had not worked correctly since 2.0 :-)
[Bug #15552]

Updated by Eregon (Benoit Daloze) 26 days ago

mame (Yusuke Endoh) Thanks for the quick fix!
With srand(0), I think we could add some verification, e.g., by computing a checksum of the image.

Also available in: Atom PDF