Bug #10931: [PERF]bm_app_mandelbrot.rb 20% slower.

• Status: Closed
• Assignee: nobu (Nobuyoshi Nakada)
• Target version: -
• ruby -v: ruby 2.3.0dev
• Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN

[tgxworld (Guo Xiang Tan)]

http://rubybench.org/ruby/ruby/commits?result_type=app_mandelbrot

Relevant Commits:
https://github.com/ruby/ruby/commit/909c4c0
https://github.com/ruby/ruby/commit/41b2f1a

Before: 0.786s
After Commit 909c4c0: 0.856s
After Commit 41b2f1a: 0.923s

Just want to raise awareness on this. Feel free to close this issue if the performance trade-offs were intended. Let me know if more information is required. Thank you!

[nobu (Nobuyoshi Nakada)]

My results.

---

benchmark results:

Execution time (sec)

name	r49722	r49723	r49724	r49732	r49743	r49754	r49757
app_mandelbrot	0.969	0.980	0.957	0.971	0.996	0.976	1.004
so_mandelbrot	1.787	1.743	1.740	1.741	1.744	1.752	1.755

Speedup ratio: compare with the result of `r49722' (greater is better)

name	r49723	r49724	r49732	r49743	r49754	r49757
app_mandelbrot	0.989	1.013	0.998	0.973	0.993	0.966
so_mandelbrot	1.026	1.027	1.027	1.025	1.020	1.018

Log file: bmlog-20150304-170709.40624

[tgxworld (Guo Xiang Tan)]

Hi Nobu,

I ran the benchmarks on my local machine and I'm seeing the same results as those on RubyBench's dedicated server. How are you running your benchmarks? I'll like to try running them too.

total: 2 trial(s) (2 trial(s) for 1 benchmark(s))
..

benchmark results:
Execution time (sec)
name ruby 2.3.0dev (2015-02-25 trunk 49753) [x86_64-linux]
app_mandelbrot 0.957

total: 2 trial(s) (2 trial(s) for 1 benchmark(s))
..

benchmark results:
Execution time (sec)
name ruby 2.3.0dev (2015-02-25 trunk 49754) [x86_64-linux]
app_mandelbrot 1.037

total: 2 trial(s) (2 trial(s) for 1 benchmark(s))
..

benchmark results:
Execution time (sec)
name ruby 2.3.0dev (2015-02-26 trunk 49757) [x86_64-linux]
app_mandelbrot 1.159

[nobu (Nobuyoshi Nakada)]

On MacBook Pro late 2013, OSX 10.10.2, 2.6GHz 16GiB.
At a git-svn working directory, ($build is the build directory):

$ execs=; for c in `git log --since=2015/2/1 --format=%H --reverse -- complex.c`; do v=`git log -1 $c | sed '/git-svn-id:/!d;s/.*@//;s/ .*//'`; git co $c -- complex.c numeric.c && echo $'#define RUBY_REVISION '$v$'\n#define RUBY_BRANCH_NAME "bm_complex"' > revision.h && rm -f $build/{complex,numeric,version}.o && make -C $build EXEEXT=-$v mini && execs="${execs};r$v::$build/miniruby-$v"; done && ruby benchmark/driver.rb -fm -e "${execs}" -p mandel 

---

benchmark results:
Execution time (sec)

name	r49723	r49724	r49732	r49743	r49754	r49757
app_mandelbrot	0.982	0.959	1.004	0.988	1.092	1.150
so_mandelbrot	1.750	1.750	1.739	1.823	1.729	1.725

Speedup ratio: compare with the result of `r49723' (greater is better)

name	r49724	r49732	r49743	r49754	r49757
app_mandelbrot	1.024	0.978	0.994	0.899	0.854
so_mandelbrot	1.000	1.006	0.960	1.012	1.015

Log file: bmlog-20150305-115133.69279.md

[tgxworld (Guo Xiang Tan)]

Nobu from your benchmarks, can we say that it is a regression or is it a trade off made?

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed