Bug #13861
closed
Performance regressoion in Hash literal in Ruby 2.5.0-dev
Added by watson1978 (Shizuo Fujita) over 6 years ago.
Updated over 6 years ago.
Status:
Third Party's Issue
ruby -v:
ruby 2.5.0dev (2017-09-02 trunk 59626) [x86_64-linux]
[ruby-dev:50216]
Description
Seems Ruby 2.5.0-dev has performance regressoion in Hash literal.
I've confirmed it with below benchmark script
Hash literal performance -> 35.7 % slow down since Ruby 2.4.1
Environment¶
- Ubuntu 17.04
- gcc version 7.0.1
Ruby 2.5.0-dev¶
$ ./miniruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.5.0dev (2017-09-02 trunk 59626) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 93.650k i/100ms
String 226.366k i/100ms
Calculating -------------------------------------
Hash 1.713M (± 0.2%) i/s - 8.616M in 5.030997s
String 16.477M (± 0.1%) i/s - 82.397M in 5.000892s
Ruby 2.4.1¶
$ ruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 117.135k i/100ms
String 259.608k i/100ms
Calculating -------------------------------------
Hash 2.663M (± 0.3%) i/s - 13.353M in 5.014087s
String 17.259M (± 0.2%) i/s - 86.449M in 5.009036s
Benchmark code¶
require 'benchmark/ips'
Benchmark.ips do |x|
x.report "Hash" do |loop|
count = 0
while count < loop
hash = {foo: 12, bar: 34, baz: 56}
count += 1
end
end
x.report "String" do |loop|
count = 0
while count < loop
string = "hello world"
count += 1
end
end
end
Files
watson1978 (Shizuo Fujita) wrote:
Seems Ruby 2.5.0-dev has performance regression in Hash literal.
Similar results on MinGW Win7 gcc 7.1.0 using ruby for both
ruby 2.5.0dev (2017-09-02 trunk 59723) [x64-mingw32]
ruby 2.4.1p111 (2017-03-22 revision 58053) [x64-mingw32]
watson1978 (Shizuo Fujita) wrote:
Hash literal performance -> 35.7 % slow down since Ruby 2.4.1
I have to disagree.
require 'benchmark/ips'
n = 32
Benchmark.ips do |x|
# precalc literals
lit = []
(0..n).each{|i|lit[i]="x.report(#{i}){{".dup}
(0..n).each{|i|(0...i).each{|j|lit[n-j]<< "i#{i}: #{i}, "}}
(0..n).each{|i|lit[i]<< "}}"}
bench = lit.join("\n")
# let's go
eval bench
end
This benchmark yields attached result on my machine.
It is clear that trunk is overall much faster than 2.4. Hash literals of 3 and 4 elements are the only cases that needs improvements.
shyouhei (Shyouhei Urabe) wrote:
I have to disagree.
Thank you for the thorough analysis and the reminder to always vary the length when testing anything with one...
- Status changed from Open to Third Party's Issue
After looking at this more closely, I now suspect it's benchmark/ips who's doing something nasty. Without it, trunk runs order of magnitude faster than 2.4.
zsh % time ruby@trunk --disable-gems -ve 'i=0; while i<100_000_000 do { foo: 1, bar: 2, baz: 3 }; i+=1 end'
ruby 2.5.0dev (2017-09-03 trunk 59738) [x86_64-darwin15]
ruby --disable-gems -ve 1.77s user 0.03s system 98% cpu 1.833 total
zsh % time ruby@ruby_2_4 --disable-gems -ve 'i=0; while i<100_000_000 do { foo: 1, bar: 2, baz: 3 }; i+=1 end'
ruby 2.4.2p181 (2017-08-05 revision 59520) [x86_64-darwin15]
ruby --disable-gems -ve 25.55s user 0.08s system 99% cpu 25.708 total
I think this is not 3rd party issue.
Because this regression appears since 2017-04-28
$ ./miniruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.5.0dev (2017-04-27 trunk 58469) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 119.163k i/100ms
String 194.413k i/100ms
Calculating -------------------------------------
Hash 2.587M (± 0.5%) i/s - 12.989M in 5.021489s
String 13.874M (±20.8%) i/s - 66.489M in 5.005903s
$ ./miniruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.5.0dev (2017-04-28 trunk 58489) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 102.010k i/100ms
String 227.168k i/100ms
Calculating -------------------------------------
Hash 1.674M (± 2.4%) i/s - 8.365M in 5.000468s
String 12.829M (± 0.6%) i/s - 64.289M in 5.011156s
Seems this regression was fixed by https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?revision=59744&view=revision
Thank you for fixing!!!
before¶
$ ./miniruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.5.0dev (2017-09-05 trunk 59753) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 96.137k i/100ms
String 219.522k i/100ms
Calculating -------------------------------------
Hash 1.723M (± 0.6%) i/s - 8.652M in 5.021705s
String 16.975M (± 0.3%) i/s - 84.955M in 5.004690s
after¶
$ ./miniruby -v -Ilib -I../benchmark-ips/lib ~/tmp/bench.rb
ruby 2.5.0dev (2017-09-06 trunk 59810) [x86_64-linux]
/home/watson/tmp/bench.rb:7: warning: assigned but unused variable - hash
/home/watson/tmp/bench.rb:15: warning: assigned but unused variable - string
Warming up --------------------------------------
Hash 135.385k i/100ms
String 234.168k i/100ms
Calculating -------------------------------------
Hash 2.863M (± 0.1%) i/s - 14.351M in 5.012065s
String 16.869M (± 7.1%) i/s - 83.832M in 5.010604s
Also available in: Atom
PDF
Like0
Like0Like0Like0Like0Like0Like0