Bug #16009: Performance regression in 2.7

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: -
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN

[ioquatix (Samuel Williams)]

Number is requests/s

dcf5c19c9f89d732da70a1a16a2fe60cd1999bcc          7561.82

c53f87943e53c96b86d50b496d2a410ff1245b4c          7534.09

d2003a6d392b3b0054d7528e2e731584196aefad          7467.44, 7458.18, 7111.09, 7293.74, 7493.13

f54aa6c5b286b2b44bcdb1958fc9b1ebfce3559e          7558.63

18e43e823106f15c8aaceb1f56874bdf67bc36a3          7439.23, 7356.25, 7437.61, 7489.07

7069f64c419ebb9a7fd3e48d81454148ed4b2fba          7580.31

a160b2f56716f70fa3e485ae89875da48baefc1d          7597.52

88449100bc6d23a00dbf3addb97665f4f606f2b8          7258.96, 7184.62, 7190.71 *** Performance regression

e0f0ab959e9a0fa3db8dfdb2a493b057d6e7541b          7145.69, 7061.17, 7217.51

4d9c3a8c2362b7d5973057435258e447ce733741          7276.46, 7203.8

498113d5d39a4227c2b9a9c11bea895fe316e6b9          7218.93

c55db6aa271df4a689dc8eb0039c929bf6ed43ff          7149.20

[ioquatix (Samuel Williams)]

@ko1 (Koichi Sasada) it took some time to find this, and it's late, so issue is brief. Feel free to discuss with me if you need more information.

[ioquatix (Samuel Williams)]

Comparing master branch with master + reverted 88449100bc6d23a00dbf3addb97665f4f606f2b8

koyoko% make benchmark ITEM=vm2_fiber_gc COMPARE_RUBY=`which ruby`
../revision.h unchanged
/home/samuel/.rvm/rubies/ruby-2.6.3/bin/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \
            --executables="compare-ruby::/home/samuel/.rvm/rubies/ruby-head-master/bin/ruby -I.ext/common --disable-gem" \
            --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common  ../tool/runruby.rb --extout=.ext  -- --disable-gems --disable-gem" \
            $(find ../benchmark -maxdepth 1 -name '*vm2_fiber_gc*.yml' -o -name '*vm2_fiber_gc*.rb' | sort) 
Calculating -------------------------------------
                     compare-ruby  built-ruby 
  vm2_fiber_reuse_gc       27.009      35.982 i/s -     100.000 times in 3.702437s 2.779146s

Comparison:
               vm2_fiber_reuse_gc
          built-ruby:        36.0 i/s 
        compare-ruby:        27.0 i/s - 1.33x  slower

[ko1 (Koichi Sasada)]

More info about this ticket:

This benchmark is on falcon application, many Fiber creation benchmark.
The table shows commit hash and the results (Requests per seconds).
In brief, there is a regression between master and ruby 2.6.3.
Samuel found it and run it with bisect. This table is the result.

https://github.com/ruby/ruby/commit/88449100bc6d23a00dbf3addb97665f4f606f2b8 is a patch to enable lazy sweep.
Before this commit, lazy sweep is disabled accidentally (my fault).

Conclusion: I decide to ignore it.

Maybe this problem is lazy sweep + big memory object.
Bulk freeing is faster than incremental freeing for such big memory objects.
With small objects, lazy sweep is not so slow or faster.

So maybe it is rare case so I decide to ignore it, at this time.
Maybe we need to care about big memory objects, in a future.

[ko1 (Koichi Sasada)]

• Status changed from Open to Closed

[ioquatix (Samuel Williams)]

require 'benchmark'
  
require 'objspace'

# this line prohibit lazy sweep
# Thread.new{ObjectSpace.each_object{sleep}}

Benchmark.bm{|x|
  x.report{
    100.times{
      fibers = []

      2000.times do
        fiber = Fiber.new{Fiber.yield}.resume
        fibers << fiber
      end
    }
  }
}