Project

General

Profile

Feature #10634

Baselining with Benchmark

Added by rklemme (Robert Klemme) over 4 years ago. Updated almost 4 years ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:67049]

Description

Add a method Benchmark::Report#baseline. This does not report measurements but stores them as a baseline value in the Benchmark::Report instance. This baseline is nil initially. Any invocation of Benchmark::Report#report will do one of two things:

  1. If baseline is nil, report as today.
  2. If baseline is set, report the difference of the current measurement (Benchmark::Tms) - the baseline value.

We use this to subtract effort for a baseline to get at the net effort. This is how code might look like:

Benchmark.bm do |x|
  x.baseline do
    REP.times {}
  end

  x.report "m" do
    REP.times { f() }
  end
end

Output could be something like this:

       user     system      total        real
m  0.220000   0.000000   0.220000 (  0.221528)

Note the absence of any output for the baseline invocation.

Notes:

  1. Return values of #report, #bm and #bmbm are not changed, i.e. they still contain the raw measurement (i.e. without subtracting the baseline).
  2. We probably want to add another method #clear_baseline to set the baseline value back to nil.
  3. A reasonable variant would be to add a second line to the output presenting measurement minus baseline like this:
       user     system      total        real
m  0.220000   0.000000   0.220000 (  0.221528) measured
m  0.120000   0.000000   0.120000 (  0.121497) normalized

Also available in: Atom PDF