Project

General

Profile

Bug #14191

Coverage decreased between 2.4.3 and 2.5.0rc1

Added by kddeisz (Kevin Deisz) over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
ruby -v:
2.5.0rc1
[ruby-core:84294]

Description

Just ran the test suite on my Rails app on the new rc and noticed that the coverage dropped by a full 3%. After some investigation discovered a lot of lines aren't being reported as covered anymore even though they clearly are. Here are all of the instances that I found that don't report coverage when they should:

# test.rb
require 'coverage'
Coverage.start

require_relative './file.rb'
p Coverage.result
# file.rb
FOO = [
  { foo: 'bar' },
  { bar: 'baz' }
]

'some string'.split
             .map(&:length)

some =
  'value'

Struct.new(
  :foo,
  :bar
).new

class Test
  def foo(bar)
    {
      foo: bar
    }
  end
end

Test.new.foo(Object.new)

In this case we're not getting coverage on the constant assignment, on the second method of the string manipulation, on the multi-line assignment, on the instantiation of the struct, or on the return value of the Test#foo method.

Updated by mame (Yusuke Endoh) over 2 years ago

  • Target version set to 2.5
  • Assignee set to mame (Yusuke Endoh)
  • Status changed from Open to Assigned

Thank you for the report!

I found that #14104 changed TracePoint's lineno, which broke coverage.so. I'll fix it with ko1.

#2

Updated by mame (Yusuke Endoh) over 2 years ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r61313.


iseq.c (finish_iseq_build): fix coverage leakage [Bug #14191]

Before this change, coverage.so had failed to measure some multiple-line
code fragments. This is because removing trace instructions (#14104)
changed TracePoint's lineno (new lineno), and coverage counter array was
based on old lineno.
This change initializes coverage counter array based on new lineno.

Updated by mame (Yusuke Endoh) over 2 years ago

I have just fixed the issue. I'd appreciate if you could try r61313 and tell me if the coverage is measured correctly.

Your report was really helpful. Thank you very much!

Updated by mame (Yusuke Endoh) over 2 years ago

Sangyong Sim told me that 2.5's line coverage measurement was slower than 2.4 because of some reasons. I talked with ko1, and decided to revert the changes of line coverage (2.4 and 2.5 use a special event and special instruction for line coverage). I committed r61350. This change also means that line numbers are back to 2.4. In short, everything about 2.5's line coverage should be the same as 2.4.

Also available in: Atom PDF