Project

General

Profile

Bug #14191

Coverage decreased between 2.4.3 and 2.5.0rc1

Added by kddeisz (Kevin Deisz) 12 months ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Target version:
ruby -v:
2.5.0rc1
[ruby-core:84294]

Description

Just ran the test suite on my Rails app on the new rc and noticed that the coverage dropped by a full 3%. After some investigation discovered a lot of lines aren't being reported as covered anymore even though they clearly are. Here are all of the instances that I found that don't report coverage when they should:

# test.rb
require 'coverage'
Coverage.start

require_relative './file.rb'
p Coverage.result
# file.rb
FOO = [
  { foo: 'bar' },
  { bar: 'baz' }
]

'some string'.split
             .map(&:length)

some =
  'value'

Struct.new(
  :foo,
  :bar
).new

class Test
  def foo(bar)
    {
      foo: bar
    }
  end
end

Test.new.foo(Object.new)

In this case we're not getting coverage on the constant assignment, on the second method of the string manipulation, on the multi-line assignment, on the instantiation of the struct, or on the return value of the Test#foo method.

Associated revisions

Revision 2e24a66b
Added by mame (Yusuke Endoh) 12 months ago

iseq.c (finish_iseq_build): fix coverage leakage [Bug #14191]

Before this change, coverage.so had failed to measure some multiple-line
code fragments. This is because removing trace instructions (#14104)
changed TracePoint's lineno (new lineno), and coverage counter array was
based on old lineno.
This change initializes coverage counter array based on new lineno.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61313 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 61313
Added by mame (Yusuke Endoh) 12 months ago

iseq.c (finish_iseq_build): fix coverage leakage [Bug #14191]

Before this change, coverage.so had failed to measure some multiple-line
code fragments. This is because removing trace instructions (#14104)
changed TracePoint's lineno (new lineno), and coverage counter array was
based on old lineno.
This change initializes coverage counter array based on new lineno.

Revision 61313
Added by mame (Yusuke Endoh) 12 months ago

iseq.c (finish_iseq_build): fix coverage leakage [Bug #14191]

Before this change, coverage.so had failed to measure some multiple-line
code fragments. This is because removing trace instructions (#14104)
changed TracePoint's lineno (new lineno), and coverage counter array was
based on old lineno.
This change initializes coverage counter array based on new lineno.

Revision c08e8886
Added by mame (Yusuke Endoh) 12 months ago

compile.c: add a RUBY_EVENT_COVERAGE_LINE event for line coverage

2.5's line coverage measurement was about two times slower than 2.4
because of two reasons; (1) vm_trace uses rb_iseq_event_flags (which
takes O(n) currently where n is the length of iseq) to get an event
type, and (2) RUBY_EVENT_LINE uses setjmp to call an event hook.

This change adds a special event for line coverage,
RUBY_EVENT_COVERAGE_LINE, and adds tracecoverage instructions where
the event occurs in iseq.
tracecoverage instruction calls an event hook without vm_trace.
And, RUBY_EVENT_COVERAGE_LINE is an internal event which does not
use setjmp.

This change also cancells lineno change due to the deletion of trace
instructions [Feature #14104]. So fixes [Bug #14191].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 61350
Added by mame (Yusuke Endoh) 12 months ago

compile.c: add a RUBY_EVENT_COVERAGE_LINE event for line coverage

2.5's line coverage measurement was about two times slower than 2.4
because of two reasons; (1) vm_trace uses rb_iseq_event_flags (which
takes O(n) currently where n is the length of iseq) to get an event
type, and (2) RUBY_EVENT_LINE uses setjmp to call an event hook.

This change adds a special event for line coverage,
RUBY_EVENT_COVERAGE_LINE, and adds tracecoverage instructions where
the event occurs in iseq.
tracecoverage instruction calls an event hook without vm_trace.
And, RUBY_EVENT_COVERAGE_LINE is an internal event which does not
use setjmp.

This change also cancells lineno change due to the deletion of trace
instructions [Feature #14104]. So fixes [Bug #14191].

Revision 61350
Added by mame (Yusuke Endoh) 12 months ago

compile.c: add a RUBY_EVENT_COVERAGE_LINE event for line coverage

2.5's line coverage measurement was about two times slower than 2.4
because of two reasons; (1) vm_trace uses rb_iseq_event_flags (which
takes O(n) currently where n is the length of iseq) to get an event
type, and (2) RUBY_EVENT_LINE uses setjmp to call an event hook.

This change adds a special event for line coverage,
RUBY_EVENT_COVERAGE_LINE, and adds tracecoverage instructions where
the event occurs in iseq.
tracecoverage instruction calls an event hook without vm_trace.
And, RUBY_EVENT_COVERAGE_LINE is an internal event which does not
use setjmp.

This change also cancells lineno change due to the deletion of trace
instructions [Feature #14104]. So fixes [Bug #14191].

History

#1 [ruby-core:84297] Updated by mame (Yusuke Endoh) 12 months ago

  • Target version set to 2.5
  • Assignee set to mame (Yusuke Endoh)
  • Status changed from Open to Assigned

Thank you for the report!

I found that #14104 changed TracePoint's lineno, which broke coverage.so. I'll fix it with ko1.

#2 Updated by mame (Yusuke Endoh) 12 months ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r61313.


iseq.c (finish_iseq_build): fix coverage leakage [Bug #14191]

Before this change, coverage.so had failed to measure some multiple-line
code fragments. This is because removing trace instructions (#14104)
changed TracePoint's lineno (new lineno), and coverage counter array was
based on old lineno.
This change initializes coverage counter array based on new lineno.

#3 [ruby-core:84322] Updated by mame (Yusuke Endoh) 12 months ago

I have just fixed the issue. I'd appreciate if you could try r61313 and tell me if the coverage is measured correctly.

Your report was really helpful. Thank you very much!

#4 [ruby-core:84357] Updated by mame (Yusuke Endoh) 12 months ago

Sangyong Sim told me that 2.5's line coverage measurement was slower than 2.4 because of some reasons. I talked with ko1, and decided to revert the changes of line coverage (2.4 and 2.5 use a special event and special instruction for line coverage). I committed r61350. This change also means that line numbers are back to 2.4. In short, everything about 2.5's line coverage should be the same as 2.4.

Also available in: Atom PDF