Feature #22202: Add Coverage.branch_stub and Coverage.method_stub as companions to Coverage.line_stub

• Status: Open
• Assignee: -
• Target version: -

[sferik (Erik Berlin)]

Coverage.line_stub(path) compiles a file and returns its line coverage array as if the file had been loaded but never run. SimpleCov uses this method to include never-loaded files in a report so that their lines count toward the denominator. But there is no equivalent for branch or method coverage. A tool that wants a never-loaded file to participate in branch or method totals must reconstruct the structures itself with a parser.

The difficulty is that reconstruction must match the VM byte-for-byte. Merged coverage reports key branches and methods on their full location tuples, for example [:if, id, start_line, start_col, end_line, end_col]. If a tool synthesizes a tuple whose location differs from what the VM records for the same construct, merging a synthesized entry with a real one produces a duplicate branch that no execution can ever hit. The report then shows a permanently missed branch in a fully covered file.

These location conventions are not documented and they change between Ruby versions. Cases SimpleCov has had to reverse engineer recently:

1. The range of an elsif clause changed between Ruby 3.3 and 3.4 (https://github.com/simplecov-ruby/simplecov/issues/1226).
2. A safe navigation call followed by a block ends its branch range at the call, excluding the block (https://github.com/simplecov-ruby/simplecov/issues/1233).
3. Conditions that are compile-time literals (if true, ternary on an integer) are folded and produce no branch at all, while while true still does.
4. On Ruby 3.3 the location of an empty branch arm depends on whether the construct is in value or void position. Ruby 3.4 removed this distinction.
5. x => pattern and x in pattern produce a :case branch on Ruby 3.3 and nothing on 3.4.

SimpleCov now carries roughly 500 lines of Prism-based reconstruction plus a differential fuzzer that compiles thousands of generated programs under real Coverage and diffs the tuples, only to keep parity with conventions the VM already knows. Each new Ruby version can silently change a convention and reintroduce this class of bug.

Proposal

Add two class methods to the Coverage standard library:

require "coverage"

Coverage.branch_stub("foo.rb")
# => {[:if, 0, 2, 2, 6, 5] => {[:then, 1, 3, 4, 3, 10] => 0, [:else, 2, 5, 4, 5, 10] => 0}}

Coverage.method_stub("foo.rb")
# => {["Foo", :bar, 2, 2, 4, 5] => 0}

Return value shapes are identical to Coverage.result[path][:branches] and Coverage.result[path][:methods] so existing merge code works unchanged. Like line_stub, the file is compiled but not executed, and Coverage does not need to be running. It should raise the same errors line_stub raises for missing or unparsable files.

The VM already computes these structures at compile time whenever coverage is enabled with branches: true or methods: true. line_stub is implemented by compiling the file and walking iseq.trace_points. A similar approach should work here.

I am happy to help with a patch if this direction is acceptable.

[mame (Yusuke Endoh)]

Good news: starting with Ruby 4.1, thanks to [Bug #22018], files compiled via RubyVM::InstructionSequence.compile_file will be subject to coverage measurement.

# not-loaded.rb
if rand > 0.5
  foo
else
  bar
end

Coverage.start(lines: true, branches: true)

RubyVM::InstructionSequence.compile_file("not-loaded.rb")

pp Coverage.result
#=> {"not-loaded.rb" =>
#     {lines: [nil, 0, 0, nil, 0, nil],
#      branches: {[:if, 0, 2, 0, 6, 3] => {
#        [:then, 1, 3, 2, 3, 5] => 0,
#        [:else, 2, 5, 2, 5, 5] => 0}}}}

For your use case of including never-loaded files in a report, I think the simplest solution would be to call RubyVM::InstructionSequence.compile_file on each never-loaded file right before calling Coverage.result. What do you think?

That said, as for method coverage, its result key includes the class on which each method is defined, so I think it is fundamentally difficult to obtain a stub without execution. Line and branch coverage stubs are static information (available at compile time), but method coverage is purely dynamic information, so I am afraid this is unavoidable. If you really need something similar, the only approach I can think of is to build an approximate stub using Prism.