Bug #12015: Improving code coverage visibility

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN

[mlarraz (Matt Larraz)]

In https://bugs.ruby-lang.org/issues/10189 the simplecov gem was added to generate coverage reports for specs.

However, the docs in coverage/README are very sparse and appear to be out of date. At least, I haven't been able to generate a coverage report locally.

Even if I can do so, and that process is made easier, it still would be nice to have a definitive source of the current coverage level. This mirrors using Travis and AppVeyor as the definitive sources for the current status of test runs.

I think that especially on a large project with many contributors, code coverage is a valid part of a CI pipeline. Discouraging patches that drop coverage is beneficial in the same way as discouraging ones that break tests.

Personally, I've used coveralls.io for this purpose because they integrate nicely with Travis and follow a similar licensing model. However, any similar solution would be just as great.

Also, I would eventually like to see unified coverage reports for Ruby's C code as well its Ruby code, but I understand that may not be very feasible currently.

[mame (Yusuke Endoh)]

• Status changed from Open to Closed

Closing as it is not a bug.

Now you can see unified coverage reports for Ruby's C and Ruby code in https://rubyci.org/coverage. It is measured under GitHub Actions: https://github.com/ruby/actions/blob/master/.github/workflows/coverage.yml

I agree that coverage/README is less informative. A contribution is welcome. But at least, you can see the coverage locally by the command in the doc:

$ COVERAGE=1 make test-all TESTS=test/cgi
...

# Running tests:

Finished tests in 0.195891s, 2241.0406 tests/s, 11899.4661 assertions/s.                  
439 tests, 2331 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.7.0dev (2019-12-18T05:53:19Z master efbc0d499c) [x86_64-linux]
Coverage report generated for Ruby's `make test-all` to /path/to/coverage. 1020 / 1492 LOC (68.36%) covered.

And open /path/to/coverage/index.html in your browser.