Feature #13666


Development: Writing test code for new features/bug fixes can be done as specs under spec/rubyspec instead of tests under test/

Added by Eregon (Benoit Daloze) almost 6 years ago. Updated over 5 years ago.

Target version:


Dear CRuby committers,

I would like to reach an agreement with the ruby core team on a subject that was once highly debated.
I believe nowadays the context is quite different and it's better to start the discussion from the beginning.

Currently, when adding a new feature or fixing a bug, I believe the consensus is to add a corresponding test under test/.

I would like to change this to also allow writing a spec under spec/rubyspec,
and when doing so allow to write the spec instead of the test under test/ (because nobody wants to write test code twice).

So the model would become:

  1. Write a new feature/fix a bug/etc
  2. Write a test (under test/) OR a spec (under spec/rubyspec)

I believe this makes sense given that:

  • make test-spec is part of make exam
  • spec/rubyspec is tested on every CI of MRI, just like test/

I want to encourage MRI developers to write specs under spec/rubyspec.
This change is needed so that we do not need to write the same test twice (which also has little value).

My motivation is that spec/rubyspec specs is so much more useful to other Ruby implementations than tests under test/.
I believe most alternative Ruby implementers would agree about this.

I do not want to be too verbose here but here are the main points.
Most Ruby implementations run ruby/spec (the same as spec/rubyspec as they are synchronized regularly).
Some Ruby implementations also run tests from test/, but have to exclude some files as they do not load,
and when a single assertion fails they have to exclude the entire test method.
Specs under spec/rubyspec use small examples/tests with a textual description that makes it immediately clear what is being tested.
When it fails, it gives a good idea of what is failing via the description, without having to look all the time at the test code.
When implementing a new feature, it allows to focus on specific aspects instead of having to pass everything at once.
When some edge case of a feature is hard to implement, one can just tag (exclude) that specific spec example, but keep running all the others.

As a summary, I believe more specs under spec/rubyspec can help to have more compatible Ruby implementations,
help alternative implementations to implement faster new features and document more precisely the semantics of these features.
Since most Ruby implementations contribute to ruby/spec, we can therefore work together to make a great test/spec suite for Ruby.


Also available in: Atom PDF