Feature #13666: Development: Writing test code for new features/bug fixes can be done as specs under spec/rubyspec instead of tests under test/

• Status: Closed
• Assignee: -
• Target version: -

[Eregon (Benoit Daloze)]

Dear CRuby committers,

I would like to reach an agreement with the ruby core team on a subject that was once highly debated.
I believe nowadays the context is quite different and it's better to start the discussion from the beginning.

Currently, when adding a new feature or fixing a bug, I believe the consensus is to add a corresponding test under test/.

I would like to change this to also allow writing a spec under spec/rubyspec,
and when doing so allow to write the spec instead of the test under test/ (because nobody wants to write test code twice).

So the model would become:

1. Write a new feature/fix a bug/etc
2. Write a test (under test/) OR a spec (under spec/rubyspec)

I believe this makes sense given that:

• make test-spec is part of make exam
• spec/rubyspec is tested on every CI of MRI, just like test/

I want to encourage MRI developers to write specs under spec/rubyspec.
This change is needed so that we do not need to write the same test twice (which also has little value).

My motivation is that spec/rubyspec specs is so much more useful to other Ruby implementations than tests under test/.
I believe most alternative Ruby implementers would agree about this.

I do not want to be too verbose here but here are the main points.
Most Ruby implementations run ruby/spec (the same as spec/rubyspec as they are synchronized regularly).
Some Ruby implementations also run tests from test/, but have to exclude some files as they do not load,
and when a single assertion fails they have to exclude the entire test method.
Specs under spec/rubyspec use small examples/tests with a textual description that makes it immediately clear what is being tested.
When it fails, it gives a good idea of what is failing via the description, without having to look all the time at the test code.
When implementing a new feature, it allows to focus on specific aspects instead of having to pass everything at once.
When some edge case of a feature is hard to implement, one can just tag (exclude) that specific spec example, but keep running all the others.

As a summary, I believe more specs under spec/rubyspec can help to have more compatible Ruby implementations,
help alternative implementations to implement faster new features and document more precisely the semantics of these features.
Since most Ruby implementations contribute to ruby/spec, we can therefore work together to make a great test/spec suite for Ruby.

[Eregon (Benoit Daloze)]

• Subject changed from Development: Writing test code for new features/bug fixes can be done specs under spec/rubyspec instead of tests under test/ to Development: Writing test code for new features/bug fixes can be done as specs under spec/rubyspec instead of tests under test/

[duerst (Martin Dürst)]

• Status changed from Open to Closed

We discussed this this issue at today's Ruby Committer meeting.

We agreed that the new setup makes it easier to write specs. It is no longer necessary to also write tests for MRI.

We noted that different developers may be more or less familiar with specs, and more or less at ease with writing specs. Therefore, each developer should write tests/specs the way it is most convenient for them.

We are closing this issue so that it is excluded from further tracking.

We would like to thank you for your help in integrating ruby spec better into MRI.

[Eregon (Benoit Daloze)]

Awesome!

This should make writing specs in MRI possible without duplicated work,
and at the same time make other Ruby implementations benefit greatly from it.

I see @nobu (Nobuyoshi Nakada) already showed the example with https://github.com/ruby/ruby/commit/0f6e7edf72cdbb071214f91854e1f002916f7af0

(Somehow I missed this reply, I think due to the issue rename and checking my mails less often on holidays)