Bug #19872
closedTestRequireLib#test_thread_size test case is unstable
Description
Just exploring the difference in skipped test cases, it seems to be due to TestRequireLib#test_thread_size
. One test run might look like:
84) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/history.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
85) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/line_editor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
86) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/net/http/generic_request.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Net
87) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/formatter.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
88) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/generator/pot/message_extractor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
89) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/block_quote.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
90) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/irb/context.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant IRB
91) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/indented_paragraph.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
92) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/parser/text.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
93) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/to_label.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
While in the other run, there is e.g. just one test skipped:
40) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markdown/entities.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
I can't help myself, but this test case contains too much randomness to my taste.
Updated by vo.x (Vit Ondruch) about 1 year ago
Maybe predefining some well know namespaces could help? Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?
Updated by vo.x (Vit Ondruch) about 1 year ago
vo.x (Vit Ondruch) wrote in #note-1:
Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?
Randomly looking at lib/reline/history.rb, maybe the Reline
namespace could be defined on top of the file, why not?
Updated by mame (Yusuke Endoh) about 1 year ago
Maybe this is becaue the test of TestRequireLib
does sampling: https://github.com/ruby/ruby/blob/f1c78b23313e63eb31a213cc7277b1eb206a581b/test/ruby/test_require_lib.rb#L13
@znz (Kazuhiro NISHIYAMA) Should we fix the random seed at least? @nobu (Nobuyoshi Nakada) says that it should test only toplevel ruby files like drb
(and some special files like net/http
)
Updated by nobu (Nobuyoshi Nakada) about 1 year ago
- Status changed from Open to Closed
Applied in changeset git|234722ed9fce33d7ec8152920636af663d527912.
[Bug #19872] Refine TestRequireLib
Always test only the scripts just under “lib", and just under
child directories which has not the same name script in the upper
level; instead of random sampling from whole libraries.
Updated by vo.x (Vit Ondruch) about 1 year ago
Have not chance to test it yet, but the change LGTM. Thx!