Bug #19872: TestRequireLib#test_thread_size test case is unstable

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-09-05 master 7c8932365f) [x86_64-linux]
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[vo.x (Vit Ondruch)]

Just exploring the difference in skipped test cases, it seems to be due to TestRequireLib#test_thread_size. One test run might look like:

 84) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/history.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
 85) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/line_editor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
 86) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/net/http/generic_request.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Net
 87) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/formatter.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 88) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/generator/pot/message_extractor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 89) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/block_quote.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 90) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/irb/context.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant IRB
 91) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/indented_paragraph.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 92) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/parser/text.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 93) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/to_label.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc

While in the other run, there is e.g. just one test skipped:

 40) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markdown/entities.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc

I can't help myself, but this test case contains too much randomness to my taste.

[vo.x (Vit Ondruch)]

Maybe predefining some well know namespaces could help? Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?

[vo.x (Vit Ondruch)]

vo.x (Vit Ondruch) wrote in #note-1:

Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?

Randomly looking at lib/reline/history.rb, maybe the Reline namespace could be defined on top of the file, why not?

[mame (Yusuke Endoh)]

Maybe this is becaue the test of TestRequireLib does sampling: https://github.com/ruby/ruby/blob/f1c78b23313e63eb31a213cc7277b1eb206a581b/test/ruby/test_require_lib.rb#L13

@znz (Kazuhiro NISHIYAMA) Should we fix the random seed at least? @nobu (Nobuyoshi Nakada) says that it should test only toplevel ruby files like drb (and some special files like net/http)

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|234722ed9fce33d7ec8152920636af663d527912.

---

[Bug #19872] Refine TestRequireLib

Always test only the scripts just under “lib", and just under
child directories which has not the same name script in the upper
level; instead of random sampling from whole libraries.

[vo.x (Vit Ondruch)]

Have not chance to test it yet, but the change LGTM. Thx!