Project

General

Profile

Actions

Bug #19872

closed

TestRequireLib#test_thread_size test case is unstable

Added by vo.x (Vit Ondruch) about 1 year ago. Updated about 1 year ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.3.0dev (2023-09-05 master 7c8932365f) [x86_64-linux]
[ruby-core:114678]

Description

Just exploring the difference in skipped test cases, it seems to be due to TestRequireLib#test_thread_size. One test run might look like:

 84) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/history.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
 85) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/reline/line_editor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Reline
 86) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/net/http/generic_request.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant Net
 87) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/formatter.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 88) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/generator/pot/message_extractor.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 89) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/block_quote.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 90) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/irb/context.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant IRB
 91) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/indented_paragraph.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 92) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/parser/text.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc
 93) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markup/to_label.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc

While in the other run, there is e.g. just one test skipped:

 40) Skipped:
TestRequireLib#test_thread_size:/builddir/build/BUILD/ruby-3.3.0-7c8932365f/lib/rdoc/markdown/entities.rb [/builddir/build/BUILD/ruby-3.3.0-7c8932365f/test/ruby/test_require_lib.rb:21]:
uninitialized constant RDoc

I can't help myself, but this test case contains too much randomness to my taste.

Updated by vo.x (Vit Ondruch) about 1 year ago

Maybe predefining some well know namespaces could help? Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?

Updated by vo.x (Vit Ondruch) about 1 year ago

vo.x (Vit Ondruch) wrote in #note-1:

Or is it issue of the respective libraries, which does not support proper loading of nested files, without loading their dependencies first?

Randomly looking at lib/reline/history.rb, maybe the Reline namespace could be defined on top of the file, why not?

Updated by mame (Yusuke Endoh) about 1 year ago

Maybe this is becaue the test of TestRequireLib does sampling: https://github.com/ruby/ruby/blob/f1c78b23313e63eb31a213cc7277b1eb206a581b/test/ruby/test_require_lib.rb#L13

@znz (Kazuhiro NISHIYAMA) Should we fix the random seed at least? @nobu (Nobuyoshi Nakada) says that it should test only toplevel ruby files like drb (and some special files like net/http)

Actions #4

Updated by nobu (Nobuyoshi Nakada) about 1 year ago

  • Status changed from Open to Closed

Applied in changeset git|234722ed9fce33d7ec8152920636af663d527912.


[Bug #19872] Refine TestRequireLib

Always test only the scripts just under “lib", and just under
child directories which has not the same name script in the upper
level; instead of random sampling from whole libraries.

Updated by vo.x (Vit Ondruch) about 1 year ago

Have not chance to test it yet, but the change LGTM. Thx!

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0