Bug #13500: MinGW TestArity#test_proc_err_mess stops testing

• Status: Open
• Priority: Normal
• Assignee: -
• Target version: -
• ruby -v: ruby 2.5.0dev (2017-04-24 trunk 58462) [x64-mingw32]
• Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
• Tags: win, test

[MSP-Greg (Greg L)]

With MinGW build of ruby 2.5.0dev (2017-04-24 trunk 58462) [x64-mingw32], both test-all and an external test stop in TestArity#test_proc_err_mess.

Solution is to either add the following line args.to_s --

  def err_mess(method_proc = nil, argc = 0)
    args = (1..argc).to_a
    assert_raise_with_message(ArgumentError, /wrong number of arguments \(.*\b(\d+)\b.* (\d\S*?)\)/) do
      case method_proc
      when nil
        yield
      when Symbol
        method(method_proc).call(*args)
      else 
+       args.to_s
        method_proc.call(*args)
      end
    end
    [$1, $2]
  end

or comment out the two lines --

  def test_proc_err_mess
#   assert_equal %w[0 1..2],  err_mess(->(b, c=42){}, 0)
    assert_equal %w[1 2+],    err_mess(->(a, b, c=42, *d){}, 1)
    assert_equal %w[3 4+],    err_mess(->(a, b, *c, d, e){}, 3)
    assert_equal %w[3 1..2],  err_mess(->(b, c=42){}, 3)
#   assert_equal %w[1 0],     err_mess(->(&block){}, 1)
    # Double checking:
    p = Proc.new{|b, c=42| :ok}
    assert_equal :ok,  p.call(1, 2, 3)
    assert_equal :ok,  p.call
  end

Once again, seems very odd. I have no idea why the first solution would change anything... Since this is a silent failure, if there's anything I can do to get more info, please let me know.

Also, there may be more issues; I haven't gone back to test-all with this patched to see if it completes.

Thank you.

[MSP-Greg (Greg L)]

This morning, a MinGW build of

ruby 2.5.0dev (2017-04-24 trunk 58467) [x64-mingw32]

completed test-all with

16621 tests, 2227533 assertions, 8 failures, 1 errors, 114 skips

I've mentioned that I am not fluent in c. Closest I've come is C#, and that was quite a while ago, and I realize that C# is managed and GC'd. Conversely, I have been coding for a very long time.

If there is anything I can do to help solve this issue (which is intermittent), please let me know.

Today, I grabbed the tests that have intermittently stopped test-all, and ran them 200 times. They completed, which is a good sign, but I've never tried it before. I'll continue to do that with all builds.

The tests were the following:

ruby --disable-gems runner_t.rb -v -druby_stop
Run options: -v --show-skip

# Running tests:

[1/6] TestArity#test_proc_err_mess = 0.00 s
[2/6] TestKeywordArguments#test_block_required_keyword = 0.00 s
[3/6] TestKeywordArguments#test_f4 = 0.00 s
[4/6] TestKeywordArguments#test_f5 = 0.00 s
[5/6] TestLambdaParameters#test_exact_parameter = 0.00 s
[6/6] TestRubyYieldGen#test_yield_lambda = 1.73 s

Finished tests in 1.762800s, 3.4037 tests/s, 3041.7518 assertions/s.
6 tests, 5362 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.5.0dev (2017-04-24 trunk 58467) [x64-mingw32]

Just in case, I've summarized my build code below.

I have several patches that I've needed over the past few months to allow test-all to complete. For this build, I had patches for:

test/ruby/test_enum.rb
test/socket/test_socket.rb

Previously, for test-all to complete, I've needed patches for:

test/fileutils/test_fileutils.rb
test/ruby/test_arity.rb
test/ruby/test_keyword.rb
test/ruby/test_lambda.rb
test/ruby/test_yield.rb

My configure 'specials' are:

  --with-baseruby=${RUBY} \
  --disable-install-doc \
  --with-git=${GIT} \
  --with-out-ext=pty,syslog

Patches to 'base' ruby are listed in 13496.

My make lines are:

make BASERUBY=${RUBY} touch-unicode-files
make after-update BASERUBY=${RUBY} UNICODE_FILES=.
make -j3 all

If I've got anything incorrect in the above, please let me know. Finally, thanks for everyone's help.

[MSP-Greg (Greg L)]

Today (2017-04-26), I built ruby 2.5.0dev (2017-04-26 trunk 58487) [x64-mingw32]

To complete test-all, it required additional patches for:

test/fileutils/test_fileutils.rb
test/ruby/test_arity.rb
test/ruby/test_keyword.rb
test/ruby/test_yield.rb

After those patches, it had the following result:

16622 tests, 2222464 assertions, 8 failures, 1 errors, 114 skips

Yesterday, I built ruby 2.5.0dev (2017-04-25 trunk 58476) [x64-mingw32]

With the following result (no 'additional' patches were required):

16622 tests, 2227747 assertions, 8 failures, 1 errors, 114 skips

Today and yesterday's builds both used patches for the following:

test/ruby/test_enum.rb
test/socket/test_socket.rb

I believe most of the assertion differential is due to the patch for test/ruby/test_yield.rb, which is:

--- a/test/ruby/test_yield.rb	Sat Apr 22 07:41:22 2017
+++ b/test/ruby/test_yield.rb	Sat Apr 22 15:45:20 2017
@@ -372,7 +372,7 @@
   end
 
   def test_yield_lambda
-    assert_all_sentences(Syntax, :test_lambda, 4) {|t|
+    assert_all_sentences(Syntax, :test_lambda, 2) {|t|
       check_nofork(t, true)
     }
   end

Many of the tests that require patches seem to involve error handling. If one's code is not heavily reliant on rescue clauses, this may not cause user issues.

Regardless, test-all should not silently stop.