Bug #6673

No longer able to run tests in parallel (-j2)

Added by Luis Lavena almost 2 years ago. Updated over 1 year ago.

[ruby-core:45975]
Status:Closed
Priority:Low
Assignee:Shota Fukumori
Category:test
Target version:2.0.0
ruby -v:ruby 2.0.0dev [i386-mingw32] at r36254 Backport:

Description

=begin
Hello,

Since a few revisions back, I'm no longer able to run tests in parallel, getting output like the following:

# Running tests:

[666/690] 116=testutils 4824=testerrincallbackC:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:75:in split': invalid byte sequence in UTF-8 (ArgumentError)
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:75:in
block in module:MakeMakefile'
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:74:in each'
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:74:in
module:MakeMakefile'
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:42:in <top (required)>'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in
require'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/customrequire.rb:36:in require'
from C:/Users/Worker/Code/ruby/ruby/test/win32ole/test_err_in_callback.rb:11:in
'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom
require.rb:36:in require'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in
require'
from C:/Users/Worker/Code/ruby/ruby/lib/test/unit/parallel.rb:114:in run'
from C:/Users/Worker/Code/ruby/ruby/lib/test/unit/parallel.rb:177:in
'
Some worker was crashed. It seems ruby interpreter's bug
or, a bug of test/unit/parallel.rb. try again without -j
option.

This was working previously, even on Windows (the platform I'm reporting right now)

Base Ruby is 1.9.3-p194 [i386-mingw32]
Target Ruby is ruby 2.0.0dev (2012-06-29) [i386-mingw32] at r36254

=end

Associated revisions

Revision 37032
Added by Nobuyoshi Nakada over 1 year ago

testgeminstaller.rb: restore ARGV

  • test/rubygems/testgeminstaller.rb (TestGemInstaller#testinstallcreatesbinstubthatdonttrust_encoding): restore ARGV properly, non-ascii strings in different encodings are not equal. [Bug #6673]

History

#1 Updated by Shota Fukumori almost 2 years ago

I can't reproduce this on OS X, is this still reproducable?

If you can still reproduce this on trunk, I'll prepare mingw environment on my Windows box.

#2 Updated by Shota Fukumori almost 2 years ago

  • Status changed from Open to Feedback
  • Assignee set to Shota Fukumori

#3 Updated by Luis Lavena almost 2 years ago

sorah (Shota Fukumori) wrote:

I can't reproduce this on OS X, is this still reproducable?

Yes, this still reproduces on trunk (r36393)

If you can still reproduce this on trunk, I'll prepare mingw environment on my Windows box.

Perhaps you can take a look to RubyChallengePack I prepared, which includes a base Ruby, binaries used by RubyInstaller, GCC + MSYS for building.

You only need Git and a clone of Ruby source.

Details here:

https://groups.google.com/d/topic/rubyinstaller/A92K_EYJX2A/discussion

One note about cloning: ensure Git autocrlf is false:

git config --global core.autocrlf false

So sed works properly with configure script (cloning with CRLF generates some issues).

#4 Updated by Shota Fukumori almost 2 years ago

  • Status changed from Feedback to Assigned

Thanks, I'll take a look for this, but it may take long time to start investigating. sorry!

#5 Updated by Hiroshi Shirosaki over 1 year ago

=begin
I was able to reproduce this with mingw ruby.
ruby 2.0.0dev (2012-09-24 trunk 37025) [x64-mingw32]

If testgeminstaller.rb and testerrin_callback.rb run in same process, this error seems to occur.

$ make test-all TESTS="-q -j2 drb rubygems/testgeminstaller.rb win32ole/testerrin_callback.rb"

[6/6] 4324=testdrbssl 4340=testerrincallbackc:/Users/hiroshi/work/ruby/lib/mkmf.rb:80:in split'
: invalid byte sequence in UTF-8 (ArgumentError)
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:80:in
block in module:MakeMakefile'
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:79:in each'
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:79:in
module:MakeMakefile'
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:46:in <top (required)>'
c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in
require'
c:/Users/hiroshi/work/ruby/lib/rubygems/customrequire.rb:36:in require'
c:/Users/hiroshi/work/ruby/test/win32ole/test_err_in_callback.rb:11:in
'
c:/Users/hiroshi/work/ruby/lib/rubygems/custom
require.rb:36:in require'
c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in
require'
c:/Users/hiroshi/work/ruby/lib/test/unit/parallel.rb:114:in run'
c:/Users/hiroshi/work/ruby/lib/test/unit/parallel.rb:181:in
'

Some worker was crashed. It seems ruby interpreter's bug
or, a bug of test/unit/parallel.rb. try again without -j
option.

testgeminstaller.rb sets invalid encoding string to ARGV. To clear ARGV before running test is a possible fix.

diff --git a/lib/test/unit/parallel.rb b/lib/test/unit/parallel.rb
index 921c6b0..7546630 100644
--- a/lib/test/unit/parallel.rb
+++ b/lib/test/unit/parallel.rb
@@ -110,6 +110,7 @@ module Test
@options = @opts.dup
suites = MiniTest::Unit::TestCase.test_suites

  • ARGV.clear begin require $1 rescue LoadError =end

#6 Updated by Nobuyoshi Nakada over 1 year ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r37032.
Luis, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


testgeminstaller.rb: restore ARGV

  • test/rubygems/testgeminstaller.rb (TestGemInstaller#testinstallcreatesbinstubthatdonttrust_encoding): restore ARGV properly, non-ascii strings in different encodings are not equal. [Bug #6673]

Also available in: Atom PDF