Bug #6765

test_filename_utf8_raw_name and test_filename_euc_jp (test_dir_m17n.rb) fail on non-JapaneseWindows

Added by Oleg Sukhodolsky about 3 years ago. Updated about 3 years ago.

[ruby-core:46589]
Status:Closed
Priority:Normal
Assignee:Luis Lavena
ruby -v:ruby 2.0.0dev (2012-07-21) [i386-mingw32] Backport:

Description

It looks like the problem is related to http://bugs.ruby-lang.org/issues/4799
The problem is that these test use "\u3042" as a file name to create and to test its name in ASCII-8BIT mode.
But it expects "\x82\x40" as a result, but this is "\u3042".encode('sjis') ('jsis' is a filesystem encoding
on Japanese Windows).
To fix the test I do changed it to calculate expected file name using original name and filesystem's encoding.
Also, I've added two more tests with Windows-1251 and Windows-1252 symbols as file name.
And they does demonstrated another problem of the test on Windows: result of the function Ruby uses to
get binary version of filename (WideCharToMultiByte) is not deterministric (see

http://msdn.microsoft.com/en-us/library/windows/desktop/dd374047(v=vs.85).aspx#SC_char_conv_func for more details).
This is why I've added additional check to which test to check if at least we have expected file name in case we use fylesystem's encoding.

test_dir_m17n.rb.diff Magnifier - patch for test_dir_m17n.rb (3.55 KB) Oleg Sukhodolsky, 07/21/2012 09:04 PM

test_dir_m17n.rb.v2.diff Magnifier - new version of the patch (4 KB) Oleg Sukhodolsky, 07/22/2012 12:30 AM

Associated revisions

Revision 36509
Added by Luis Lavena about 3 years ago

Fix raw UTF-8 test for non-Japanase Windows systems

  • test/ruby/test_dir_m17n.rb (create_and_check_raw_file_name): add new
    helper method to ease encoding testing. Patch by Oleg Sukhodolsky.
    [Bug #6765]

  • test/ruby/test_dir_m17n.rb (test_filename_extutf8): use filesystem
    encoding when reading entries and comparing.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_name): removed.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_jp_name): split test.

Revision 36509
Added by Luis Lavena about 3 years ago

Fix raw UTF-8 test for non-Japanase Windows systems

  • test/ruby/test_dir_m17n.rb (create_and_check_raw_file_name): add new
    helper method to ease encoding testing. Patch by Oleg Sukhodolsky.
    [Bug #6765]

  • test/ruby/test_dir_m17n.rb (test_filename_extutf8): use filesystem
    encoding when reading entries and comparing.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_name): removed.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_jp_name): split test.

History

#1 Updated by Luis Lavena about 3 years ago

  • Status changed from Open to Feedback

=begin
Thank you Oleg for the patch.

I just tested it, but still have a failure:

Before patch:

# Running tests:

[ 4/10] TestDir_M17N#test_filename_euc_jp = 0.08 s
1) Failure:
test_filename_euc_jp(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:170]:
ruby exit status is not success: #

[ 9/10] TestDir_M17N#test_filename_utf8_raw_name = 0.08 s
2) Failure:
test_filename_utf8_raw_name(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:190]:
ruby exit status is not success: #

Finished tests in 0.657084s, 15.2188 tests/s, 27.3938 assertions/s.
10 tests, 18 assertions, 2 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

After patch:

# Running tests:

[ 4/12] TestDir_M17N#test_filename_euc_jp = 0.08 s
1) Failure:
test_filename_euc_jp(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:196]:
ruby exit status is not success: #

Finished tests in 0.809603s, 14.8221 tests/s, 27.1738 assertions/s.
12 tests, 22 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

Seems like (({test_filename_euc_jp})) is still failing.

My codepage is 1252, changing it to 1251 seems to solve the issue, which indicate some minor issue with this test.

Can you confirm with chcp 1252?

Thank you.
=end

#2 Updated by Oleg Sukhodolsky about 3 years ago

The patch updated to make test_filename_euc_jp pass on Windows-1252.

#3 Updated by Luis Lavena about 3 years ago

  • Status changed from Feedback to Assigned
  • Target version set to 2.0.0

=begin
Thank you Oleg,

I can confirm new patch works:

# Running tests:

Finished tests in 0.785099s, 15.2847 tests/s, 28.0219 assertions/s.
12 tests, 22 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

Nobu, Usa, can I commit?

Any with Japanese-Windows to double check?

Thank you
=end

#4 Updated by Luis Lavena about 3 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r36509.
Oleg, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


Fix raw UTF-8 test for non-Japanase Windows systems

  • test/ruby/test_dir_m17n.rb (create_and_check_raw_file_name): add new
    helper method to ease encoding testing. Patch by Oleg Sukhodolsky.
    [Bug #6765]

  • test/ruby/test_dir_m17n.rb (test_filename_extutf8): use filesystem
    encoding when reading entries and comparing.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_name): removed.

  • test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_jp_name): split test.

Also available in: Atom PDF