Project

General

Profile

Bug #7758

Ruby on Windows crashes when active codepage is codepage 65001 and outputting unicode character

Added by joshc (Josh C) over 7 years ago. Updated almost 5 years ago.

Status:
Assigned
Priority:
Normal
Target version:
-
ruby -v:
ruby 1.9.3p374
Backport:
[ruby-core:51763]

Description

=begin
I have a script that contains the unicode BOX_DRAWINGS_LIGHT_VERTICAL_AND_RIGHT character (http://www.fileformat.info/info/unicode/char/251c/index.htm). When the console code page is set to 65001 (utf-8), both ruby 1.9.3 p125 and p374 crash, but ruby 1.8.7 p357 does not crash (nor does it output the character correctly). When the code page is set to 437, ruby 1.9.3 (both versions) correctly output the character, while ruby 1.8.7 outputs the wrong character again.

I don't care about ruby 1.8.7 so much, but I would expect ruby 1.9.3 to output the correct character, especially since the console codepage is set to 65001 (utf-8).

Here is my script:

# -- coding: utf-8 --
puts "a"
puts "├"

I verified that the file is correctly encoded:

C:\test>od.exe -t x1 test65001.rb
0000000 23 20 2d 2a 2d 20 63 6f 64 69 6e 67 3a 20 75 74
0000020 66 2d 38 20 2d 2a 2d 0a 70 75 74 73 20 22 61 22
0000040 0a 70 75 74 73 20 22 e2 94 9c 22 0a 0a

Note the UTF-8 sequence e2 94 9c

Here are the codepage settings in the registry:
C:\test>reg query hklm\SYSTEM\CurrentControlSet\Control\Nls\CodePage /v acp
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage
acp REG_SZ 1252
C:\test>reg query hklm\SYSTEM\CurrentControlSet\Control\Nls\CodePage /v oemcp
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage
oemcp REG_SZ 437

On ruby 1.8.7:

C:\test>ruby --version
ruby 1.8.7 (2011-12-28 patchlevel 357) [i386-mingw32]

C:\test>chcp 437
Active code page: 437

C:\test>ruby test65001.rb
a
├

C:\test>chcp 65001
Active code page: 65001

C:\test>ruby test65001.rb
a
├

On ruby 1.9.3p125:

C:\test>ruby --version
ruby 1.9.3p125 (2012-02-16) [i386-mingw32]

C:\test>chcp 437
Active code page: 437

C:\test>ruby test65001.rb
a
Ã

C:\test>chcp 65001
Active code page: 65001

C:\test>ruby test65001.rb
a
├test65001.rb:3: [BUG] rb_sys_fail() - errno == 0
ruby 1.9.3p125 (2012-02-16) [i386-mingw32]

-- Control frame information -----------------------------------------------
c:0006 p:---- s:0014 b:0014 l:000013 d:000013 CFUNC :write
c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC :puts
c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC :puts
c:0003 p:0023 s:0006 b:0006 l:00215c d:0015b4 EVAL test65001.rb:3
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:00215c d:00215c TOP

-- Ruby level backtrace information ----------------------------------------
test65001.rb:3:in <main>'
test65001.rb:3:in
puts'
test65001.rb:3:in puts'
test65001.rb:3:in
write'

-- C level backtrace information -------------------------------------------
C:\Windows\SysWOW64\ntdll.dll(ZwWaitForSingleObject+0x15) [0x77bbf8b1]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObjectEx+0x43) [0x772d1194]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObject+0x12) [0x772d1148]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_bugreport+0xf9) [0x62e5c61d]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_name_err_mesg_new+0x17a) [0x62d3a6fa]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_bug+0x2f) [0x62d3b45b]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_sys_fail+0x163) [0x62d3cfb7]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_close+0x171f) [0x62d651cb]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e54586]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_funcall+0x59) [0x62e55035]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_puts+0x7e) [0x62d5fb7e]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_thread_mark+0x741) [0x62e49ca1]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e54586]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_puts+0x200) [0x62d5fd00]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_thread_mark+0x741) [0x62e49ca1]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_call+0x2b2) [0x62e5855a]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x1f66) [0x62e4ed8a]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x650e) [0x62e53332]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_iseq_eval_main+0x98) [0x62e596f0]
c:\ruby193\bin\msvcrt-ruby191.dll(rb_check_frozen+0x2a89) [0x62d3fed9]
c:\ruby193\bin\msvcrt-ruby191.dll(ruby_run_node+0x48) [0x62d41ff4]
[0x0040136f]
[0x004010b9]
[0x00401284]
C:\Windows\syswow64\kernel32.dll(BaseThreadInitThunk+0x12) [0x772d339a]
C:\Windows\SysWOW64\ntdll.dll(RtlInitializeExceptionChain+0x63) [0x77bd9ef2]

-- Other runtime information -----------------------------------------------

  • Loaded script: test65001.rb

  • Loaded features:

    0 enumerator.so
    1 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/encdb.so
    2 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_1.so
    3 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/transdb.so
    4 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb
    5 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb
    6 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/deprecate.rb
    7 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb
    8 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/defaults/operating_system.rb
    9 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb
    10 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems.rb
    11 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/utf_16le.so
    12 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/utf_16_32.so
    13 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/single_byte.so

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

On ruby 1.9.3p374:

c:\test>ruby --version
ruby 1.9.3p374 (2013-01-15) [i386-mingw32]

c:\test>chcp 437
Active code page: 437

c:\test>ruby test65001.rb
a
Ã

c:\test>chcp 65001
Active code page: 65001

c:\test>ruby test65001.rb
a
├test65001.rb:3: [BUG] rb_sys_fail_str() - errno == 0
ruby 1.9.3p374 (2013-01-15) [i386-mingw32]

-- Control frame information -----------------------------------------------
c:0006 p:---- s:0014 b:0014 l:000013 d:000013 CFUNC :write
c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC :puts
c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC :puts
c:0003 p:0023 s:0006 b:0006 l:001afc d:00059c EVAL test65001.rb:3
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:001afc d:001afc TOP

-- Ruby level backtrace information ----------------------------------------
test65001.rb:3:in <main>'
test65001.rb:3:in
puts'
test65001.rb:3:in puts'
test65001.rb:3:in
write'

-- C level backtrace information -------------------------------------------
C:\Windows\SysWOW64\ntdll.dll(ZwWaitForSingleObject+0x15) [0x77bbf8b1]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObjectEx+0x43) [0x772d1194]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObject+0x12) [0x772d1148]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_bugreport+0xf9) [0x62e5c661]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_name_err_mesg_new+0x17a) [0x62d3a81a]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_bug+0x2f) [0x62d3b4f7]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_sys_fail_str+0x15f) [0x62d3d3a3]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_flush+0x1336) [0x62d64e22]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e5498a]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_funcall+0x59) [0x62e553e9]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_puts+0x78) [0x62d60484]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_mark+0x54d) [0x62e49b85]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e5498a]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_puts+0x248) [0x62d60654]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_mark+0x54d) [0x62e49b85]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_call+0x2b2) [0x62e5882e]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x1f84) [0x62e4f1b0]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x6496) [0x62e536c2]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_iseq_eval_main+0xd7) [0x62e597ef]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_check_frozen+0x2a85) [0x62d4055d]
c:\Ruby193p374\bin\msvcrt-ruby191.dll(ruby_run_node+0x48) [0x62d42688]
[0x0040136f]
[0x004010b9]
[0x00401284]
C:\Windows\syswow64\kernel32.dll(BaseThreadInitThunk+0x12) [0x772d339a]
C:\Windows\SysWOW64\ntdll.dll(RtlInitializeExceptionChain+0x63) [0x77bd9ef2]

-- Other runtime information -----------------------------------------------

  • Loaded script: test65001.rb

  • Loaded features:

    0 enumerator.so
    1 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/encdb.so
    2 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_1.so
    3 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/transdb.so
    4 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb
    5 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb
    6 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/deprecate.rb
    7 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb
    8 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/defaults/operating_system.rb
    9 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb
    10 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems.rb
    11 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/utf_16le.so
    12 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/utf_16_32.so
    13 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/single_byte.so

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Based on the line c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_1.so it makes me think ruby is looking at the registry for the active code page (cp1252) instead of calling GetConsoleOutputCP to see which codepage it should be using, but that is just a guess.

=end


Files

win32-bug-7758.diff (2.54 KB) win32-bug-7758.diff nobu (Nobuyoshi Nakada), 02/15/2013 01:38 PM

Also available in: Atom PDF