Bug #8676

ruby 2.0 can not require or load the source file with non-ascii path name

Added by 贾 延平 9 months ago. Updated 8 months ago.

[ruby-core:56136]
Status:Closed
Priority:Normal
Assignee:-
Category:platform/windows
Target version:-
ruby -v:ruby 2.0.0p277 (2013-07-23 revision 42121) [i386-mingw32] Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN

Description

=begin
Sorry for my poor english:)

I attached the patch to fix the problem, but I don't know is it the right way.

Changelog:
include/ruby/intern.h change the declaration of "rbloadfile";change parameter from char to VALUE
*load.c change the caller
*ruby.c change the "rbloadfile"'s implement
*win32/file.c change the win32 api call from ANSI type to UNICODE type.
=end

win32_file_open_patch.patch Magnifier - patch file (2.3 KB) 贾 延平, 07/24/2013 11:01 AM

win32_file_open_patch.patch Magnifier (4.96 KB) 贾 延平, 07/26/2013 09:37 AM

Associated revisions

Revision 42183
Added by Nobuyoshi Nakada 9 months ago

load.c: search in OS path encoding

  • load.c (rbloadinternal): use rbloadfile_str() to keep path encoding.
  • load.c (rbrequiresafe): search in OS path encoding for Windows.
  • ruby.c (rbloadfile_str): load file with keeping path encoding.
  • win32/file.c (rbfileload_ok): use WCHAR type API assuming incoming path is encoded in UTF-8. [Bug #8676]

History

#1 Updated by Charlie Somerville 9 months ago

This patch looks good to me for trunk. It can't be backported though because that would break API compatibility.

#2 Updated by Nobuyoshi Nakada 9 months ago

  • Subject changed from ruby 2.0 can not require or load the source file with utf-8 encoding and non-asii chars to ruby 2.0 can not require or load the source file with non-ascii path name
  • Description updated (diff)
  • Category changed from platform/mingw to platform/windows

#3 Updated by 贾 延平 9 months ago

By the way,the file nacl/pepper_main.c has a same name function at line 824.
Is the file need change too?

#4 Updated by 贾 延平 9 months ago

Update the patch
Using the encoding from path name

#5 Updated by Nobuyoshi Nakada 9 months ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r42183.
贾, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


load.c: search in OS path encoding

  • load.c (rbloadinternal): use rbloadfile_str() to keep path encoding.
  • load.c (rbrequiresafe): search in OS path encoding for Windows.
  • ruby.c (rbloadfile_str): load file with keeping path encoding.
  • win32/file.c (rbfileload_ok): use WCHAR type API assuming incoming path is encoded in UTF-8. [Bug #8676]

#6 Updated by 贾 延平 9 months ago

There is a new bug from the change.
FILE's encoding is wrong.
In file ruby.c and function loadfileinternal type VALUE convert to char, and lost the encoding info
(({const char *origfname = StringValueCStr(argp->fname);}))
and call the *rb
parsercompilefile
with the char* variable
(({ tree = rbparsercompilefile(parser, origfname, f, linestart);}))
and in file parse.y and function *rb
parsercompilefile, type char convert to VALUE using the filesystem encoding.
(({ return rbparsercompilefilepath(vparser, rbfilesystemstrnewcstr(f), file, start);}))
But the beginning parameter argp->fname's encoding not the filesystem encoding.

What's the principle of ruby's encoding?Why so many VALUE to char* converting?Is the char* has the regular encoding?

I think we should convert the encoding at the boundary and every thing should encoding to internal encoding in the internal, am I right?

#7 Updated by Thomas Thomassen 8 months ago

This patch makes the file functions explicitly call the *W version of the file functions. Isn't it better to provide the UNICODE compile flag. http://msdn.microsoft.com/en-us/library/cc194801.aspx

CreateFile, and the likes, would then call CreateFileW instead of CreateFileA.

Also available in: Atom PDF