Bug #20699: On Windows, the `__dir__` keyword is garbled in paths containing Japanese characters, and `require_relative` fails as well

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x64-mingw-ucrt]
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
• Tags: win

[flatland001 (Hayato Arai)]

In paths containing Japanese characters, such as:
C:\テスト_test\test.rb

Code:

#     this code -> C:\テスト_test\test.rb
# external file -> C:\テスト_test\lib\foo.rb
p __dir__
p __dir__.encoding
p File.dirname(File.expand_path(__FILE__))
p File.dirname(File.expand_path(__FILE__)).encoding
print "__dir__ == File.dirname(File.expand_path(__FILE__)): "
p __dir__ == File.dirname(File.expand_path(__FILE__))

begin
  require_relative "lib/foo"
  puts "foo.rb loaded"
rescue LoadError
  puts "LoadError: #{$!}"
end

Results:

"C:/?e?X?g_test"
#<Encoding:Windows-31J>
"C:/\x{8365}\x{8358}\x{8367}_test"
#<Encoding:Windows-31J>
__dir__ == File.dirname(File.expand_path(__FILE__)): false
LoadError: cannot load such file -- C:/?e?X?g_test/lib/foo

However, in ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x64-mingw-ucrt], the same code produces the following expected results:

"C:/\x{8365}\x{8358}\x{8367}_test"
#<Encoding:Windows-31J>
"C:/\x{8365}\x{8358}\x{8367}_test"
#<Encoding:Windows-31J>
__dir__ == File.dirname(File.expand_path(__FILE__)): true
foo.rb loaded

Is this behavior expected, or could this be a bug in the Ruby interpreter on Windows? If this is my misunderstanding or a known issue, I apologize. Thank you for your consideration.

Environment Information:

• OS: Windows 11 Pro 23H2 (OS build 22631.4037)
• Installed via: RubyInstaller
• Shell: Command Prompt
• File System: NTFS
• System Locale: Japanese (Japan), Language: Japanese

[nobu (Nobuyoshi Nakada)]

This may be fixed already on the master?

[larskanis (Lars Kanis)]

I verified that this issue is still present on current master. On codepage 850 the output looks like so:

C:\Users\kanis\ruby\テスト_test>ruby.exe test.rb
"C:/Users/kanis/ruby/???_test"
#<Encoding:CP850>
"C:/Users/kanis/ruby/???_test"
#<Encoding:CP850>
__dir__ == File.dirname(File.expand_path(__FILE__)): true
LoadError: cannot load such file -- C:/Users/kanis/ruby/???_test/lib/foo

It is fixed by https://github.com/ruby/ruby/pull/12377 .

[hsbt (Hiroshi SHIBATA)]

• Tags set to windows

[hsbt (Hiroshi SHIBATA)]

• Tags changed from windows to win