Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112023-01-26T20:52:24ZRuby Issue Tracking System
Redmine Ruby master - Bug #19383 (Assigned): Time.now.zone encoding for German display language in Window...https://bugs.ruby-lang.org/issues/193832023-01-26T20:52:24Zstringsn88keys (Thomas Powell)
<p>OS:<br>
Verified on Windows 10 and Windows Server 2022 and Ruby 2.7.7 through 3.1.3</p>
<p>Display language:<br>
Verified on German, but may impact other languages in which Time.now.zone returns characters that aren't [A-Za-z].</p>
<p>Time zone:<br>
CET (UTC +01:00) Amsterdam, Berlin, ...</p>
<p>Time.now.zone # => "Mitteleuro\xE3ische Zeit"<br>
Time.now.zone.encoding # => #<a href="Encoding:IBM437" class="external">Encoding:IBM437</a><br>
puts Time.now.zone # => "Mitteleurop∑ische Zeit" (should be "Mitteleuropäische Zeit")<br>
Time.now.zone.encode(Encoding::UTF_8) # => "Mitteleurop∑ische Zeit"</p>
<p>Doing a force_encoding on all encodings in Encoding.list reveals that ISO-8859-(1..16) and Windows-125(0,2,4,7) work to coerce the ä out of the time zone string:<br>
Time.now.zone.force_encoding(Encoding::WINDOWS_1252) # => "Mitteleuro\xE3ische Zeit"<br>
... but ...<br>
Time.now.zone.force_encoding(Encoding::WINDOWS_1252).encode(Encoding::UTF_8) #=> "Mitteleuropäische Zeit"</p>
<p>Related issue: This improper encoding/rendering caused Ohai's JSON output to be unparseable. Workaround was forcing to Windows-1252.<br>
<a href="https://github.com/chef/ohai/pull/1781" class="external">https://github.com/chef/ohai/pull/1781</a></p> Ruby master - Bug #19378 (Assigned): Windows: Use less syscalls for faster require of big gemshttps://bugs.ruby-lang.org/issues/193782023-01-26T07:02:23Zaidog (Andi Idogawa)andi@idogawa.com
<p>Hello 🙂</p>
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>require is slow on windows for big gems. (example: require 'gtk3'=> 3 seconds+). This is a problem for people who want to make cross platform GUI apps with ruby.</p>
<a name="Possible-Reason"></a>
<h2 >Possible Reason<a href="#Possible-Reason" class="wiki-anchor">¶</a></h2>
<p>As touched on in <a href="https://bugs.ruby-lang.org/issues/15797" class="external">#15797</a> it seems like require uses realpath, which is emulated on windows. It checks every parent directory. The same syscalls run many times.</p>
<a name="Testfile"></a>
<h2 >Testfile<a href="#Testfile" class="wiki-anchor">¶</a></h2>
<p>C:\tmp\speedtest\testrequire.rb:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld1.rb"</span>
<span class="nb">require</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld2.rb"</span>
</code></pre>
<pre><code class="shell syntaxhl" data-language="shell">ruby <span class="nt">--disable-gems</span> C:<span class="se">\t</span>mp<span class="se">\s</span>peedtest<span class="se">\t</span>estrequire.rb
</code></pre>
<a name="Syscalls-per-FileDirectory"></a>
<h3 >Syscalls per File/Directory:<a href="#Syscalls-per-FileDirectory" class="wiki-anchor">¶</a></h3>
<ol>
<li>CreateFile</li>
<li>QueryInformationVolume</li>
<li>QueryIdInformation</li>
<li>QueryAllInformationFile</li>
<li>QueryNameInformationFile</li>
<li>QueryNameInformationFile</li>
<li>QueryNormalizedNameInformationFile</li>
<li>CloseFile</li>
</ol>
<a name="FilesDirectories-checked"></a>
<h3 >Files/Directories checked<a href="#FilesDirectories-checked" class="wiki-anchor">¶</a></h3>
<ol>
<li>C:\tmp</li>
<li>C:\tmp\speedtest</li>
<li>C:\tmp\speedtest\helloworld1.rb</li>
<li>C:\tmp</li>
<li>C:\tmp\speedtest</li>
<li>C:\tmp\speedtest\helloworld2.rb</li>
</ol>
<p>For two required files Ruby had to do 8*6 = <strong>48</strong> syscalls.<br>
The syscalls orginate from rb_w32_reparse_symlink_p / lstat</p>
<p>Rubygems live in subfolders with 9+ parts: "C:\Ruby32-x64\lib\ruby\gems\3.2.0\gems\glib2-4.0.8\lib\glib2\variant.rb"<br>
Each file takes 8 * 9 = <strong>72</strong>+ calls. For variant.rb it is <strong>80</strong> calls.<br>
The result for the syscalls don't change in such a short time, so it should be possible to cache it.</p>
<p>With require_relative it's twice as many calls.</p>
<a name="Other-testcases"></a>
<h2 >Other testcases<a href="#Other-testcases" class="wiki-anchor">¶</a></h2>
<p>Same result:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">File</span><span class="p">.</span><span class="nf">realpath</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld1.rb"</span>
<span class="no">File</span><span class="p">.</span><span class="nf">realpath</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld2.rb"</span>
</code></pre>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">File</span><span class="p">.</span><span class="nf">stat</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld1.rb"</span>
<span class="no">File</span><span class="p">.</span><span class="nf">stat</span> <span class="n">__dir__</span> <span class="o">+</span> <span class="s2">"/helloworld2.rb"</span>
</code></pre>
<p>It does not happen in $LOAD_PATH.resolve_feature_path(<strong>dir</strong> + "/helloworld1.rb")</p>
<a name="Request"></a>
<h2 >Request<a href="#Request" class="wiki-anchor">¶</a></h2>
<p>Would it be possible to cache the stat calls when using require?<br>
I tried to implement a cache inside the ruby source code, but failed.<br>
If not, is there now a way to combine ruby files into one?</p>
<p>I previously talked about require here: <a href="https://bugs.ruby-lang.org/issues/19325#note-11" class="external">YJIT: Windows support lacking.</a></p>
<a name="How-to-reproduce"></a>
<h2 >How to reproduce<a href="#How-to-reproduce" class="wiki-anchor">¶</a></h2>
<p>Ruby versions: At least 3.0+, most likely older ones too.<br>
Tested using Ruby Installer 3.1 and 3.2.<br>
<a href="https://learn.microsoft.com/en-us/sysinternals/downloads/procmon" class="external">Procmon Software by Sysinternals</a></p> Ruby master - Feature #19193 (Assigned): drop DOS TEXT mode supporthttps://bugs.ruby-lang.org/issues/191932022-12-09T16:38:20ZYO4 (Yoshinao Muramatsu)
<p>On Windows platform, <code>File.open(path, "r")</code> returns an object different from "rt" and "rb". I call that DOS TEXT mode here.</p>
<p>DOS TEXT mode does</p>
<ul>
<li>crlf conversion</li>
<li>0x1a treated EOF charactor on read</li>
</ul>
<p>and others (see Bug <a class="issue tracker-1 status-1 priority-4 priority-default" title="Bug: IO has third data mode, document is incomplete. (Open)" href="https://bugs.ruby-lang.org/issues/19192">#19192</a>).<br>
But DOS TEXT mode is almost unnecessary today and it seems to introduce lot of code complexities.</p>
<p>Now there is less need for dos text mode</p>
<ul>
<li>Microsoft's most apps works without CRLF newline.</li>
<li>Creating a crlf text file today should be explicit. (but that is default mode on windows now)</li>
<li>Interpreting EOF charactor can cause trouble.</li>
</ul>
<p>I think it's time to consider dropping DOS TEXT mode.<br>
What challenges are there and what preparation is needed?</p> Ruby master - Bug #14640 (Open): [win32] File.realpath treats a relative path with a drive letter...https://bugs.ruby-lang.org/issues/146402018-03-28T16:18:34Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<p>When <code>t</code> exists in the current directory under the drive C:,</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">File</span><span class="p">.</span><span class="nf">realpath</span><span class="p">(</span><span class="s2">"c:t"</span><span class="p">)</span> <span class="c1">#=> No such file or directory @ realpath_rec - c:/t (Errno::ENOENT)</span>
</code></pre>
<p>whereas <code>File.expand_path</code> returns <code>Dir.pwd + "/t"</code>.</p> Ruby master - Feature #12656 (Assigned): Expand short paths with File.expand_pathhttps://bugs.ruby-lang.org/issues/126562016-08-04T20:06:47Zdavispuh (Dāvis Mosāns)
<p>Currently File.expand_path expands short path only if it's last part.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">puts</span> <span class="no">File</span><span class="p">.</span><span class="nf">expand_path</span><span class="p">(</span><span class="s1">'C:/VERYLO~1'</span><span class="p">)</span>
<span class="nb">puts</span> <span class="no">File</span><span class="p">.</span><span class="nf">expand_path</span><span class="p">(</span><span class="s1">'C:/VERYLO~1/OTHERL~1'</span><span class="p">)</span>
</code></pre>
<p>Produces</p>
<pre><code>C:/VeryLongName12345
C:/VERYLO~1/OtherLongName54321
</code></pre>
<p>With attached patch it will always be long path</p>
<pre><code>C:/VeryLongName12345
C:/VeryLongName12345/OtherLongName54321
</code></pre>
<p>This also fixes TestDir#test_glob test because it was failing due short path.</p> Ruby master - Feature #12653 (Assigned): Use wide WinAPI for rb_w32_getcwdhttps://bugs.ruby-lang.org/issues/126532016-08-03T18:33:23Zdavispuh (Dāvis Mosāns)
<p>Use wide WinAPI for rb_w32_getcwd.<br>
This will be needed so that Dir.pwd can support Unicode current directory on Windows.</p>
<p>I've attached a patch.</p> Ruby master - Bug #12040 (Assigned): [Win32] File.stat fails on a mounted volumehttps://bugs.ruby-lang.org/issues/120402016-02-01T08:13:24Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<p>On Windows, <code>File.stat</code> fails on the volume mount point directory whose name contains <code>"..."</code>.</p>
<p>Where <code>%vol%</code> is the volume ID of a new VHD volume,</p>
<pre><code>C:> set vol
\\?\Volume{3C458AE9-C8B1-11E5-A233-0800271D089F}\
C:> mkdir x...y
C:> mountvol x...y %vol%
C:> .\miniruby -e "p Dir.chdir('x...y'){File.stat('.')}" -e "p File.stat('x...y')"
#<File::Stat dev=0x2, ino=1407374883553285, mode=040755, nlink=1, uid=0, gid=0, rdev=0x2, size=4096, blksize=nil, blocks=nil, atime=2016-02-01 16:35:45 +0900, mtime=2016-02-01 16:35:45 +0900, ctime=2016-02-01 16:35:45 +0900>
-e:2:in `stat': No such file or directory @ rb_file_s_stat - x...y (Errno::ENOENT)
from -e:2:in `<main>'
</code></pre>
<p>Note that <code>Dir.chdir</code> and <code>File.stat</code> there succeed.<br>
This failures depends on the mount point name, because of <code>check_valid_dir()</code>.</p> Ruby master - Bug #11438 (Open): native_thread_init_stack() get machine.stack_start unequal to th...https://bugs.ruby-lang.org/issues/114382015-08-13T07:31:10Zrickerliang (l ly)rickerliang@outlook.com
<p>In function native_thread_init_stack() use VirtualQuery to get thread's stack start address.But some situation(ruby embbed in other application and initial it on the fly),native_thread_init_stack() will be called at low stack address and VirtualQuery return memory info BaseAddress + RegionSize < thread stack base(teb.StackBase).<br>
In this situation,subsequently call stack_check() at high stack address will cause stack_overflow exception,because esp > machine.stack_start:<br>
(teb.StackLimit < machine.stack_start < esp < teb.StackBase)<br>
but actually it is not stack overflow at this time.<br>
Use teb.StackBase instead of VirtualQuery get thread stack base is a more reliable solution.</p> Ruby master - Bug #11142 (Open): Command line argument parser on windows handles double quotes in...https://bugs.ruby-lang.org/issues/111422015-05-12T16:09:28Zksubrama (Kartik Cating-Subramanian)ksubramanian@chef.io
<p>I believe the issue is with <a href="https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L1671" class="external">https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L1671</a> through 1673.</p>
<p>C:\Users\ksubrama>ruby -e "puts ARGV" "foo""bar"<br>
foo"bar</p>
<p>C:\Users\ksubrama>ruby -e "puts ARGV" "foo"" bar"<br>
foo"<br>
bar</p>
<p>I believe the intent is that if ruby encounters "" inside a " quoted string, then it interprets it as a literal " and doesn't close out the string. If that's the case, then the code should read:</p>
<pre><code> if (quote == L'"' && quote == ptr[1])
ptr++;
else
quote = L'\0';
</code></pre>
<p>Otherwise, the string gets closed out anyway and the ptr++ here combined with the ptr++ at the bottom of the switch at line 1685 simply skip over both "" characters while considering the string closed.</p>
<p>As a further test case consider:</p>
<p>C:\Users\ksubrama>ruby -e "puts ARGV" "foo""bar""baz"<br>
foo"barbaz</p>
<p>The parser is now very confused because the first "" closed out the string and the next "" is not interpreted as a literal " but as "open and close and empty string element" and the trailing " just gets dropped.</p> Ruby master - Bug #10128 (Open): Quoting problem for arguments of Kernel.system, Kernel.exec on W...https://bugs.ruby-lang.org/issues/101282014-08-12T16:18:30ZMaxLap (Maxime Lapointe)hunter_spawn@hotmail.com
<p>On Windows, the methods that call shell commands and receive the parameters individually sometimes do not wrap the parameters sent in quotes.<br>
This results in Windows either splitting the parameter in 2 parameters or, worse, splitting the command in 2 commands.</p>
<p>I joined the file <em>puts_first.bat</em>, which simply outputs the first argument it received. When the parameter received was wrapped by ruby, you will see quotes, it's normal. Just run a irb from from the directory containing that file.</p>
<p>Lines that don't work properly (Using Kernel.exec will do the same thing):</p>
<pre><code># these write *hello*, then says 'world' is not recognized as an internal or external command
Kernel.system 'puts_first.bat', 'hello&world'
Kernel.system 'puts_first.bat', 'hello|world'
# these write *hello*
Kernel.system 'puts_first.bat', 'hello,world'
Kernel.system 'puts_first.bat', 'hello;world'
Kernel.system 'puts_first.bat', 'hello<world'
# this writes *hello* in the file world
Kernel.system 'puts_first.bat', 'hello>world'
# this writes *helloworld* without the ^
Kernel.system 'puts_first.bat', 'hello^world'
</code></pre>
<p>If we add a space anywhere in the above hello world strings, it will work as expected because ruby wraps the parameter if it finds a space.</p>
<pre><code># Ruby does try to wrap if it finds a double quote, but it escapes double quotes incorrectly:
# this writes *"hello\"world"*, double quotes should be escaped by putting 2 of them, so we should see: *"hello""world"*
Kernel.system 'puts_first.bat', 'hello"world'
# adding a space show the problem in action, this writes *"hello\"*
Kernel.system 'puts_first.bat', 'hello" world'
</code></pre>
<p>As a side note, the single quote is not special in Windows, so there is no need to wrap this (but I don't think it's a problem):</p>
<pre><code># this writes *"hello'world"*
Kernel.system 'puts_first.bat', "hello'world"
</code></pre>
<p>This bug also happens in 1.9.3, do you think this be backported?</p>
<p>Unless I did a mistake, this should be all of the problematic characters, I tested with every printable ascii characters.</p>
<p>Thank you</p> Ruby master - Feature #8083 (Assigned): Exit status is limited to one-byte values which is invali...https://bugs.ruby-lang.org/issues/80832013-03-13T19:00:42Zrutsky (Vladimir Rutsky)rutsky.vladimir@gmail.com
<p>=begin<br>
Windows uses 32-bit process exit codes so Ruby incorrectly truncates them to one byte:</p>
<p>C:\Ruby193\bin>ruby -e "system('C:\windows\system32\cmd.exe /c exit 255'); puts $?.exitstatus"<br>
255</p>
<p>C:\Ruby193\bin>ruby -e "system('C:\windows\system32\cmd.exe /c exit 256'); puts $?.exitstatus"<br>
0</p>
<p>C:\Ruby193\bin>ruby -e "system('C:\windows\system32\cmd.exe /c exit 257'); puts $?.exitstatus"<br>
1</p>
<p>Similar code works correctly in Python:</p>
<p>C:\Python27>python -c "import subprocess; print subprocess.call('C:\windows\system32\cmd.exe /c exit 255')"<br>
255</p>
<p>C:\Python27>python -c "import subprocess; print subprocess.call('C:\windows\system32\cmd.exe /c exit 256')"<br>
256</p>
<p>C:\Python27>python -c "import subprocess; print subprocess.call('C:\windows\system32\cmd.exe /c exit 257')"<br>
257<br>
=end</p> Ruby master - Bug #7968 (Assigned): Poor UDPSocket#send performance in ruby 2.0.0 on windowshttps://bugs.ruby-lang.org/issues/79682013-02-26T20:55:25Zcs96and (Alan Davies)alan.n.davies@gmail.com
<p>I have noticed that the performance of UDPSocket#send on ruby 2.0.0 on windows is much poorer than that of 1.9.3 or 1.8.7. Running the attahced script on 2.0.0 gives the following...</p>
<p>d:\scripts>bash -c "ruby --version"<br>
ruby 2.0.0p0 (2013-02-24) [x64-mingw32]</p>
<p>d:\scripts>bash -c "time ruby socketsendtest.rb"</p>
<p>real 0m2.572s<br>
user 0m0.000s<br>
sys 0m0.016s</p>
<p>However, running the same test with 1.9.3 is much faster...</p>
<p>d:\scripts>pik 193</p>
<p>d:\scripts>bash -c "ruby --version"<br>
ruby 1.9.3p374 (2013-01-15) [i386-mingw32]</p>
<p>d:\scripts>bash -c "time ruby socketsendtest.rb"</p>
<p>real 0m0.993s<br>
user 0m0.015s<br>
sys 0m0.016s</p>
<p>Additionally, if I change the send call to a print (commented out in the script), then the performance is fine on 2.0.0....</p>
<p>d:\scripts>pik 200</p>
<p>d:\scripts>bash -c "ruby --version"<br>
ruby 2.0.0p0 (2013-02-24) [x64-mingw32]</p>
<p>d:\scripts>bash -c "time ruby socketsendtest.rb"</p>
<p>real 0m0.907s<br>
user 0m0.000s<br>
sys 0m0.015s</p>
<p>What is send() doing that print() doesn't do that is causing the massive performance drop?</p>
<p>Thanks<br>
Alan.</p> Ruby master - Feature #1644 (Assigned): recv on inherited socket wrapped in TCPSocket does not re...https://bugs.ruby-lang.org/issues/16442009-06-17T14:37:54Zkntuaf (Kedar H)knutaf@gmail.com
<p>=begin<br>
On Windows, if you try to wrap an existing winsock socket that was inherited from a parent process into a TCPSocket, any calls to recv on this socket in the child process will not retrieve any data.</p>
<p>This is because the inherited socket is not a member of the internal "socklist" structure in win32.c. rb_w32_select filters out all sockets that are not in this internal list, prior to calling ws2_32!select.</p>
<p>One good solution is for TCPSocket.for_fd to make sure to insert its argument into the socklist structure so that subsequent Winsock functions wrapped by Ruby will work as though the socket were created by this Ruby process. Another possible solution is to expose another method, something like TCPSocket.from_new_fd.<br>
=end</p>