Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-09-04T23:06:54ZRuby Issue Tracking System
Redmine Ruby master - Bug #18995 (Open): IO#set_encoding sometimes set an IO's internal encoding to the d...https://bugs.ruby-lang.org/issues/189952022-09-04T23:06:54Zjavanthropus (Jeremy Bopp)jeremy@bopp.net
<p>This script demonstrates the behavior:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">show</span><span class="p">(</span><span class="n">io</span><span class="p">)</span>
<span class="nb">printf</span><span class="p">(</span>
<span class="s2">"external encoding: %-25p internal encoding: %-25p</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span>
<span class="n">io</span><span class="p">.</span><span class="nf">external_encoding</span><span class="p">,</span>
<span class="n">io</span><span class="p">.</span><span class="nf">internal_encoding</span>
<span class="p">)</span>
<span class="k">end</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_external</span> <span class="o">=</span> <span class="s1">'iso-8859-1'</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_internal</span> <span class="o">=</span> <span class="s1">'iso-8859-2'</span>
<span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="s1">'/dev/null'</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">,</span> <span class="kp">nil</span><span class="p">)</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="c1"># f.internal_encoding is iso-8859-2, as expected</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">,</span> <span class="s1">'invalid'</span><span class="p">)</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="c1"># f.internal_encoding is now iso-8859-1!</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_external</span> <span class="o">=</span> <span class="s1">'iso-8859-3'</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_internal</span> <span class="o">=</span> <span class="s1">'iso-8859-4'</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="c1"># f.internal_encoding is now iso-8859-3!</span>
<span class="k">end</span>
</code></pre>
<p>In the 1st case, we see that the IO's internal encoding is set to the current setting of Encoding.default_internal. In the 2nd case, the IO's internal encoding is set to Encoding.default_external instead. The 3rd case is more interesting because it shows that the IO's internal encoding is actually following the current setting of Encoding.default_external. It didn't just copy it when #set_encoding was called. It changes whenever Encoding.default_external changes.</p>
<p>What should the correct behavior be?</p> Ruby master - Bug #18899 (Closed): Inconsistent argument handling in IO#set_encodinghttps://bugs.ruby-lang.org/issues/188992022-07-06T12:34:32Zjavanthropus (Jeremy Bopp)jeremy@bopp.net
<p><code>IO#set_encoding</code> behaves differently when processing a single String argument than it does when processing 2 arguments (whether Strings or Encodings) in the case where the external encoding is being set to binary and the internal encoding is being set to any other encoding.</p>
<p>This script demonstrates the resulting values of the external and internal encodings for an IO instance given different ways to equivalently call <code>#set_encoding</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1">#!/usr/bin/env ruby</span>
<span class="k">def</span> <span class="nf">show</span><span class="p">(</span><span class="n">io</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="nb">printf</span><span class="p">(</span>
<span class="s2">"args: %-50s external encoding: %-25s internal encoding: %-25s</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span>
<span class="n">args</span><span class="p">.</span><span class="nf">inspect</span><span class="p">,</span>
<span class="n">io</span><span class="p">.</span><span class="nf">external_encoding</span><span class="p">.</span><span class="nf">inspect</span><span class="p">,</span>
<span class="n">io</span><span class="p">.</span><span class="nf">internal_encoding</span><span class="p">.</span><span class="nf">inspect</span>
<span class="p">)</span>
<span class="k">end</span>
<span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="s1">'/dev/null'</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="n">args</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'binary:utf-8'</span><span class="p">]</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="n">args</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'binary'</span><span class="p">,</span> <span class="s1">'utf-8'</span><span class="p">]</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="n">args</span> <span class="o">=</span> <span class="p">[</span><span class="no">Encoding</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="s1">'binary'</span><span class="p">),</span> <span class="no">Encoding</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)]</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>This behavior is the same from Ruby 2.7.0 to 3.1.2.</p> Ruby master - Bug #18898 (Closed): IO#set_encoding with invalid arguments leads to a segfaulthttps://bugs.ruby-lang.org/issues/188982022-07-06T02:30:09Zjavanthropus (Jeremy Bopp)jeremy@bopp.net
<p>Save the following to a file and run it:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1">#!/usr/bin/env ruby</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_external</span> <span class="o">=</span> <span class="s1">'utf-8'</span>
<span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="kp">__FILE__</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="n">f</span><span class="p">.</span><span class="nf">set_encoding</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">,</span> <span class="s1">'invalid'</span><span class="p">)</span>
<span class="nb">printf</span><span class="p">(</span>
<span class="s2">"default external: %p</span><span class="se">\n</span><span class="s2">default internal: %p</span><span class="se">\n</span><span class="s2">external: %p</span><span class="se">\n</span><span class="s2">internal: %p</span><span class="se">\n\n</span><span class="s2">"</span><span class="p">,</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_external</span><span class="p">,</span>
<span class="no">Encoding</span><span class="p">.</span><span class="nf">default_internal</span><span class="p">,</span>
<span class="n">f</span><span class="p">.</span><span class="nf">external_encoding</span><span class="p">,</span>
<span class="n">f</span><span class="p">.</span><span class="nf">internal_encoding</span>
<span class="p">)</span>
<span class="n">f</span><span class="p">.</span><span class="nf">read</span>
<span class="k">end</span>
</code></pre>
<p>The above script will result in a segfault at <code>f.read</code>. This seems to happen because the call to <code>#set_encoding</code> results in the internal encoding of the IO object being set to follow <code>Encoding.default_external</code> while also setting the external encoding of the IO object to match. Ovbiously, there shouldn't be a segfault, but I actually expected the IO object's internal encoding to be set to nil due to the invalid encoding being specified for it.</p>
<p>I was able to reproduce this on all versions of Ruby from 2.7.0 to 3.0.2.</p> Ruby master - Bug #18881 (Rejected): IO#read_nonblock raises IOError when called following buffer...https://bugs.ruby-lang.org/issues/188812022-06-26T16:35:07Zjavanthropus (Jeremy Bopp)jeremy@bopp.net
<p>The following example code works as expected on Linux but raises an <code>IOError</code> on Windows:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">r</span><span class="p">,</span> <span class="n">w</span> <span class="o">=</span> <span class="no">IO</span><span class="p">.</span><span class="nf">pipe</span>
<span class="n">w</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="s2">"foobar"</span><span class="p">)</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">r</span><span class="p">.</span><span class="nf">getc</span>
<span class="n">r</span><span class="p">.</span><span class="nf">ungetc</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
<span class="n">r</span><span class="p">.</span><span class="nf">read_nonblock</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
</code></pre>
<p>The error message is:</p>
<pre><code>IOError: byte oriented read for character buffered IO
</code></pre> Ruby master - Bug #18880 (Closed): IO#sysread on Windows does not validate argumentshttps://bugs.ruby-lang.org/issues/188802022-06-26T16:26:46Zjavanthropus (Jeremy Bopp)jeremy@bopp.net
<p>When passing an invalid number of bytes to read, such as <code>-1</code> , to <code>IO#sysread</code> on Linux, an <code>ArgumentError</code> is raised. On Windows <code>Errno::EINVAL</code> is raised instead.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">r</span><span class="p">,</span> <span class="n">w</span> <span class="o">=</span> <span class="no">IO</span><span class="p">.</span><span class="nf">pipe</span>
<span class="n">r</span><span class="p">.</span><span class="nf">sysread</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
</code></pre>
<p>This raises <code>ArgumentError</code> on non-Windows and <code>Errno::EINVAL</code> on Windows.</p>