Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112021-05-21T04:38:44ZRuby Issue Tracking System
Redmine Ruby master - Bug #17877 (Closed): Backport a fix of RDoc for CVE-2021-31799https://bugs.ruby-lang.org/issues/178772021-05-21T04:38:44Zaycabta (aycabta .)aycabta@gmail.com
<p><a href="https://github.com/ruby/rdoc/commit/a7f5d6ab88632b3b482fe10611382ff73d14eed7.diff" class="external">https://github.com/ruby/rdoc/commit/a7f5d6ab88632b3b482fe10611382ff73d14eed7.diff</a></p> Ruby master - Feature #15483 (Rejected): Proc or Method combination with Symbolhttps://bugs.ruby-lang.org/issues/154832018-12-29T10:40:18Zaycabta (aycabta .)aycabta@gmail.com
<p>In [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add composition for procs (Closed)" href="https://bugs.ruby-lang.org/issues/6284">#6284</a>], Matz said</p>
<blockquote>
<p>We need more discussion if we would add combination methods to the Symbol class.</p>
</blockquote>
<p>Right, let's get started to discuss.</p>
<p>For your information, recent a few months I'm discussing this with <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/13476">@osyo (manga osyo)</a> .</p>
<a name="This-is-a-discussion-of-design"></a>
<h2 >This is a discussion of "design"<a href="#This-is-a-discussion-of-design" class="wiki-anchor">¶</a></h2>
<p>I understand that all features of this issue have both merits and demerits, but I guess that language design is most important. All features of this issue related to each other.</p>
<a name="Abstract"></a>
<h2 >Abstract<a href="#Abstract" class="wiki-anchor">¶</a></h2>
<p>At present, you can use <code>Proc#>></code> or <code>Proc#<<</code> with <code>Symbol#to_proc</code>.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_i</span><span class="p">.</span><span class="nf">to_proc</span> <span class="o">>></span> <span class="ss">:chr</span><span class="p">.</span><span class="nf">to_proc</span><span class="p">))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre>
<p>This is convenient but methods that take block can take a proc with <code>&</code> syntax sugar instead of <code>#to_proc</code> by right, like <code>[1, 2, 3].map(&:to_s)</code>. So <code>Symbol#to_proc</code> looks like too long for <code>Proc#>></code> or <code>Proc#<<</code>. Therefore, you need new syntax sugar.</p>
<a name="Receiver"></a>
<h2 >Receiver<a href="#Receiver" class="wiki-anchor">¶</a></h2>
<h3>
<code>Symbol#>></code> and <code>Symbol#<<</code>
</h3>
<p><code>Symbol#>></code> and <code>Symbol#<<</code> will be considered, but this means that <code>Symbol</code> is treated as <code>Proc</code> partially. The <code>[1, 2, 3].map(&:to_s)</code> treats <code>Symbol</code> as <code>Proc</code> partially too, but it's with pre-positioned <code>&</code>.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_i</span> <span class="o">>></span> <span class="ss">:chr</span><span class="p">.</span><span class="nf">to_proc</span><span class="p">))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre>
<p>I can't come up with other ideas for the <code>Symbol</code> receiver.</p>
<h3>New <code>&:symbol_name</code> syntax sugar for <code>:symbol_name.to_proc</code>
</h3>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_i</span> <span class="o">>></span> <span class="ss">:chr</span><span class="p">.</span><span class="nf">to_proc</span><span class="p">)))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre>
<a name="Argument"></a>
<h2 >Argument<a href="#Argument" class="wiki-anchor">¶</a></h2>
<a name="Calls-to_proc-by-Procgtgt-or-Procltlt-internally-as-a-duck-typing"></a>
<h3 >Calls <code>#to_proc</code> by <code>Proc#>></code> or <code>Proc#<<</code> internally as a duck typing<a href="#Calls-to_proc-by-Procgtgt-or-Procltlt-internally-as-a-duck-typing" class="wiki-anchor">¶</a></h3>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_i</span><span class="p">.</span><span class="nf">to_proc</span> <span class="o">>></span> <span class="ss">:chr</span><span class="p">))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre>
<p>In this case, <code>Proc#>></code>(<code>:to_i.to_proc >></code>) calls <code>Symbol#to_proc</code>(for <code>:chr</code>) inside.</p>
<p>This is useful to use with <code>Hash#to_proc</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">h</span> <span class="o">=</span> <span class="p">{</span> <span class="no">Alice</span><span class="p">:</span> <span class="mi">30</span><span class="p">,</span> <span class="no">Bob</span><span class="p">:</span> <span class="mi">60</span><span class="p">,</span> <span class="no">Cris</span><span class="p">:</span> <span class="mi">90</span> <span class="p">}</span>
<span class="sx">%w{Alice Bob Cris}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_sym</span><span class="p">.</span><span class="nf">to_proc</span> <span class="o">>></span> <span class="n">h</span><span class="p">))</span>
<span class="c1"># => [30, 60, 90]</span>
</code></pre>
<h3>
<code>Proc#>></code> and <code>Proc#<<</code> take block as an argument</h3>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_i</span><span class="p">.</span><span class="nf">to_proc</span> <span class="o">>></span> <span class="o">&</span><span class="ss">:chr</span><span class="p">))</span>
</code></pre>
<a name="Combination-of-receiver-and-argument"></a>
<h2 >Combination of receiver and argument<a href="#Combination-of-receiver-and-argument" class="wiki-anchor">¶</a></h2>
<p><code>Symbol#>></code> and calling <code>#to_proc</code> internally:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="ss">:to_i</span> <span class="o">>></span> <span class="ss">:chr</span><span class="p">))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre>
<p><code>&:symbol_name</code> syntax sugar for <code>:symbol_name.to_proc</code> and <code>Symbol#>></code> and taking block:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sx">%w{72 101 108 108 111}</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_i</span> <span class="o">>></span> <span class="o">&</span><span class="ss">:chr</span><span class="p">))</span>
<span class="c1"># => ["H", "e", "l", "l", "o"]</span>
</code></pre> Ruby master - Feature #14919 (Open): Add String#byteinserthttps://bugs.ruby-lang.org/issues/149192018-07-18T05:18:26Zaycabta (aycabta .)aycabta@gmail.com
<p>It's important for multibyte String editing. Unicode grapheme characters sometimes have plural code points. In text editing, software sometimes should add a new code point to an existing grapheme character. String#byteinsert is important for it.</p>
<p>I implemented by pure Ruby in my code.<br>
<a href="https://github.com/aycabta/reline/blob/b17e5fd61092adfd7e87d576301e4e19a4d9e6d8/lib/reline/line_editor.rb#L255-L260" class="external">https://github.com/aycabta/reline/blob/b17e5fd61092adfd7e87d576301e4e19a4d9e6d8/lib/reline/line_editor.rb#L255-L260</a></p> Ruby master - Bug #14918 (Closed): Use Reline for fallback of ext/readlinehttps://bugs.ruby-lang.org/issues/149182018-07-18T04:48:25Zaycabta (aycabta .)aycabta@gmail.com
<p>Ruby loses readline standard library when the system doesn't have libreadline during Ruby installation, At <a href="https://bugs.ruby-lang.org/issues/11084" class="external">https://bugs.ruby-lang.org/issues/11084</a>, <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/572">@hsbt (Hiroshi SHIBATA)</a> proposed pure Ruby implementation. But I think that rb-readline is not good for Ruby standard library and I described it at <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/86213" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/86213</a>. The important reason for it personally is that rb-readline doesn't have vi mode.</p>
<p>So I'm implementing Reline for a new standard library as GNU Readline and existing ext/readline compatible. I can maintain it.<br>
<a href="https://github.com/aycabta/reline" class="external">https://github.com/aycabta/reline</a></p>
<p>I don't think that Reline should replace ext/readline fully, it's just for fallback. But it's important for Ruby.</p> Ruby master - Misc #14917 (Assigned): Add RDoc documents to tar ballhttps://bugs.ruby-lang.org/issues/149172018-07-18T04:08:50Zaycabta (aycabta .)aycabta@gmail.com
<p>I guess that distribution packages should include RDoc documents' files because RDoc documents installation step needs too long time.</p>
<p>There is the other reason. RDoc sometimes fails during "generating RDoc documentation" phase of installation.</p>
<p>Some case examples:</p>
<ul>
<li><a href="https://bugs.ruby-lang.org/issues/11494" class="external">https://bugs.ruby-lang.org/issues/11494</a></li>
<li><a href="https://bugs.ruby-lang.org/issues/14663" class="external">https://bugs.ruby-lang.org/issues/14663</a></li>
<li><a href="https://bugs.ruby-lang.org/issues/14874" class="external">https://bugs.ruby-lang.org/issues/14874</a></li>
<li><a href="https://bugs.ruby-lang.org/issues/11514" class="external">https://bugs.ruby-lang.org/issues/11514</a></li>
</ul>
<p>Maybe, generating RDoc documentation is so heavy task for low memory situation, and other tasks are lighter than it.</p> Ruby master - Feature #14808 (Rejected): Last token of endless range should have EXPR_ENDhttps://bugs.ruby-lang.org/issues/148082018-06-02T01:23:16Zaycabta (aycabta .)aycabta@gmail.com
<p>In 2.5.1:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">></span> <span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">></span> <span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="s2">"case 5</span><span class="se">\n</span><span class="s2">when 3..</span><span class="se">\n</span><span class="s2"> puts(true)</span><span class="se">\n</span><span class="s2">end</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span>
<span class="p">[[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"case"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"5"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="o">|</span><span class="no">EXPR_ENDARG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"when"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"3"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="o">|</span><span class="no">EXPR_ENDARG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_op</span><span class="p">,</span> <span class="s2">".."</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">8</span><span class="p">],</span> <span class="ss">:on_ignored_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"puts"</span><span class="p">,</span> <span class="no">EXPR_ARG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_lparen</span><span class="p">,</span> <span class="s2">"("</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="o">|</span><span class="no">EXPR_LABEL</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"true"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span> <span class="ss">:on_rparen</span><span class="p">,</span> <span class="s2">")"</span><span class="p">,</span> <span class="no">EXPR_ENDFN</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">12</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"end"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">]]</span>
</code></pre>
<p>This is invalid code in 2.5.1, so I understand this result.</p>
<p>In 63451:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">></span> <span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">></span> <span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="s2">"case 5</span><span class="se">\n</span><span class="s2">when 3..</span><span class="se">\n</span><span class="s2"> puts(true)</span><span class="se">\n</span><span class="s2">end</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span>
<span class="p">[[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"case"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"5"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"when"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"3"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_op</span><span class="p">,</span> <span class="s2">".."</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">8</span><span class="p">],</span> <span class="ss">:on_ignored_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"puts"</span><span class="p">,</span> <span class="no">EXPR_ARG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_lparen</span><span class="p">,</span> <span class="s2">"("</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="o">|</span><span class="no">EXPR_LABEL</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"true"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span> <span class="ss">:on_rparen</span><span class="p">,</span> <span class="s2">")"</span><span class="p">,</span> <span class="no">EXPR_ENDFN</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">12</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"end"</span><span class="p">,</span> <span class="no">EXPR_END</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="no">EXPR_BEG</span><span class="p">]]</span>
</code></pre>
<p>This is correct code in this revision.</p>
<p>I think that lex_state of the last token of endless range, <code>[[2, 6], :on_op, "..", EXPR_BEG]</code>, it should be EXPR_END. Because it's the end of an argument. It's important for REPL, RDoc, and so on.</p>
<p>But lex_state is parser matter in parse.y. How about this for the parser of Ruby compiler?</p> Ruby master - Feature #14787 (Closed): Show documents when completionhttps://bugs.ruby-lang.org/issues/147872018-05-25T22:44:23Zaycabta (aycabta .)aycabta@gmail.com
<p>The "irb/completion" provides completion for classes, modules, symbols, and methods. This patch provides a new feature for it that shows documents when press TAB key one more after exact matches. This uses RDoc as a library.</p>
<p>Example:</p>
<pre><code>$ ruby -Ilib bin/irb
irb(main):001:0> "".gsub # press TAB key twice
"".gsub "".gsub!
irb(main):001:0> "".gsub # press TAB key one more
# show RI document and press q to quit
irb(main):001:0> "".gsub
</code></pre>
<p>I'm the RDoc maintainer. A lot of Ruby users tell me "I'm using <em>Google</em> for reading documents of Ruby".</p>
<p>RDoc installs all documents to Ruby's directory by default. Many users never use it because it's just for RI("ri" command). I think that it is a reason of that many users don't attach importance to documentation.</p>
<p>Perl has "perldoc" feature and users easily access documents of modules by "perldoc" command. Python has "docstring" feature and users can access it on REPL. Those are each language's design of importance. Users use the language on the documentation design, so library developers write documents on the documentation design. Ruby doesn't have documentation design like Perl and Python. Ruby just has RDoc, IRB, and any other supports, but these are just fragmented features, these are not a documentation design.</p>
<p>Ruby needs many new features of documentation. This is one of them. This new feature gives new development environment.</p> Ruby master - Feature #14683 (Closed): IRB with Ripperhttps://bugs.ruby-lang.org/issues/146832018-04-12T15:06:07Zaycabta (aycabta .)aycabta@gmail.com
<p>I replaced lexical analyzer with Ripper. It's important for supporting new syntax.</p>
<p>I explain what I did.</p>
<a name="Replace-with-Ripper"></a>
<h2 >Replace with Ripper<a href="#Replace-with-Ripper" class="wiki-anchor">¶</a></h2>
<p>I talked with <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/13">@matz (Yukihiro Matsumoto)</a> about it and I decided to use mirb of mruby as a reference.<br>
<a href="https://github.com/mruby/mruby/blob/1905091634a6a2925c911484434448e568330626/mrbgems/mruby-bin-mirb/tools/mirb/mirb.c#L118-L217" class="external">https://github.com/mruby/mruby/blob/1905091634a6a2925c911484434448e568330626/mrbgems/mruby-bin-mirb/tools/mirb/mirb.c#L118-L217</a></p>
<p>The mirb uses compile error messages and token information such as token type and lex_state for that whether or not inputted code continues(and PROMPT_N). I used RubyVM::InstructionSequence#compile and Ripper for the same action.</p>
<p>ref. IRB#each_top_level_statement, IRB#lex and IRB#check_code_block</p>
<a name="ContinuePROMPT_N-literal-typePROMPT_S-nesting-levelNNi"></a>
<h2 >Continue(PROMPT_N), literal type(PROMPT_S), nesting level(%NNi)<a href="#ContinuePROMPT_N-literal-typePROMPT_S-nesting-levelNNi" class="wiki-anchor">¶</a></h2>
<p>IRB needs statuses of code snippets. <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> implemented it as Ripper's features at irb-ripper branch on GitHub.<br>
<a href="https://github.com/nobu/ruby/commits/feature/irb-ripper" class="external">https://github.com/nobu/ruby/commits/feature/irb-ripper</a></p>
<p>I implemented by pure Ruby because I heard what <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> thinks pure Ruby implementation for it. I think that it's the best way for that Ripper keeps simple.</p>
<p>ref. IRB#process_continue, IRB#process_nesting_level, IRB#check_string_literal and IRB#process_literal_type</p>
<a name="Erase-irb_debug-option"></a>
<h2 >Erase --irb_debug option<a href="#Erase-irb_debug-option" class="wiki-anchor">¶</a></h2>
<p>Because this is just for pure Ruby lexical analyzer.</p> Ruby master - Bug #14654 (Closed): Resurvey performance of RDoc by frozen_string_literal: truehttps://bugs.ruby-lang.org/issues/146542018-03-31T13:39:18Zaycabta (aycabta .)aycabta@gmail.com
<p>I added "frozen_string_literal: true" to RDoc by <a href="https://github.com/ruby/rdoc/pull/551" class="external">https://github.com/ruby/rdoc/pull/551</a>. Back then, I reported "It reduces documents generation time by 5%" by <a href="https://github.com/ruby/rdoc/issues/504" class="external">https://github.com/ruby/rdoc/issues/504</a>. The report was added to NEWS file at <a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61354/diff/NEWS" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61354/diff/NEWS</a>.</p>
<p>I resurveyed it: <a href="https://gist.github.com/aycabta/abdfaa75ea8a6877eeb734e942e73800" class="external">https://gist.github.com/aycabta/abdfaa75ea8a6877eeb734e942e73800</a></p>
<blockquote>
<p>The Pull Request <a href="https://github.com/ruby/rdoc/pull/551" class="external">https://github.com/ruby/rdoc/pull/551</a> has many changes for frozen string literal because RDoc expected that all strings are non-frozen. The log "before" that has "frozen_string_literal: false" is faster than the log "after-false" that has "frozen_string_literal: false". Only the implementation changes increase document generation time by 2.6%. But in the case of the same implementation, the log "after" is faster than the log "after-false". The addition "frozen_string_literal: true" reduces document generation time by 1.6%. Finally, the implementation changes and "frozen_string_literal: true" (what means <a href="https://github.com/ruby/rdoc/pull/551" class="external">https://github.com/ruby/rdoc/pull/551</a>) increases document generation time by 0.9%.</p>
</blockquote>
<p>The attached file fixes the report of NEWS file.</p> Ruby master - Feature #14618 (Open): Add display width method to String for CLIhttps://bugs.ruby-lang.org/issues/146182018-03-19T19:00:23Zaycabta (aycabta .)aycabta@gmail.com
<a name="Abstract"></a>
<h2 >Abstract<a href="#Abstract" class="wiki-anchor">¶</a></h2>
<p>Unicode has display width data of characters, "Narrow" or "Wide".<br>
For example, "A" is "Narrow", "💎" ("\u{1f48e}") is "Wide".<br>
<a href="http://unicode.org/reports/tr11/" class="external">http://unicode.org/reports/tr11/</a><br>
This data is very important for CLI tools.</p>
<a name="Use-case"></a>
<h2 >Use-case<a href="#Use-case" class="wiki-anchor">¶</a></h2>
<p>I'm developing Readline compatible library by pure Ruby implementation for Ruby core.<br>
<a href="https://github.com/aycabta/reline" class="external">https://github.com/aycabta/reline</a></p>
<p>I'm discussing it with <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/572">@hsbt (Hiroshi SHIBATA)</a>, and I think that the pure Ruby version should be used only when the native extension version doesn't exist.<br>
ref. <a href="https://bugs.ruby-lang.org/issues/11084" class="external">https://bugs.ruby-lang.org/issues/11084</a><br>
The Readline library is very important for that IRB <em>always</em> provides Readline's features.<br>
So display width method is needed by Ruby core.</p>
<a name="Implementation-approach"></a>
<h2 >Implementation approach<a href="#Implementation-approach" class="wiki-anchor">¶</a></h2>
<a name="Uses-the-official-data-table"></a>
<h3 >Uses the official data table<a href="#Uses-the-official-data-table" class="wiki-anchor">¶</a></h3>
<p>Unicode Consortium provides display width data as "EastAsianWidth.txt".<br>
<a href="http://www.unicode.org/Public/10.0.0/ucd/EastAsianWidth.txt" class="external">http://www.unicode.org/Public/10.0.0/ucd/EastAsianWidth.txt</a></p>
<p>This name is based on historical reasons.<br>
This table is not exclusively for East Asian's characters in the present day, for example, Emoji.</p>
<a name="Uses-new-Regexp-feature-work-in-progress"></a>
<h3 >Uses new Regexp feature (work in progress)<a href="#Uses-new-Regexp-feature-work-in-progress" class="wiki-anchor">¶</a></h3>
<p>I propose new Unicode properties for Onigmo like Perl's one.<br>
<a href="https://github.com/k-takata/Onigmo/pull/102" class="external">https://github.com/k-takata/Onigmo/pull/102</a></p>
<p>I think that this is a better approach if the proposal for Onigmo is merged because String#grapheme_clusters what is based on Unicode specification uses Onigmo's feature inside.</p>
<a name="Cases-of-other-languages-or-libraries"></a>
<h2 >Cases of other languages or libraries<a href="#Cases-of-other-languages-or-libraries" class="wiki-anchor">¶</a></h2>
<p>Python: unicodedata.east_asian_width (standard library)<br>
<a href="https://docs.python.org/3.6/library/unicodedata.html#unicodedata.east_asian_width" class="external">https://docs.python.org/3.6/library/unicodedata.html#unicodedata.east_asian_width</a></p>
<p>Perl: "East_Asian_Width: *" of Unicode properties (regular expression in language)<br>
<a href="https://perldoc.perl.org/perluniprops.html" class="external">https://perldoc.perl.org/perluniprops.html</a></p>
<p>Go: golang.org/x/text/width<br>
<a href="https://godoc.org/golang.org/x/text/width" class="external">https://godoc.org/golang.org/x/text/width</a></p>
<p>PHP: mb_strwidth (standard library)<br>
<a href="http://php.net/manual/en/function.mb-strwidth.php" class="external">http://php.net/manual/en/function.mb-strwidth.php</a></p>
<p>JavaScript: eastasianwidth (npm library)<br>
<a href="https://www.npmjs.com/package/eastasianwidth" class="external">https://www.npmjs.com/package/eastasianwidth</a></p>
<p>RubyGems: unicode-display_width gem<br>
<a href="https://rubygems.org/gems/unicode-display_width" class="external">https://rubygems.org/gems/unicode-display_width</a></p> Ruby master - Bug #14556 (Closed): Please fetch Ripper's new features from nobu's patches bag (ir...https://bugs.ruby-lang.org/issues/145562018-02-27T08:38:12Zaycabta (aycabta .)aycabta@gmail.com
<p>I need new features for Ripper because I'm trying to use Ripper for IRB.</p>
<p>@nobu's feature/irb-ripper branch on GitHub contains what all I need:<br>
<a href="https://github.com/nobu/ruby/commits/feature/irb-ripper" class="external">https://github.com/nobu/ruby/commits/feature/irb-ripper</a></p>
<p>Specifically, I need commits below:<br>
<a href="https://github.com/nobu/ruby/commit/ade428032655838d00747e65517c75b23fc4e1a2" class="external">https://github.com/nobu/ruby/commit/ade428032655838d00747e65517c75b23fc4e1a2</a><br>
<a href="https://github.com/nobu/ruby/commit/5425e26af97f73cefdb16509aade3cdf574371ca" class="external">https://github.com/nobu/ruby/commit/5425e26af97f73cefdb16509aade3cdf574371ca</a><br>
<a href="https://github.com/nobu/ruby/commit/93e604992ecd2c8a883cdffcd6db6ba1f8d58cf1" class="external">https://github.com/nobu/ruby/commit/93e604992ecd2c8a883cdffcd6db6ba1f8d58cf1</a><br>
<a href="https://github.com/nobu/ruby/commit/7f3a3faeee11a9569cc389b891526677839c5d27" class="external">https://github.com/nobu/ruby/commit/7f3a3faeee11a9569cc389b891526677839c5d27</a><br>
<a href="https://github.com/nobu/ruby/commit/51f874c6dbe673ac578bac7ecbdadfefce6ee587" class="external">https://github.com/nobu/ruby/commit/51f874c6dbe673ac578bac7ecbdadfefce6ee587</a><br>
<a href="https://github.com/nobu/ruby/commit/e1bc7437939997f64472e31fe86ea7f29882397f" class="external">https://github.com/nobu/ruby/commit/e1bc7437939997f64472e31fe86ea7f29882397f</a><br>
<a href="https://github.com/nobu/ruby/commit/ef2ef0ac5084b40336923cf099be16db8ff806f1" class="external">https://github.com/nobu/ruby/commit/ef2ef0ac5084b40336923cf099be16db8ff806f1</a></p> Ruby master - Bug #14354 (Closed): Remove confusing doc comments for 2.3https://bugs.ruby-lang.org/issues/143542018-01-14T13:41:03Zaycabta (aycabta .)aycabta@gmail.com
<p>Broken doc comments are removed at @56285:<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/56285/diff" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/56285/diff</a></p>
<p>An issue is reported to RDoc because of this is not backported to 2.3:<br>
<a href="https://github.com/ruby/rdoc/issues/429" class="external">https://github.com/ruby/rdoc/issues/429</a></p> Ruby master - Feature #14170 (Closed): Add allbits?, anybits and nobits? to Ripper::Lexer::Statehttps://bugs.ruby-lang.org/issues/141702017-12-12T12:30:37Zaycabta (aycabta .)aycabta@gmail.com
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> added Ripper::Lexer::State at @60665<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60665/diff/ext/ripper/lib/ripper/lexer.rb" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60665/diff/ext/ripper/lib/ripper/lexer.rb</a><br>
and I think it's necessary for RDoc, I commented it at ruby-core:84111.<br>
<a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/84111" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/84111</a></p>
<p>aycabta (aycabta .) wrote:</p>
<blockquote>
<p>In Ruby 2.5, Ripper::Lexer::State is introduced:<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/ext/ripper/lib/ripper/lexer.rb#L49" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/ext/ripper/lib/ripper/lexer.rb#L49</a></p>
<p>It is for lex_state of parse.y, and has #& and #| for bit operations with lex_state_bits:<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/parse.y#L78" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/parse.y#L78</a></p>
<p>RDoc uses it:<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/lib/rdoc/parser/ripper_state_lex.rb#L321" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60945/entry/lib/rdoc/parser/ripper_state_lex.rb#L321</a></p>
<p>If Integer#allbit? is implemented at 2.5, it's good for Ripper::Lexer::State and I'll use it for RDoc on 2.5.</p>
</blockquote>
<p>I add allbits?, anybits and nobits? to Ripper::Lexer::State by the attached patch.</p> Ruby master - Misc #13871 (Closed): Remove commented out code of URI::HTTP.newhttps://bugs.ruby-lang.org/issues/138712017-09-05T16:52:03Zaycabta (aycabta .)aycabta@gmail.com
<p>It's treated as document for URI::HTTP#request_uri:</p>
<p><a href="https://ruby-doc.org/stdlib-2.4.0/libdoc/uri/rdoc/URI/HTTP.html#method-i-request_uri" class="external">https://ruby-doc.org/stdlib-2.4.0/libdoc/uri/rdoc/URI/HTTP.html#method-i-request_uri</a></p>
<p>It's broken, so the commented out code should be removed.</p>
<p>It's the same type problem of <a href="https://bugs.ruby-lang.org/issues/13870" class="external">https://bugs.ruby-lang.org/issues/13870</a>.</p> Ruby master - Misc #13870 (Closed): Remove commented out code of SecureRandom.random_numberhttps://bugs.ruby-lang.org/issues/138702017-09-05T16:37:18Zaycabta (aycabta .)aycabta@gmail.com
<p>It's treated as document for Random::Formatter#uuid:</p>
<p><a href="https://ruby-doc.org/stdlib-2.4.0/libdoc/securerandom/rdoc/Random/Formatter.html#method-i-uuid" class="external">https://ruby-doc.org/stdlib-2.4.0/libdoc/securerandom/rdoc/Random/Formatter.html#method-i-uuid</a></p>
<p>The commented out code should be removed.</p> Ruby master - Misc #13814 (Closed): URI::FTP documentation for RDoc is brokenhttps://bugs.ruby-lang.org/issues/138142017-08-14T17:37:40Zaycabta (aycabta .)aycabta@gmail.com
<p>This is related to <a class="issue tracker-5 status-5 priority-4 priority-default closed" title="Misc: Ripper documentation for RDoc is broken (Closed)" href="https://bugs.ruby-lang.org/issues/13813">#13813</a>.</p>
<p>In lib/uri/ftp.rb, documentation for #path below:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="c1"># Returns the path from an FTP URI.</span>
<span class="c1">#</span>
<span class="c1"># RFC 1738 specifically states that the path for an FTP URI does not</span>
<span class="c1"># include the / which separates the URI path from the URI host. Example:</span>
<span class="c1">#</span>
<span class="c1"># ftp://ftp.example.com/pub/ruby</span>
<span class="c1">#</span>
<span class="c1"># The above URI indicates that the client should connect to</span>
<span class="c1"># ftp.example.com then cd pub/ruby from the initial login directory.</span>
<span class="c1">#</span>
<span class="c1"># If you want to cd to an absolute directory, you must include an</span>
<span class="c1"># escaped / (%2F) in the path. Example:</span>
<span class="c1">#</span>
<span class="c1"># ftp://ftp.example.com/%2Fpub/ruby</span>
<span class="c1">#</span>
<span class="c1"># This method will then return "/pub/ruby"</span>
<span class="c1">#</span>
</code></pre>
<p>The URIs what are started with "ftp://" is indented,<br>
and indented documentation is treated as Ruby code.<br>
So this becomes strange documentation, it's highlighted as some Ruby tokens.</p>
<p>The screenshot below is rendered HTML of URI::FTP documentation on ruby-doc.org:<br>
<a href="http://ruby-doc.org/stdlib-2.4.1/libdoc/uri/rdoc/URI/FTP.html" class="external">http://ruby-doc.org/stdlib-2.4.1/libdoc/uri/rdoc/URI/FTP.html</a></p>
<p><img src="https://i.gyazo.com/ac8890e335466ce13ac43b0a01dc0ebf.png" alt="URI of FTP as Ruby code"></p>
<p>The second URI of FTP isn't highlighted,<br>
because when a string is invalid as Ruby tokens in RDoc lexical analyser,<br>
it's treated as plain text.<br>
<a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/59592/entry/lib/rdoc/markup/to_html.rb#L207" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/59592/entry/lib/rdoc/markup/to_html.rb#L207</a></p>
<p>I think this should be fixed to correct documentation.<br>
The patch what I attached on this issue fixes it like below:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="c1"># Returns the path from an FTP URI.</span>
<span class="c1">#</span>
<span class="c1"># RFC 1738 specifically states that the path for an FTP URI does not</span>
<span class="c1"># include the / which separates the URI path from the URI host. Example:</span>
<span class="c1">#</span>
<span class="c1"># +ftp://ftp.example.com/pub/ruby+</span>
<span class="c1">#</span>
<span class="c1"># The above URI indicates that the client should connect to</span>
<span class="c1"># ftp.example.com then cd pub/ruby from the initial login directory.</span>
<span class="c1">#</span>
<span class="c1"># If you want to cd to an absolute directory, you must include an</span>
<span class="c1"># escaped / (%2F) in the path. Example:</span>
<span class="c1">#</span>
<span class="c1"># +ftp://ftp.example.com/%2Fpub/ruby+</span>
<span class="c1">#</span>
<span class="c1"># This method will then return "/pub/ruby"</span>
<span class="c1">#</span>
</code></pre> Ruby master - Misc #13813 (Closed): Ripper documentation for RDoc is brokenhttps://bugs.ruby-lang.org/issues/138132017-08-14T15:27:06Zaycabta (aycabta .)aycabta@gmail.com
<p>In ext/ripper/lib/ripper.rb, license section of documentation for RDoc below:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># == License</span>
<span class="c1">#</span>
<span class="c1"># Ruby License.</span>
<span class="c1">#</span>
<span class="c1"># Minero Aoki</span>
<span class="c1"># aamine@loveruby.net</span>
<span class="c1"># http://i.loveruby.net</span>
</code></pre>
<p>This is indented, and indented documentation is treated as Ruby code.<br>
So this becomes strange documentation, it's highlighted as some Ruby tokens.</p>
<p>The screenshot below is rendered HTML of <a href="http://ruby-doc.org/stdlib-2.4.1/libdoc/ripper/rdoc/Ripper.html" class="external">Ripper documentation on ruby-doc.org</a>:</p>
<p><img src="https://i.gyazo.com/a31dad476db18bc44307d2b3d4ebab92.png" alt="e-mail and URL as Ruby code"></p>
<p>I think this should be fixed to correct documentation.<br>
The patch what I attached on this issue fixes it like below:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># == License</span>
<span class="c1">#</span>
<span class="c1"># Ruby License.</span>
<span class="c1">#</span>
<span class="c1"># - Minero Aoki</span>
<span class="c1"># - aamine@loveruby.net</span>
<span class="c1"># - http://i.loveruby.net</span>
</code></pre> Ruby master - Feature #13686 (Closed): Add states of scanner to tokens from Ripper.lex and Ripper...https://bugs.ruby-lang.org/issues/136862017-06-27T12:58:41Zaycabta (aycabta .)aycabta@gmail.com
<p>I'm writing syntax analysis software by pure Ruby, for processing Ruby's source code and meta information what are classes, methods, constants, comments and others. I'm using Ripper for it. But the results of Ripper.sexp doesn't have comments, and the results of Ripper.lex doesn't have token states of scanner.</p>
<p>I think that the behavior of Ripper.sexp is correct because the position of comments in Ruby's syntax tree is blurred and unhandled.</p>
<p>On the other hand, Ripper.lex has comments token but Ripper.lex clearly drops the states of scanner for each tokens under "<a href="https://ruby-hacking-guide.github.io/contextual.html" class="external">finite-state scanner</a>". The states are very important for many Ripper use cases. EXPR_END is especially required to know the end of statement. If the states aren't provided, we must re-implement finite-state analyzer for tokens from Ripper. For example, borderlines of conditions, args, constants and others. It's just not realistic.</p>
<p>In Ripper.lex's behavior as of now:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="o"><<</span><span class="no">EOM</span><span class="p">)</span><span class="sh">
def str?(v)
String === v # check
end
</span><span class="no">EOM</span>
<span class="c1">#=> [[[1, 0], :on_kw, "def" ],</span>
<span class="c1"># [[1, 3], :on_sp, " " ],</span>
<span class="c1"># [[1, 4], :on_ident, "str?" ],</span>
<span class="c1"># [[1, 8], :on_lparen, "(" ],</span>
<span class="c1"># [[1, 9], :on_ident, "v" ],</span>
<span class="c1"># [[1, 10], :on_rparen, ")" ],</span>
<span class="c1"># [[1, 11], :on_ignored_nl, "\n" ],</span>
<span class="c1"># [[2, 0], :on_sp, " " ],</span>
<span class="c1"># [[2, 2], :on_const, "String" ],</span>
<span class="c1"># [[2, 8], :on_sp, " " ],</span>
<span class="c1"># [[2, 9], :on_op, "===" ],</span>
<span class="c1"># [[2, 12], :on_sp, " " ],</span>
<span class="c1"># [[2, 13], :on_ident, "v" ],</span>
<span class="c1"># [[2, 14], :on_sp, " " ],</span>
<span class="c1"># [[2, 15], :on_comment, "# check\n"],</span>
<span class="c1"># [[3, 0], :on_kw, "end" ],</span>
<span class="c1"># [[3, 3], :on_nl, "\n" ]]</span>
</code></pre>
<p>In Ripper.lex's behavior with attached patch:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="o"><<</span><span class="no">EOM</span><span class="p">)</span><span class="sh">
def str?(v)
String === v # check
end
</span><span class="no">EOM</span>
<span class="c1">#=> [[[1, 0], :on_kw, "def", Ripper::EXPR_FNAME ],</span>
<span class="c1"># [[1, 3], :on_sp, " ", Ripper::EXPR_FNAME ],</span>
<span class="c1"># [[1, 4], :on_ident, "str?", Ripper::EXPR_ENDFN ],</span>
<span class="c1"># [[1, 8], :on_lparen, "(", Ripper::EXPR_BEG | Ripper::EXPR_LABEL],</span>
<span class="c1"># [[1, 9], :on_ident, "v", Ripper::EXPR_ARG ],</span>
<span class="c1"># [[1, 10], :on_rparen, ")", Ripper::EXPR_ENDFN ],</span>
<span class="c1"># [[1, 11], :on_ignored_nl, "\n", Ripper::EXPR_BEG ],</span>
<span class="c1"># [[2, 0], :on_sp, " ", Ripper::EXPR_BEG ],</span>
<span class="c1"># [[2, 2], :on_const, "String", Ripper::EXPR_CMDARG ],</span>
<span class="c1"># [[2, 8], :on_sp, " ", Ripper::EXPR_CMDARG ],</span>
<span class="c1"># [[2, 9], :on_op, "===", Ripper::EXPR_BEG ],</span>
<span class="c1"># [[2, 12], :on_sp, " ", Ripper::EXPR_BEG ],</span>
<span class="c1"># [[2, 13], :on_ident, "v", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [[2, 14], :on_sp, " ", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [[2, 15], :on_comment, "# check\n", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [[3, 0], :on_kw, "end", Ripper::EXPR_END ],</span>
<span class="c1"># [[3, 3], :on_nl, "\n", Ripper::EXPR_BEG ]]</span>
</code></pre>
<p>In Ripper::Filter#on_* with attached patch, you can use #state metohd:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="k">class</span> <span class="nc">MyFilter</span> <span class="o"><</span> <span class="no">Ripper</span><span class="o">::</span><span class="no">Filter</span>
<span class="k">def</span> <span class="nf">on_default</span><span class="p">(</span><span class="n">event</span><span class="p">,</span> <span class="n">tok</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
<span class="n">data</span><span class="p">.</span><span class="nf">push</span><span class="p">([</span><span class="n">event</span><span class="p">,</span> <span class="n">tok</span><span class="p">,</span> <span class="n">state</span><span class="p">])</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># Ripper::Filter#parse works like Enumerable#inject</span>
<span class="n">pp</span> <span class="no">MyFilter</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="o"><<</span><span class="no">EOM</span><span class="p">).</span><span class="nf">parse</span><span class="p">([])</span><span class="sh">
def str?(v)
String === v # check
end
</span><span class="no">EOM</span>
<span class="c1">#=> [[:on_kw, "def", Ripper::EXPR_FNAME ],</span>
<span class="c1"># [:on_sp, " ", Ripper::EXPR_FNAME ],</span>
<span class="c1"># [:on_ident, "str?", Ripper::EXPR_ENDFN ],</span>
<span class="c1"># [:on_lparen, "(", Ripper::EXPR_BEG | Ripper::EXPR_LABEL],</span>
<span class="c1"># [:on_ident, "v", Ripper::EXPR_ARG ],</span>
<span class="c1"># [:on_rparen, ")", Ripper::EXPR_ENDFN ],</span>
<span class="c1"># [:on_ignored_nl, "\n", Ripper::EXPR_BEG ],</span>
<span class="c1"># [:on_sp, " ", Ripper::EXPR_BEG ],</span>
<span class="c1"># [:on_const, "String", Ripper::EXPR_CMDARG ],</span>
<span class="c1"># [:on_sp, " ", Ripper::EXPR_CMDARG ],</span>
<span class="c1"># [:on_op, "===", Ripper::EXPR_BEG ],</span>
<span class="c1"># [:on_sp, " ", Ripper::EXPR_BEG ],</span>
<span class="c1"># [:on_ident, "v", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [:on_sp, " ", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [:on_comment, "# check\n", Ripper::EXPR_END | Ripper::EXPR_LABEL],</span>
<span class="c1"># [:on_kw, "end", Ripper::EXPR_END ],</span>
<span class="c1"># [:on_nl, "\n", Ripper::EXPR_BEG ]]</span>
</code></pre> Ruby master - Misc #13117 (Closed): Remove Japanese comment from lib/irb/slex.rbhttps://bugs.ruby-lang.org/issues/131172017-01-09T02:09:54Zaycabta (aycabta .)aycabta@gmail.com
<p>The file lib/irb/slex.rb contains Japanese comment in ISO-2022-JP. This patch removes it.</p> Ruby master - Bug #12840 (Closed): Add test for Forwardable#def_delegator on private methodshttps://bugs.ruby-lang.org/issues/128402016-10-14T12:07:37Zaycabta (aycabta .)aycabta@gmail.com
<p>This is for <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Forwardable breaks on private methods in ruby 2.4 (Closed)" href="https://bugs.ruby-lang.org/issues/12782">#12782</a>.</p> Ruby master - Bug #12837 (Closed): Add test for what Forwardable#def_delegator extends object wha...https://bugs.ruby-lang.org/issues/128372016-10-13T14:47:04Zaycabta (aycabta .)aycabta@gmail.com
<p>In <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: ruby2.2.5でオブジェクトに対して def_delegatorを行うとNoMethodError undefined method `method_defined?' が発生する。 (Closed)" href="https://bugs.ruby-lang.org/issues/12478">#12478</a>, fixed error for what Forwardable#def_delegator extends object what doesn't mix-in Module class. This is test for it.</p>