https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112019-09-09T14:22:45ZRuby Issue Tracking SystemRuby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=814812019-09-09T14:22:45ZDan0042 (Daniel DeLorme)
<ul></ul><p>This is an issue for <em>all</em> delegation code. This <code>pass_positional_hash</code> workaround is ok for the stdlib, but what about gems? What if a gem wants to stay compatible with 2.6? This is a question I really don't know the answer to, so I'm asking for an authoritative answer in <a class="issue tracker-5 status-1 priority-4 priority-default" title="Misc: What is the correct and *portable* way to do generic delegation? (Open)" href="https://bugs.ruby-lang.org/issues/16157">#16157</a>.</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=815052019-09-10T17:34:46Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>As some background for the dev meeting, I tried a couple of different approaches before this that didn't work.</p>
<p>One approach was to flag methods and have all positional hashes be treated as positional hashes (Ruby 3 behavior). That didn't work as it changed the behavior for the case where <code>bar</code> accepts keyword arguments.</p>
<p>Another approach was to flag methods and ignore positional hash to keyword warnings for the method. But in that case, this is no warning for the case where <code>bar</code> accepts keyword arguments, even though that behavior will change in Ruby 3.</p>
<p>I chose to use <code>Module#pass_positional_hash</code> as the way to turn on the flag, as that seemed easiest. I thought about using a per-method magic comment, but the issue with that approach is it probably doesn't work well for methods defined with <code>define_method</code>.</p>
<p>Instead of using a VM frame flag in the next frame when doing positional hash to keyword argument conversion, we could set an object flag on the keyword argument hash, and check the flag later. That will probably be more challenging to implement, as I think it will require compiler modifications, because when you use <code>**kw</code>, the object passed to the callee is not <code>kw</code>, but a copy of <code>kw</code>. It also raises issues if pass <code>kw</code> to another method as a positional argument, and that method splats it.</p>
<p>I've updated the pull request to document that <code>pass_positional_hash</code> should only be used with methods that do not modify keyword arguments or append positional arguments before delegation. Doing so can result in behavior changes in Ruby 3 without any warning in 2.7. See <a href="https://github.com/jeremyevans/ruby/commit/97eeab3d90fe2f560996618c391b68aed4783610#diff-5dbe613f725860801fa4c2c812b1d181" class="external">https://github.com/jeremyevans/ruby/commit/97eeab3d90fe2f560996618c391b68aed4783610#diff-5dbe613f725860801fa4c2c812b1d181</a>.</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=815232019-09-11T21:30:51Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I added an alternative approach for handling delegate keyword warnings in <a href="https://github.com/ruby/ruby/pull/2449" class="external">https://github.com/ruby/ruby/pull/2449</a>. This approach also uses a VM frame flag and a Module method (<code>Module#pass_keywords</code>), and allows for passing keywords through a regular argument splat. The advantage of this approach is it should allow for using the same delegation method code on older versions of Ruby without modification, by just marking existing methods that do delegation to have them pass through the keyword arguments.</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=816172019-09-19T15:32:59Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>At the dev meeting yesterday, matz recommended changing the method name from <code>pass_keywords</code> to <code>ruby2_keywords</code>. I have updated the pull request to do that: <a href="https://github.com/ruby/ruby/pull/2449" class="external">https://github.com/ruby/ruby/pull/2449</a></p>
<p>akr disliked the approach of using a VM frame flag. I believe it would not be too difficult to change the implementation from using a VM frame flag to using a flag on the final hash object. It's possible you could support that via an option to <code>ruby2_keywords</code> (e.g. <code>ruby2_keywords :method_name, flag: :hash</code>). There are some cases that are handled by a VM frame flag and not handled by a hash object flag (e.g. <code>def foo(*args) args[-1] = args[-1].merge(...) if args[-1].is_a?(Hash); bar(*args) end</code>), and others that are handled by a hash object flag and not a VM frame flag (e.g. <code>def bar(*args) @args = args end; def baz; quux(*@args) end</code>).</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=816502019-09-21T20:54:09Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I worked on an alternative approach of using <code>ruby2_keywords</code> with a hash flag approach instead of a VM frame flag approach: <a href="https://github.com/ruby/ruby/pull/2477" class="external">https://github.com/ruby/ruby/pull/2477</a> . This approach is significantly less invasive to implement, and I think handles more common argument delegation cases than the VM frame flag approach (both approaches handle the simple case). I think we should use the hash flag approach.</p>
<p>There have been discussions about whether to make the <code>ruby2_keywords</code> behavior the default behavior. I do not think it is a good idea. Consider this example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">args</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">arg</span><span class="o">|</span>
<span class="n">bar</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="o">**</span><span class="p">{})</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bar</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="n">baz</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="k">unless</span> <span class="n">kw</span><span class="p">[</span><span class="ss">:skip</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">baz</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="p">[</span><span class="n">h</span><span class="p">,</span> <span class="n">kw</span><span class="p">]</span>
<span class="k">end</span>
</code></pre>
<p>Here <code>foo</code> is designed to take only positional arguments, not to pass through keywords. Let's say you call <code>foo(a: 1)</code>. Because the last argument to foo is flagged as keywords, when <code>baz</code> is called, the <code>{a: 1}</code> hash intended to be a positional argument is treated as a keyword argument. Note that <code>baz</code> could be at a completely different place in the program, far from the definition of <code>foo</code>, which will make cases like this difficult to debug.</p>
<p>If you would like to experiment with the approach of passing keywords through argument splats by default, I have a branch that implements it: <a href="https://github.com/jeremyevans/ruby/tree/ruby2_keywords-by-default" class="external">https://github.com/jeremyevans/ruby/tree/ruby2_keywords-by-default</a> . The only <code>make check</code> failures are in the keyword arguments tests, due to empty hash splats no longer being dropped in the case where a method accepts an argument splat but no keywords.</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=816762019-09-23T20:01:03ZDan0042 (Daniel DeLorme)
<ul></ul><p>When reading the code it seems like the intent of <code>foo(test:42)</code> is for baz to return <code>[{:test=>42}, {}]</code> but that's not what I get when running it on 2.7. I get a warning and <code>[1, {:test=>42}]</code>. To fix the code on 2.7 I need to change <code>baz(*args)</code> to <code>baz(*args, **{})</code>. And at that point, I believe enabling ruby2_keywords by default would not change the result.</p>
<p>Also let's say that such code exists today out there in the wild. It would not use <code>**{}</code> because no one uses that no-op. The result of baz would be <code>[1, {}]</code>. Enabling ruby2_keywords by default would simply preserve the existing behavior without warnings in 2.7, whether that's a good or a bad thing.</p>
<p>What I'm getting at here is that by enabling ruby2_keywords by default we get a behavior that is a bit closer to 2.6. Both the good and the bad. So the separation of keyword arguments is a bit more muddy, but at least it can be said to be backward compatible. It's debatable which is better.</p> Ruby master - Bug #16154: lib/delegate.rb issues keyword argument warningshttps://bugs.ruby-lang.org/issues/16154?journal_id=817242019-09-25T19:35:36Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>The hash-flag approach for ruby2_keywords has been merged at 3b302ea8c95d34d5ef072d7e3b326f28a611e479. That commit uses ruby2_keywords in delegate, fixing the keyword argument separation issues.</p>