https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17097754782020-07-01T10:40:12ZRuby Issue Tracking SystemRuby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863772020-07-01T10:40:12Zsawa (Tsuyoshi Sawada)
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/86377/diff?detail_id=57430">diff</a>)</li></ul> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863802020-07-01T15:41:19Zheadius (Charles Nutter)headius@headius.com
<ul></ul><p>In <code>rb_whether_the_return_value_is_used_p</code> I believe <code>whether_the</code> is redundant with <code>p</code>. <code>is</code> also seems unnecessary here, so perhaps <code>rb_return_value_used_p</code> and <code>return_value_used?</code> are good enough?</p>
<p>There is a possible concern if this is used for values that have visible side effects (changing some internal state, setting <code>$~</code> or <code>$_</code>, possibly raising an exception, ...), so it should be used very carefully. Return value may not be the only effect that's important.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863812020-07-01T15:42:49Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I can see definite performance advantages to this in my libraries (specifically Sequel). Knowing that the return value is not used can save expensive database queries to determine what the return value should be.</p>
<p>I think it may be better to make this a private Kernel method, similar to <code>block_given?</code>. <code>RubyVM</code> should only be used for code specific to CRuby, and this should be something that all Ruby implementations should support. I recommend the name <code>return_value_used?</code> for consistency with <code>block_given?</code> (we don't use <code>block_is_given?</code>).</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863832020-07-01T16:02:09Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>My first impression is that this is a very bad tool to change method behavior depending upon its caller context. However, my second consideration is that such a bad thing is already possible by some ways like <code>Kernel#caller</code>. So I'm neutral. If it is introduced. I'd see an explicit caution such as "DO NOT ABUSE THIS!" in its document.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863842020-07-01T16:10:35ZEregon (Benoit Daloze)
<ul></ul><p>What Jeremy said, so in short RubyVM is not a good place for this because it's CRuby-specific (ExperimentalFeatures or Kernel would be OK IMHO).</p>
<p>Do you have measurements on real applications, not just micro-benchmarks?</p>
<p>The masgn case (<code>'1.times {|i| x, y = self, i }'</code>) could be done transparently by the VM without exposing any predicate.<br>
Same for core methods like String#slice!.<br>
I'm not sure if it's a good idea to expose this to Ruby (and C ext) users, it seems very low level.<br>
At least, I think we should take advantage of this in language/core before exposing to users.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863852020-07-01T16:14:03ZEregon (Benoit Daloze)
<ul></ul><p>As an example, I don't think using such a manual optimization in OptCarrot (default mode) would be appropriate (it would feel like a hack).</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863862020-07-01T16:42:50Zenebo (Thomas Enebo)tom.enebo@gmail.com
<ul></ul><p>Reposted from github issue:<br>
"Having only thought about this for about half an hour I am concerned with this feature. Will this lead to people writing APIs where the users of that API need to worry about whether they are assigning the method or not? More or less assignment could end up changing the semantics of the method. Whether a method is assigned or not it should still do the same thing. This feature will allow API designers to break that."</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863872020-07-01T16:52:09Zenebo (Thomas Enebo)tom.enebo@gmail.com
<ul></ul><p>I will add that although I can see the same potential problem with the C api I am less concerned it will lead to the same problems I outlined in Core MRI code.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863882020-07-01T17:02:26Zheadius (Charles Nutter)headius@headius.com
<ul></ul><blockquote>
<p>I'd see an explicit caution such as "DO NOT ABUSE THIS!" in its document.</p>
</blockquote>
<p>It will definitely be abused.</p>
<p>I do not think this should be exposed to user code in any way. If you want a method to either return a result or not depending on how it is called, you should define a different method.</p>
<p>We have been discussing this on the JRuby Matrix chat and there are many concerns:</p>
<ul>
<li>Users of this API will have to ensure there are no side effects of <strong>any kind</strong> that would be omitted.</li>
</ul>
<p>That includes exceptions that might be raised, in-memory state changes elsewhere in the system, database changes, IO state changes, potentially even native memory changes if there are C API calls involved. I would argue that the <strong>only</strong> safe places this can be used are to wrap a simple allocation, and even that has side effects (memory effects, out of memory errors, unexpected downstream or C API calls).</p>
<ul>
<li>If this is exposed as a user-accessible feature, then every place a call is made at the end of a method will want to use the feature to optionally return nil, as below:</li>
</ul>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">foo</span>
<span class="n">do_some_work</span><span class="p">()</span>
<span class="k">if</span> <span class="no">RubyVM</span><span class="p">.</span><span class="nf">return_value_used?</span>
<span class="n">expensive_call_that_can_eliminate_return</span><span class="p">()</span>
<span class="k">else</span>
<span class="n">expensive_call_that_can_eliminate_return</span><span class="p">()</span>
<span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>This will lead to a cascading effect as other methods also try to special-case how they make calls in the context of assignment or returns, forcing other methods to also make changes to how they do calls, ad infinatum.</p>
<ul>
<li>It will be abused, and used in error, probably more often than it is used correctly. And everyone will try to use it thinking they will use it safely, and they'll probably get it wrong.</li>
</ul>
<p>One example case was to eliminate some <code>calculate_expensive_report</code> when the report won't be needed. But the report generation itself will have side effects, like caching data in memory, altering some in-memory data model, advancing an IO position or database cursor.</p>
<ul>
<li>It is not Ruby.</li>
</ul>
<p>Ruby is an expression language. This makes it possible for people to opt out of being an expression, changing visible behavior in the process.</p>
<p>...</p>
<p>I would also point out that an inlining JIT that can see through the methods you might call would already be able to do this. Adding this method essentially short-circuits the "safe" way of doing the optimization and hopes that the user will not make a mistake and eliminate some side effect that was intended.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863892020-07-01T17:04:07Ztenderlovemaking (Aaron Patterson)tenderlove@ruby-lang.org
<ul></ul><p>The more I think about this, the more it concerns me. If there is some library code like this:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">do_something_and_return_report</span>
<span class="n">something</span> <span class="o">=</span> <span class="n">do_something</span>
<span class="k">if</span> <span class="no">RubyVM</span><span class="p">.</span><span class="nf">return_value_used?</span>
<span class="n">create_report</span><span class="p">(</span><span class="n">something</span><span class="p">)</span>
<span class="k">else</span>
<span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>And I am a user of the library. I want to debug my code, so maybe I do this:</p>
<p><code>puts do_something_and_return_report</code></p>
<p>If I remove the <code>puts</code>, then the behavior of <code>do_something_and_return_report</code> would be totally different. Even worse, I cannot use <code>do_something_and_return_report</code> in IRB because the behavior of <code>do_something_and_return_report</code> in IRB would be totally different than the behavior in a script. It would be very confusing to explain "the behavior of <code>do_something_and_return_report</code> is different because IRB used the return value, but your script did not".</p>
<p>This seems like a cool trick, and something that we should use internally to MRI. But I don't think it should be exposed to users. It seems like a situation where the VM and JIT should work harder to optimize code, not library authors or library consumers.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863952020-07-01T23:42:35Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<ul></ul><p>Are there examples (for example from known gems) where this would actually be useful?</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863962020-07-02T00:18:26Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>Additional questions:</p>
<ul>
<li>Are there any other languages that have such a feature?</li>
<li>Where there are performance implications, couldn't that be solved by an<br>
additional parameter to the methods in question? Or by a better design<br>
of interfaces (e.g. different methods for cases where an expensive<br>
return value isn't needed)? (<a class="user active user-mention" href="https://bugs.ruby-lang.org/users/1604">@jeremyevans0 (Jeremy Evans)</a>)</li>
</ul> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=863982020-07-02T01:01:19Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>duerst (Martin Dürst) wrote in <a href="#note-12">#note-12</a>:</p>
<blockquote>
<ul>
<li>Where there are performance implications, couldn't that be solved by an<br>
additional parameter to the methods in question? Or by a better design<br>
of interfaces (e.g. different methods for cases where an expensive<br>
return value isn't needed)? (<a class="user active user-mention" href="https://bugs.ruby-lang.org/users/1604">@jeremyevans0 (Jeremy Evans)</a>)</li>
</ul>
</blockquote>
<p>Yes, it could definitely be solved by additional method arguments (or keyword arguments). However, that can complicate implementation or may not be possible depending on the method's API (consider a method that already accepts arbitrary arguments and arbitrary keywords).</p>
<p>One specific use case I see for this is <code>Sequel::Dataset#insert</code> (<a href="http://sequel.jeremyevans.net/rdoc/classes/Sequel/Dataset.html#method-i-insert" class="external">http://sequel.jeremyevans.net/rdoc/classes/Sequel/Dataset.html#method-i-insert</a>). The return value of this method is generally the primary key value of the last inserted row. On some databases, getting that value is expensive, potentially doubling the execution time of the method when using a remote database. If the return value is not needed, the INSERT query could still be performed, but it would not be necessary to issue another query to SELECT the return value.</p>
<p>Due to <code>Sequel::Dataset#insert</code>'s flexible API, it would be hard to support this as a method argument. I could add a different method to support it, but then I need to add a separate internal method (more indirection, lower performance), or significant duplication. Additionally, having fewer, more flexible methods can result in an API that is easier to remember and use, compared to an API that has many more methods with less flexible behavior for each (with the tradeoff that the internals become significantly more complex).</p>
<p>One potential advantage of the VM_FRAME_FLAG_DISCARDED flag that may not yet have been anticipated is not a performance advantage, but a usability advantage. Consider the following code:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="n">kw</span><span class="p">.</span><span class="nf">merge</span><span class="p">(</span><span class="no">FORCE_VALUES</span><span class="p">)</span>
<span class="n">bar</span><span class="p">(</span><span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>This code has a bug I've seen new Ruby programmers make. The bug is that <code>Hash#merge</code> returns a new hash, it doesn't modify the existing hash. This is almost certainly a bug, because there is no reason to call <code>Hash#merge</code> without using the return value. The programmer almost certainly wanted the behavior of <code>Hash#merge!</code>. Basically, <code>Hash#merge</code> is a pure function. We could add a way to mark methods as pure functions (e.g. <code>Module#pure_function</code>), and if the method is called with VM_FRAME_FLAG_DISCARDED, Ruby could warn or raise.</p>
<p>While I am in favor of this, I can certainly understand the potential for abuse. However, Ruby has never been a language that avoided features simply because it is possible to abuse them. That being said, I think it would make sense to make this strictly internal initially, and not expose it to C extensions and pure ruby code unless the internal usage demonstrates its usefulness.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864002020-07-02T05:31:38Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<ul></ul><p>Re: other languages with similar concepts.</p>
<ul>
<li>Perl has <code>wantarray</code>. In spite of its name, the intrinsic can be used to distinguish if a return value is needed or not (can tell you if the needed number of return values is zero, one, or many more).</li>
<li>If we consider warnings on unused return values be a kind of it...
<ul>
<li>C++ since C++17 has <code>[[nodiscard]]</code> function attribute.</li>
<li>GCC provides something similar to C as well.</li>
<li>In Rust that attribute is called <code>#[must_use]</code>.</li>
<li>Swift has such warnings default on, and must explicitly annotate a function with <code>@discardableResult</code> if you allow users to ignore them.</li>
</ul>
</li>
</ul> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864062020-07-02T16:13:32Zsoulcutter (Bradley Schaefer)bradley.schaefer@gmail.com
<ul></ul><p>jeremyevans0 (Jeremy Evans) wrote in <a href="#note-13">#note-13</a>:</p>
<blockquote>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="n">kw</span><span class="p">.</span><span class="nf">merge</span><span class="p">(</span><span class="no">FORCE_VALUES</span><span class="p">)</span>
<span class="n">bar</span><span class="p">(</span><span class="o">**</span><span class="n">kw</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>This code has a bug I've seen new Ruby programmers make. The bug is that <code>Hash#merge</code> returns a new hash, it doesn't modify the existing hash. This is almost certainly a bug, because there is no reason to call <code>Hash#merge</code> without using the return value. The programmer almost certainly wanted the behavior of <code>Hash#merge!</code>. Basically, <code>Hash#merge</code> is a pure function. We could add a way to mark methods as pure functions (e.g. <code>Module#pure_function</code>), and if the method is called with VM_FRAME_FLAG_DISCARDED, Ruby could warn or raise.</p>
</blockquote>
<p>What scares me about this is the idea of using interactive debuggers (or even plan-old puts debugging) changing the behavior of the code. You would have to be an expert (primed to think about this behavior) to recognize that simply observing a method means you can't make any assumptions about what happens when you're not observing it. Also, how would you test this behavior?</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864072020-07-02T16:21:53Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>soulcutter (Bradley Schaefer) wrote in <a href="#note-15">#note-15</a>:</p>
<blockquote>
<p>Also, how would you test this behavior?</p>
</blockquote>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># return value not discarded case</span>
<span class="n">some_method</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:expected_return_value</span>
<span class="n">check_for</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:some_side_effect</span>
<span class="c1"># return value discarded case</span>
<span class="n">some_method</span>
<span class="n">check_for</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:some_side_effect</span>
</code></pre> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864092020-07-02T16:45:42Zsoulcutter (Bradley Schaefer)bradley.schaefer@gmail.com
<ul></ul><p>jeremyevans0 (Jeremy Evans) wrote in <a href="#note-16">#note-16</a>:</p>
<blockquote>
<p>soulcutter (Bradley Schaefer) wrote in <a href="#note-15">#note-15</a>:</p>
<blockquote>
<p>Also, how would you test this behavior?</p>
</blockquote>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># return value not discarded case</span>
<span class="n">some_method</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:expected_return_value</span>
<span class="n">check_for</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:some_side_effect</span>
<span class="c1"># return value discarded case</span>
<span class="n">some_method</span>
<span class="n">check_for</span><span class="p">.</span><span class="nf">must_equal</span> <span class="ss">:some_side_effect</span>
</code></pre>
</blockquote>
<p>That's fair. There's still a warning flag in my head that there's some subtle case where it is trickier to test, but I might be struggling to wrap my head around all the implications. Given that I can't come up with that example at the moment, I retract that problem-statement.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864112020-07-02T17:26:02ZEregon (Benoit Daloze)
<ul></ul><p>Agreed as well on the point of "if I observe it with <code>p/puts/IRB</code> I don't want the method call to behave differently.<br>
Debug printing should avoid having side effects, and this makes a significant way to break that.<br>
Sounds also very confusing when benchmarking some method and leaving an unused variable vs removing it and seeing a large difference/maybe the benchmark doesn't do at all what it intended.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864132020-07-02T20:15:45Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<ul></ul><p>jeremyevans0 (Jeremy Evans) wrote in <a href="#note-13">#note-13</a>:</p>
<blockquote>
<p>Due to Sequel::Dataset#insert's flexible API, it would be hard to support this as a method argument.</p>
</blockquote>
<p>Seems to be that a better API for this is using a block parameter.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># need the ID:</span>
<span class="n">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">last_inserted_id</span><span class="o">|</span>
<span class="c1"># ...</span>
<span class="k">end</span>
<span class="c1"># don't need the ID:</span>
<span class="n">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</code></pre>
<p>Another possibility is to implement a lazy return value, something like:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">LazyID</span>
<span class="k">def</span> <span class="nf">to_i</span>
<span class="c1"># get the ID</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="c1"># ...</span>
<span class="no">LazyID</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>I remain unconvinced the proposal is a good idea.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=864152020-07-02T21:11:05Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>marcandre (Marc-Andre Lafortune) wrote in <a href="#note-19">#note-19</a>:</p>
<blockquote>
<p>jeremyevans0 (Jeremy Evans) wrote in <a href="#note-13">#note-13</a>:</p>
<blockquote>
<p>Due to Sequel::Dataset#insert's flexible API, it would be hard to support this as a method argument.</p>
</blockquote>
<p>Seems to be that a better API for this is using a block parameter.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># need the ID:</span>
<span class="n">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">last_inserted_id</span><span class="o">|</span>
<span class="c1"># ...</span>
<span class="k">end</span>
<span class="c1"># don't need the ID:</span>
<span class="n">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</code></pre>
</blockquote>
<p>Sequel::Dataset#insert already accepts a block, which can be used to iterate over rows returned from the insert statement (when using INSERT RETURNING).</p>
<blockquote>
<p>Another possibility is to implement a lazy return value, something like:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">LazyID</span>
<span class="k">def</span> <span class="nf">to_i</span>
<span class="c1"># get the ID</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">insert</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="c1"># ...</span>
<span class="no">LazyID</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
</blockquote>
<p>In general, this approach requires additional allocation in the case where you are using the return value, decreasing performance. In this particular case, it's not possible, because you would probably lose access to the database connection used to insert the record before calling <code>LazyID#to_i</code>, and that database connection is needed to get the value.</p> Ruby master - Feature #17004: Provide a way for methods to omit their return valuehttps://bugs.ruby-lang.org/issues/17004?journal_id=866182020-07-20T05:44:24Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>To disclose this kind of information to the Ruby level is just too much, I feel. As a compromise, how about experimenting some internal API (via Primitive) for CRuby?</p>
<p>Matz.</p>