https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112018-10-28T10:39:57ZRuby Issue Tracking SystemRuby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746392018-10-28T10:39:57Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/74639/diff?detail_id=50175">diff</a>)</li><li><strong>Status</strong> changed from <i>Open</i> to <i>Feedback</i></li></ul><p>larskanis (Lars Kanis) wrote:</p>
<blockquote>
<p>Although <code>start_adder</code> works with a <code>WeakRef</code>, the <code>Adder</code> object should still be GC marked, since <code>Adder.new</code> doesn't return before all calls to <code>start_adder</code> finished.</p>
</blockquote>
<blockquote>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="vi">@count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">WeakRef</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">),</span> <span class="o">&</span><span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:start_adder</span><span class="p">))</span>
<span class="k">end</span>
</code></pre>
</blockquote>
<p>This method doesn't wait these threads which run <code>start_adder</code>.</p>
<blockquote>
<a name="Actual-behavior"></a>
<h2 >Actual behavior:<a href="#Actual-behavior" class="wiki-anchor">¶</a></h2>
<p>The program stops with a probability of approximately 80% with the following error:</p>
</blockquote>
<p>So this behavior seems expected.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746432018-10-28T11:52:45Zlarskanis (Lars Kanis)
<ul></ul><p>Thanks <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> for looking at the issue!</p>
<blockquote>
<p>This method doesn't wait these threads which run start_adder.</p>
</blockquote>
<p>It does: <code>initialize</code> waits for all threads to have called <code>Adder#add</code> at the last line at <code>@qu.deq</code> . It only returns after this event.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746492018-10-29T01:13:46Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>larskanis (Lars Kanis) wrote:</p>
<blockquote>
<p>It does: <code>initialize</code> waits for all threads to have called <code>Adder#add</code> at the last line at <code>@qu.deq</code> . It only returns after this event.</p>
</blockquote>
<p>It waits just once.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746552018-10-29T09:00:32Zlarskanis (Lars Kanis)
<ul><li><strong>File</strong> <a href="/attachments/7440">adder-test2.rb</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7440/adder-test2.rb">adder-test2.rb</a> added</li></ul><blockquote>
<p>It waits just once.</p>
</blockquote>
<p>Yes, but the one event is sent after all 10 threads have been called, so that <code>@count</code> is decremented to 0.</p>
<p>The same error is present, when we wait for 10 calls to the queue. So the class can actually be simplified like so:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">Adder</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">start_adder</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">obj</span><span class="p">.</span><span class="nf">add</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@qu</span> <span class="o">=</span> <span class="no">Queue</span><span class="p">.</span><span class="nf">new</span>
<span class="n">count</span> <span class="o">=</span> <span class="mi">10</span>
<span class="n">count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">WeakRef</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">),</span> <span class="o">&</span><span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:start_adder</span><span class="p">))</span>
<span class="k">end</span>
<span class="n">count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="vi">@qu</span><span class="p">.</span><span class="nf">deq</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">add</span>
<span class="vi">@qu</span><span class="p">.</span><span class="nf">enq</span> <span class="kp">true</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>This version raises <code>Invalid Reference - probably recycled (WeakRef::RefError)</code> with a probability of around 50% on my Linux systems. The whole file is attached.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746632018-10-30T01:34:48Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>larskanis (Lars Kanis) wrote:</p>
<blockquote>
<blockquote>
<p>It waits just once.</p>
</blockquote>
<p>Yes, but the one event is sent after all 10 threads have been called, so that <code>@count</code> is decremented to 0.</p>
</blockquote>
<p>Why do you think so?</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746822018-10-30T15:05:44Zlarskanis (Lars Kanis)
<ul><li><strong>Subject</strong> changed from <i>GCing of object in use</i> to <i>WeakRef::RefError for object that is still in use</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/74682/diff?detail_id=50192">diff</a>)</li></ul> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746832018-10-30T15:08:55Zlarskanis (Lars Kanis)
<ul></ul><p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> I updated the bug report, so that it should be more understandable now, what's going wrong.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=746852018-10-30T16:16:37Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>I agree, this appears to a be a bug in WeakRef. The Adder instance is not garbage collected, but you still get a <code>WeakRef::RefError</code> when referencing it through the WeakRef instance. Here's a slightly modified example that just prints when the <code>WeakRef::RefError</code> is raised and prints inside <code>Adder#initialize</code> showing that the Adder instance has not yet been garbage collected after the <code>WeakRef::RefError</code> is raised (and the object ids for the instance and @qu match).</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s2">"weakref"</span>
<span class="k">class</span> <span class="nc">Adder</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">start_adder</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">oid</span><span class="p">,</span> <span class="n">q</span><span class="p">)</span>
<span class="n">obj</span><span class="p">.</span><span class="nf">add</span>
<span class="k">rescue</span> <span class="no">WeakRef</span><span class="o">::</span><span class="no">RefError</span>
<span class="nb">p</span> <span class="p">[</span><span class="no">WeakRef</span><span class="o">::</span><span class="no">RefError</span><span class="p">,</span> <span class="n">oid</span><span class="p">,</span> <span class="n">q</span><span class="p">.</span><span class="nf">object_id</span><span class="p">]</span>
<span class="n">q</span><span class="p">.</span><span class="nf">push</span> <span class="p">[</span><span class="n">oid</span><span class="p">,</span> <span class="n">q</span><span class="p">.</span><span class="nf">object_id</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@qu</span> <span class="o">=</span> <span class="no">Queue</span><span class="p">.</span><span class="nf">new</span>
<span class="n">count</span> <span class="o">=</span> <span class="mi">10</span>
<span class="n">count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">WeakRef</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">),</span> <span class="nb">object_id</span><span class="p">,</span> <span class="vi">@qu</span><span class="p">,</span> <span class="o">&</span><span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:start_adder</span><span class="p">))</span>
<span class="k">end</span>
<span class="n">count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="n">oid</span><span class="p">,</span> <span class="n">q_oid</span> <span class="o">=</span> <span class="vi">@qu</span><span class="p">.</span><span class="nf">pop</span>
<span class="k">if</span> <span class="n">oid</span>
<span class="nb">p</span> <span class="p">[</span><span class="ss">:initialize</span><span class="p">,</span> <span class="nb">self</span><span class="p">,</span> <span class="n">oid</span><span class="p">,</span> <span class="nb">object_id</span><span class="p">,</span> <span class="n">oid</span> <span class="o">==</span> <span class="nb">object_id</span><span class="p">,</span> <span class="n">q_oid</span> <span class="o">==</span> <span class="vi">@qu</span><span class="p">.</span><span class="nf">object_id</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">add</span>
<span class="vi">@qu</span><span class="p">.</span><span class="nf">push</span> <span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">test_adder</span>
<span class="n">oids</span> <span class="o">=</span> <span class="mi">10</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="no">Adder</span><span class="p">.</span><span class="nf">new</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:join</span><span class="p">)</span>
<span class="k">end</span>
<span class="mi">1000</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="n">test_adder</span>
<span class="k">end</span>
</code></pre>
<p>Output on ruby 2.5.3:</p>
<pre><code>[WeakRef::RefError, 16073782609840, 16073782609800]
[:initialize, #<Adder:0x00001d3cf0348f60 @qu=#<Thread::Queue:0x00001d3cf0348f10>>, 16073782609840, 16073782609840, true, true]
[WeakRef::RefError, 16073295767400, 16073295767360]
[:initialize, #<Adder:0x00001d3cb62b4ed0 @qu=#<Thread::Queue:0x00001d3cb62b4e80>>, 16073295767400, 16073295767400, true, true]
</code></pre> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=747002018-10-31T10:07:25Zlarskanis (Lars Kanis)
<ul></ul><p>The reference error doesn't appear when <code>WeakRef</code> is replaced by <code>_id2ref</code> like so:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">start_adder</span><span class="p">(</span><span class="n">oid</span><span class="p">)</span>
<span class="no">ObjectSpace</span><span class="p">.</span><span class="nf">_id2ref</span><span class="p">(</span><span class="n">oid</span><span class="p">).</span><span class="nf">add</span>
<span class="k">end</span>
</code></pre>
<p>So it appears to be a bug in <code>ObjectSpace::WeakMap</code>.</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=838542020-01-14T10:38:51Zlarskanis (Lars Kanis)
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Open</i></li></ul> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=838572020-01-14T11:13:00Zlarskanis (Lars Kanis)
<ul></ul><p>This issue appears to be fixed in ruby-2.7.0 by the recent WeakMap modifications. It is still present on:</p>
<pre><code>ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
ruby 2.5.7p206 (2019-10-01 revision 67816) [x86_64-linux]
ruby 2.6.5p114 (2019-10-01 revision 67812) [x64-mingw32]
</code></pre>
<p>However since the exact change isn't known, I think backporting is not necessary. So the issue can be closed (I don't have an option for closing).</p> Ruby master - Bug #15262: WeakRef::RefError for object that is still in usehttps://bugs.ruby-lang.org/issues/15262?journal_id=838672020-01-14T16:18:33Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul>