https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112021-03-02T01:25:34ZRuby Issue Tracking SystemRuby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906842021-03-02T01:25:34Zarjunmdas (arjun das)
<ul><li><strong>Subject</strong> changed from <i>Sleep in a thread hangs when Fiber.set_scheduler is set</i> to <i>Thread#join hangs when Fiber.set_scheduler is set</i></li></ul><pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">MockScheduler</span>
<span class="k">def</span> <span class="nf">block</span><span class="p">(</span><span class="n">blocker</span><span class="p">,</span> <span class="n">timeout</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">close</span>
<span class="n">byebug</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">fiber</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">blocking: </span><span class="kp">false</span><span class="p">,</span> <span class="o">&</span><span class="n">block</span><span class="p">).</span><span class="nf">tap</span><span class="p">(</span><span class="o">&</span><span class="ss">:resume</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">io_wait</span><span class="p">(</span><span class="n">io</span><span class="p">,</span> <span class="n">events</span><span class="p">,</span> <span class="n">timeout</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">kernel_sleep</span><span class="p">(</span><span class="n">duration</span><span class="o">=</span><span class="kp">nil</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">yield</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">process_wait</span><span class="p">(</span><span class="n">pid</span><span class="p">,</span> <span class="n">flags</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">unblock</span><span class="p">(</span><span class="n">blocker</span><span class="p">,</span> <span class="n">fiber</span><span class="p">)</span>
<span class="n">byebug</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">set_scheduler</span><span class="p">(</span><span class="no">MockScheduler</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span>
<span class="n">t1</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="nb">p</span> <span class="s1">'before'</span>
<span class="nb">p</span> <span class="s1">'after'</span>
<span class="k">end</span>
<span class="n">t1</span><span class="p">.</span><span class="nf">join</span>
</code></pre>
<p>Code hangs a this point.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906862021-03-02T03:42:57Zxtkoba (Tee KOBAYASHI)
<ul></ul><p>This issue seems already resolved in the latest development version. (I confirmed that it reproduces in 3.0.0p0 release version on x86_64-linux.)</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906872021-03-02T05:07:47Zarjunmdas (arjun das)
<ul></ul><p>Thanks a lot for confirming.<br>
I will check the latest development version as well. Are you aware of any other tracking bug for this?</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906882021-03-02T05:56:54Zxtkoba (Tee KOBAYASHI)
<ul></ul><p>This issue seems fixed in <a class="changeset" title="Expose scheduler as public interface & bug fixes. (#3945) * Rename `rb_scheduler` to `rb_fiber_s..." href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/5f69a7f60467fa58c2f998daffab43e118bff36c">5f69a7f60467fa58c2f998daffab43e118bff36c</a>. I'm not sure if there is a ticket here. There is a related PR in GitHub: <a href="https://github.com/ruby/ruby/pull/3945" class="external">https://github.com/ruby/ruby/pull/3945</a></p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906892021-03-02T06:18:35Zarjunmdas (arjun das)
<ul></ul><p>Thanks again.</p>
<p>Just curious, how did you identify the right PR that fixed this?</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=906912021-03-02T06:26:10Zxtkoba (Tee KOBAYASHI)
<ul></ul><p><a class="user active" href="https://bugs.ruby-lang.org/users/50554">arjunmdas (arjun das)</a> I just searched the git log with the keyword "fiber". The PR number comes from the commit message.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=907312021-03-03T22:38:40Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=907382021-03-04T00:32:34Zarjunmdas (arjun das)
<ul></ul><p><a href="https://bugs.ruby-lang.org/users/48178" class="external">xtkoba (Tee KOBAYASHI)</a> - Re-verified in the latest development. It's fixed. Thanks for the update.<br>
.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=928912021-07-15T01:57:24Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p>I think we should consider isolating this for backport to 3.0.3 if possible.</p>
<p>This three lines:</p>
<p><a href="https://github.com/ruby/ruby/blob/v3_0_2/thread.c#L547-L549" class="external">https://github.com/ruby/ruby/blob/v3_0_2/thread.c#L547-L549</a></p>
<pre><code class="c syntaxhl" data-language="c"> <span class="k">if</span> <span class="p">(</span><span class="n">target_thread</span><span class="o">-></span><span class="n">scheduler</span> <span class="o">!=</span> <span class="n">Qnil</span><span class="p">)</span> <span class="p">{</span>
<span class="n">rb_scheduler_unblock</span><span class="p">(</span><span class="n">target_thread</span><span class="o">-></span><span class="n">scheduler</span><span class="p">,</span> <span class="n">target_thread</span><span class="o">-></span><span class="n">self</span><span class="p">,</span> <span class="n">rb_fiberptr_self</span><span class="p">(</span><span class="n">join_list</span><span class="o">-></span><span class="n">fiber</span><span class="p">));</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
</code></pre>
<p>should become:</p>
<pre><code class="c syntaxhl" data-language="c"> <span class="k">if</span> <span class="p">(</span><span class="n">target_thread</span><span class="o">-></span><span class="n">scheduler</span> <span class="o">!=</span> <span class="n">Qnil</span> <span class="o">&&</span> <span class="n">rb_fiberptr_blocking</span><span class="p">(</span><span class="n">join_list</span><span class="o">-></span><span class="n">fiber</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="n">rb_scheduler_unblock</span><span class="p">(</span><span class="n">target_thread</span><span class="o">-></span><span class="n">scheduler</span><span class="p">,</span> <span class="n">target_thread</span><span class="o">-></span><span class="n">self</span><span class="p">,</span> <span class="n">rb_fiberptr_self</span><span class="p">(</span><span class="n">join_list</span><span class="o">-></span><span class="n">fiber</span><span class="p">));</span>
<span class="p">}</span>
</code></pre>
<p>We would need to backport <code>rb_fiberptr_blocking</code> from <code>cont.c</code> too:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="kt">unsigned</span> <span class="kt">int</span> <span class="nf">rb_fiberptr_blocking</span><span class="p">(</span><span class="k">struct</span> <span class="n">rb_fiber_struct</span> <span class="o">*</span><span class="n">fiber</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">fiber</span><span class="o">-></span><span class="n">blocking</span><span class="p">;</span>
<span class="p">}</span>
</code></pre>
<p>There are some other usage in <code>thread_sync.c</code> which might be candidate for backport. Check <a href="https://github.com/ruby/ruby/blob/master/thread_sync.c" class="external">https://github.com/ruby/ruby/blob/master/thread_sync.c</a> for usage of <code>rb_fiberptr_blocking</code>. These are all bugs because it can cause the fiber scheduler to be invoked when it shouldn't causing indefinite hang.</p>
<p>cc <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/404">@nagachika (Tomoyuki Chikanaga)</a></p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=928922021-07-15T04:10:07Znagachika (Tomoyuki Chikanaga)nagachika00@gmail.com
<ul><li><strong>Backport</strong> changed from <i>2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN</i> to <i>2.7: DONTNEED, 3.0: REQUIRED</i></li></ul><p>I don't have enough time to see details of the request just now. I filled the Backport field to recall myself to this ticket later.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929372021-07-18T07:32:35Znagachika (Tomoyuki Chikanaga)nagachika00@gmail.com
<ul></ul><p>Hello Samuel,<br>
I have created a backport patch according to your suggestions.</p>
<p><a href="https://github.com/nagachika/ruby/commit/c2697018d4d8cad7ea80ca6aa57f65d76072053c" class="external">https://github.com/nagachika/ruby/commit/c2697018d4d8cad7ea80ca6aa57f65d76072053c</a></p>
<p>Are there anything else should be included?<br>
Would you review my patch please?</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929402021-07-18T17:37:32Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/404">@nagachika (Tomoyuki Chikanaga)</a> thanks so much for your effort here. I have a local test case which can easily fail without this fix, so I'll try your PR to confirm it fixes the issue.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929412021-07-18T23:05:02Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p>I have made a PR to your PR.</p>
<p><a href="https://github.com/nagachika/ruby/pull/1" class="external">https://github.com/nagachika/ruby/pull/1</a></p>
<p>This adds a test case which fails without this backport.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929422021-07-18T23:10:38Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p>I also tested it against <code>async</code> <code>main</code> branch which can hang on 3.0.2, but passes on 3.0.2 + your PR. So, I can confirm this fixes the issue that was discussed here.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929712021-07-22T02:56:22Znagachika (Tomoyuki Chikanaga)nagachika00@gmail.com
<ul></ul><p>Thank you for your confirmation and adding a test!<br>
I will create a patch for ruby_3_0 branch based on your pull request.</p> Ruby master - Bug #17666: Thread#join hangs when Fiber.set_scheduler is sethttps://bugs.ruby-lang.org/issues/17666?journal_id=929832021-07-25T05:11:54Znagachika (Tomoyuki Chikanaga)nagachika00@gmail.com
<ul><li><strong>Backport</strong> changed from <i>2.7: DONTNEED, 3.0: REQUIRED</i> to <i>2.7: DONTNEED, 3.0: DONE</i></li></ul><p>ruby_3_0 95dc88c88869541dd0eccafd14924d78c8d7f427 merged revision(s) 5f69a7f60467fa58c2f998daffab43e118bff36c.</p>