https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112020-01-03T14:05:10ZRuby Issue Tracking SystemRuby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=836122020-01-03T14:05:10Zkirs (Kir Shatrov)shatrov@me.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/83612/diff?detail_id=56045">diff</a>)</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=836142020-01-03T21:08:13Zkirs (Kir Shatrov)shatrov@me.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/83614/diff?detail_id=56048">diff</a>)</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=836902020-01-07T15:37:50ZDan0042 (Daniel DeLorme)
<ul></ul><p>+1</p>
<p>This has been an issue for a very long time, and it's often been handled by installing an asynchronous DNS resolver gem, but it would be nice if it "just worked". If it's really as simple as using <code>getaddrinfo_a</code>, that sounds great.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=837412020-01-10T02:47:10Zkirs (Kir Shatrov)shatrov@me.com
<ul></ul><p>Dan0042 (Daniel DeLorme) wrote:</p>
<blockquote>
<p>+1</p>
<p>This has been an issue for a very long time, and it's often been handled by installing an asynchronous DNS resolver gem, but it would be nice if it "just worked". If it's really as simple as using <code>getaddrinfo_a</code>, that sounds great.</p>
</blockquote>
<p>Thanks for feedback Daniel!</p>
<p>I've put a PR with the suggested fix: <a href="https://github.com/ruby/ruby/pull/2827" class="external">https://github.com/ruby/ruby/pull/2827</a></p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=842292020-02-11T22:43:37Zkirs (Kir Shatrov)shatrov@me.com
<ul><li><strong>File</strong> deleted (<del><i>getaddrinfo_interrupt.rb</i></del>)</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=842302020-02-11T23:00:02Zkirs (Kir Shatrov)shatrov@me.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/84230/diff?detail_id=56371">diff</a>)</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=843972020-02-27T06:07:30Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul><li><strong>Tracker</strong> changed from <i>Bug</i> to <i>Feature</i></li><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>Glass_saga (Masaki Matsushita)</i></li><li><strong>ruby -v</strong> deleted (<del><i>ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux]</i></del>)</li><li><strong>Backport</strong> deleted (<del><i>2.5: UNKNOWN, 2.6: UNKNOWN</i></del>)</li></ul><p>We discussed this issue at the dev-meeting, and it requires @Glass_saga's review.</p>
<p>Note:</p>
<ul>
<li>It is uninterruptable under a platform that getaddrinfo_a is unavailable, but this problem is not only this proposal but also <code>timeout:</code> option of <code>Addrinfo.getaddrinfo()</code>.</li>
<li>Interruptable version can be implemented without getaddrinfo_a: Creating pthread for getaddrinfo function and pthread_cancel when interrupted. Contribution is welcome.</li>
</ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=859212020-06-01T10:20:06Zkirs (Kir Shatrov)shatrov@me.com
<ul></ul><p>mame (Yusuke Endoh) wrote in <a href="#note-7">#note-7</a>:</p>
<blockquote>
<p>We discussed this issue at the dev-meeting, and it requires @Glass_saga's review.</p>
<p>Note:</p>
<ul>
<li>It is uninterruptable under a platform that getaddrinfo_a is unavailable, but this problem is not only this proposal but also <code>timeout:</code> option of <code>Addrinfo.getaddrinfo()</code>.</li>
<li>Interruptable version can be implemented without getaddrinfo_a: Creating pthread for getaddrinfo function and pthread_cancel when interrupted. Contribution is welcome.</li>
</ul>
</blockquote>
<p>Thanks for feedback!<br>
I've opened <a href="https://github.com/ruby/ruby/pull/3171" class="external">https://github.com/ruby/ruby/pull/3171</a> with the approach you've described. Please let me know if I miss anything.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=872212020-08-27T07:56:16ZGlass_saga (Masaki Matsushita)glass.saga@gmail.com
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Closed</i></li><li><strong>Target version</strong> set to <i>36</i></li></ul><p>merged: <a href="https://github.com/ruby/ruby/commit/2038cc6cab6ceeffef3ec3a765c70ae684f829ed" class="external">https://github.com/ruby/ruby/commit/2038cc6cab6ceeffef3ec3a765c70ae684f829ed</a></p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=872332020-08-27T16:17:20ZGlass_saga (Masaki Matsushita)glass.saga@gmail.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-1 priority-4 priority-default" href="/issues/16381">Feature #16381</a>: Accept resolv_timeout in Net::HTTP</i> added</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=877912020-09-29T03:37:28Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Target version</strong> changed from <i>36</i> to <i>3.0</i></li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=889332020-12-06T02:08:38Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p>I have some feedback:</p>
<ul>
<li>I wish we don't use strange abbreviations like "resolv_timeout" and use correct English like "resolve_timeout".</li>
<li>I'm not convinced that <code>getaddrinfo_a</code> is a good idea, it has a user-space thread pool and the implementation doesn't seem great.</li>
<li>We will introduce asynchronous DNS resolution using a scheduler hook anyway, for Ruby 3: <a href="https://github.com/bruno-/ruby/pull/1" class="external">https://github.com/bruno-/ruby/pull/1</a>
</li>
</ul>
<p>Adding timeouts as arguments is not particularly useful either. It's not particularly easy to compose timeouts or use a single timeout for multiple operations, and it makes the underlying implementation more complex.</p>
<p>That being said, I'm not against this PR. I'm just not sure it's worth the effort/complexity.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=889342020-12-06T05:16:26Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<ul></ul><p>ioquatix (Samuel Williams) wrote in <a href="#note-12">#note-12</a>:</p>
<blockquote>
<p>Adding timeouts as arguments is not particularly useful either. It's not particularly easy to compose timeouts or use a single timeout for multiple operations, and it makes the underlying implementation more complex.</p>
</blockquote>
<p>One way around this is to use a deadline instead of a timeout:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># or something based on Process.clock_gettime</span>
<span class="n">deadline</span> <span class="o">=</span> <span class="no">Time</span><span class="p">.</span><span class="nf">now</span> <span class="o">+</span> <span class="mi">5</span>
<span class="no">Addrinfo</span><span class="p">.</span><span class="nf">getaddrinfo</span><span class="p">(</span><span class="s2">"www.ruby-lang.org"</span><span class="p">,</span> <span class="s2">"http"</span><span class="p">,</span> <span class="ss">deadline: </span><span class="n">deadline</span><span class="p">)</span>
<span class="no">Addrinfo</span><span class="p">.</span><span class="nf">getaddrinfo</span><span class="p">(</span><span class="s2">"docs.ruby-lang.org"</span><span class="p">,</span> <span class="s2">"http"</span><span class="p">,</span> <span class="ss">deadline: </span><span class="n">deadline</span><span class="p">)</span>
<span class="no">Addrinfo</span><span class="p">.</span><span class="nf">getaddrinfo</span><span class="p">(</span><span class="s2">"bugs.ruby-lang.org"</span><span class="p">,</span> <span class="s2">"http"</span><span class="p">,</span> <span class="ss">deadline: </span><span class="n">deadline</span><span class="p">)</span>
</code></pre>
<p>I think this approach is much better than a <code>Timeout.timeout</code> block. For one, <code>Timeout.timeout</code> requires the use of <code>Thread#raise</code> and results in less deterministic behavior. Additionally, a deadline option can potentially use a different approach than raising an exception, similar to <code>IO#read_nonblock</code>.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=895522020-12-27T09:32:34Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-1 priority-4 priority-default" href="/issues/17134">Feature #17134</a>: Add resolv_timeout to TCPSocket</i> added</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=895542020-12-27T09:45:17Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Status</strong> changed from <i>Closed</i> to <i>Open</i></li><li><strong>Target version</strong> changed from <i>3.0</i> to <i>3.1</i></li></ul><p>ioquatix (Samuel Williams) wrote in <a href="#note-12">#note-12</a>:</p>
<blockquote>
<ul>
<li>I'm not convinced that getaddrinfo_a is a good idea, it has a user-space thread pool and the implementation doesn't seem great.</li>
</ul>
</blockquote>
<p><a href="https://github.com/ruby/ruby/commit/2038cc6cab6ceeffef3ec3a765c70ae684f829ed" class="external">https://github.com/ruby/ruby/commit/2038cc6cab6ceeffef3ec3a765c70ae684f829ed</a> is reverted because of [Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Rails Active Job integration test fails with Ruby 3.0.0 since 2038cc6cab6ceeffef3ec3a765c70ae684f... (Closed)" href="https://bugs.ruby-lang.org/issues/17220">#17220</a>].</p>
<p>Since getaddrinfo doesn't provide nonblocking version, I think we need our own implementation of getaddrinfo_a.</p>
<blockquote>
<p>Adding timeouts as arguments is not particularly useful either. It's not particularly easy to compose timeouts or use a single timeout for multiple operations, and it makes the underlying implementation more complex.</p>
</blockquote>
<p>In my experience timeout is important for web applications to return a response when a DNS resolution is too slow.<br>
It's a long requested series of improvements for HTTP client like read_timeout, connect_timeout, and write_timeout.<br>
resolv_timeout is the last piece of that.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=895812020-12-27T18:27:37ZDan0042 (Daniel DeLorme)
<ul></ul><p>naruse (Yui NARUSE) wrote in <a href="#note-15">#note-15</a>:</p>
<blockquote>
<p>It's a long requested series of improvements for HTTP client like read_timeout, connect_timeout, and write_timeout.<br>
resolv_timeout is the last piece of that.</p>
</blockquote>
<p>How common is it to need separate timeouts for all of these? I can easily imagine the resolv_timeout as being part of the connect_timeout (i.e. resolv+connect may not exceed connect_timeout). And at least in my experience I usually want a single "general timeout" for resolv+connect+write+read. For that reason I <em>really</em> like Jeremy's suggestion of a single <code>deadline</code> argument. You can even have a single deadline for multiple http requests. This is really perfect for most uses cases that I'm familiar with.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=895992020-12-28T10:41:36Znaruse (Yui NARUSE)naruse@airemix.jp
<ul></ul><p>Dan0042 (Daniel DeLorme) wrote in <a href="#note-16">#note-16</a>:</p>
<blockquote>
<p>naruse (Yui NARUSE) wrote in <a href="#note-15">#note-15</a>:</p>
<blockquote>
<p>It's a long requested series of improvements for HTTP client like read_timeout, connect_timeout, and write_timeout.<br>
resolv_timeout is the last piece of that.</p>
</blockquote>
<p>How common is it to need separate timeouts for all of these? I can easily imagine the resolv_timeout as being part of the connect_timeout (i.e. resolv+connect may not exceed connect_timeout). And at least in my experience I usually want a single "general timeout" for resolv+connect+write+read. For that reason I <em>really</em> like Jeremy's suggestion of a single <code>deadline</code> argument. You can even have a single deadline for multiple http requests. This is really perfect for most uses cases that I'm familiar with.</p>
</blockquote>
<p>Both <code>Timeout.timeout</code> and <code>deadline</code> is not the essential problem of this topic. The topic this ticket handles is "getaddrinfo is not interruptable nor timeout ready". This is because getaddrinfo(3) doesn't have timeout, async, nor nonblocking API. This ticket intends to provide the underlying implementation of such feature.</p> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=980322022-06-16T01:08:11Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Target version</strong> changed from <i>3.1</i> to <i>3.2</i></li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=1001402022-11-17T06:41:14Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Target version</strong> deleted (<del><i>3.2</i></del>)</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=1051412023-11-02T07:51:20Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/19965">Feature #19965</a>: Make the name resolution interruptible</i> added</li></ul> Ruby master - Feature #16476: Socket.getaddrinfo cannot be interrupted by Timeout.timeouthttps://bugs.ruby-lang.org/issues/16476?journal_id=1051432023-11-02T08:07:36Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p><a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Make the name resolution interruptible (Closed)" href="https://bugs.ruby-lang.org/issues/19965">#19965</a> solved the problem, so I'll close it.</p>
<p>I completely forgot this ticket, but the concept (pthread_create on every call to getaddrinfo(3)) was the same as in <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Make the name resolution interruptible (Closed)" href="https://bugs.ruby-lang.org/issues/19965">#19965</a>. Just for case, <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Make the name resolution interruptible (Closed)" href="https://bugs.ruby-lang.org/issues/19965">#19965</a> is a bit more elaborate:</p>
<ul>
<li>It works as before in environments where pthread is not available.</li>
<li>It includes the same hack for not only <code>getaddrinfo(3)</code> but also <code>getnameinfo(3)</code>.</li>
<li>
<code>pthread_cancel</code> is not used because until recently <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1405071" class="external">there was a bug in glibc</a> where <code>pthread_cancel</code> of a pthread that is calling <code>getaddrinfo(3)</code> would cause subsequent name resolution to stick:</li>
<li>It attempts to use <code>pthread_setaffinity_np</code> to reduce speed degradation.</li>
</ul>