https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112017-10-12T23:41:28ZRuby Issue Tracking SystemRuby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=671982017-10-12T23:41:28Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p><a href="mailto:Greg.mpls@gmail.com" class="email">Greg.mpls@gmail.com</a> wrote:</p>
<blockquote>
<p><a href="https://bugs.ruby-lang.org/issues/14011" class="external">https://bugs.ruby-lang.org/issues/14011</a></p>
</blockquote>
<p>Maybe usa can help...</p>
<p>Or maybe sprinkle `warn "#{<strong>LINE</strong>} ...\n"' in the test code and<br>
webrick source to see where it stops?</p>
<blockquote>
<p>I noticed the recent <a href="https://ci.appveyor.com/project/ruby/ruby/build/1.0.5475#L4546" class="external">Appveyor failure</a> from <a href="https://github.com/ruby/ruby/commit/feafe07874d3f9f9aa3fa007421ecd28472959a7" class="external">‘webrick: do not hang acceptor on slow TLS connections’ svn 60172</a>.</p>
</blockquote>
<p>Is that appveyor link supposed to show test output? I don't use<br>
JavaScript or GUIs. Is there plain text or simple HTML log output?</p>
<blockquote>
<p>I’m happy to help with testing, but I suspect this is beyond<br>
my current knowledge in terms of a fix…</p>
</blockquote>
<p>Mainly, it is breaking down steps and putting warn/fprintf<br>
statements everywhere to figure out what's running or stuck.</p>
<p>There's probably platform-specific tracing tools you can use,<br>
too.</p>
<blockquote>
<p>FYI, I did run up puma locally with a static rack, http, and</p>
</blockquote>
<p>Did you mean https?</p>
<blockquote>
<p>trunk; everything works fine. I did not write code to connect<br>
it with multiple concurrent clients (I’ve done that before<br>
back when I was coding in C#). I also understand that some<br>
things may ‘just not work’ on windows.</p>
</blockquote>
<p>If you meant "https" above, try only making one TCP connection<br>
without the SSL/TLS handshake; then sleep the process:</p>
<pre><code>c = TCPSocket.new(host, port)
# no OpenSSL stuff
sleep # forever
</code></pre>
<p>Then, try to make normal HTTPS connections from a different<br>
terminal or browser and see if it works or gets stuck.</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=671992017-10-13T00:16:49ZMSP-Greg (Greg L)
<ul><li><strong>File</strong> <a href="/attachments/6795">trunk_mingw_puma.txt</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/6795/trunk_mingw_puma.txt">trunk_mingw_puma.txt</a> added</li><li><strong>File</strong> <a href="/attachments/6794">trunk_mswin_webrick.txt</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/6794/trunk_mswin_webrick.txt">trunk_mswin_webrick.txt</a> added</li></ul><p>Eric,</p>
<p>Thanks for the quick response. I've attached the two logs. I really hate the posts where people have a gazillion lines of a seg fault report...</p>
<p>Re this issue, it seems that it (mostly) involves code in ruby openssl or net/*, not code that's part of either webrick or puma.</p>
<p>I did mean http. IOW, puma set as a non ssl server works fine run on a mingw trunk build with a Chrome client. When I try to change it over to ssl (set verify_mode to none on client and server, provide key and cert files, etc), the client I set up in code can't connect. Similar 'host disconnected' errors.</p>
<p>Hope this helps. As to debugging, as it seems that the server is failing/closing, I haven't looked for how I can trap that. Reminder, I'm not a c type...</p>
<p>Thanks, Greg</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=672022017-10-13T00:51:20Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p><a href="mailto:Greg.mpls@gmail.com" class="email">Greg.mpls@gmail.com</a> wrote:</p>
<blockquote>
<p>Re this issue, it seems that it (mostly) involves code in ruby<br>
openssl ot net/*, not code that's part of either webrick or<br>
puma.</p>
</blockquote>
<p>OK, so the puma failures are independent of my webrick change<br>
(r60172), correct? Also, keep in mind puma uses it's own<br>
"minissl" C extension which could introduce more variables.</p>
<p>But the ECONNRESET failure in<br>
TestNetHTTPS#test_certificate_verify_failure is new and probably<br>
the result of my webrick change?</p>
<p>Does it still happen if you remove the WEBRick::Utils.timeout<br>
call I added and just call `sock.accept'?</p>
<blockquote>
<p>I did mean http. IOW, puma set as a non ssl server works fine<br>
run on a mingw trunk build with a Chrome client. When I try<br>
to change it over to ssl (set verify_mode to none on client<br>
and server, provide key and cert files, etc), the client I set<br>
up in code can't connect. Similar 'host disconnected' errors.</p>
</blockquote>
<p>Weird, I was about to suggest multi-threading problems with the<br>
client and server sharing process memory; but Chrome and Ruby<br>
are already different processes...</p>
<blockquote>
<p>Hope this helps. As to debugging, as it seems that the server<br>
is failing/closing, I haven't looked for how I can trap that.<br>
Reminder, I'm not a c type...</p>
</blockquote>
<p>Mainly, it's down to being stubborn and persistent :></p>
<p>Nobody starts off knowing C. I started out putting *printf<br>
statements in various places and editing code the same way I'd<br>
edit a config file until I got the results or info I wanted.</p>
<p>But I don't even think you'd have to deal with C, since I<br>
only changed the Ruby code for webrick.</p>
<p>Hopefully this gets figured out soon or usa or someone else<br>
can chime in. I'll be offline for a few days soonish.</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=672082017-10-13T01:54:30ZMSP-Greg (Greg L)
<ul></ul><p>normalperson (Eric Wong) wrote:</p>
<blockquote>
<p>OK, so the puma failures are independent of my webrick change<br>
(r60172), correct? Also, keep in mind puma uses it's own<br>
"minissl" C extension which could introduce more variables.</p>
</blockquote>
<p>Yes. They happened before your commit. I mentioned them just because 'nonblock' in ruby std-lib code seems common to both...</p>
<blockquote>
<p>But the ECONNRESET failure in<br>
TestNetHTTPS#test_certificate_verify_failure is new and probably<br>
the result of my webrick change?</p>
</blockquote>
<p>Yes. It's failing both on the Appveyor mswin build and my local MinGW build.</p>
<blockquote>
<p>Does it still happen if you remove the WEBRick::Utils.timeout<br>
call I added and just call `sock.accept'?</p>
</blockquote>
<p>I've think you're referring to:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">if</span> <span class="n">sock</span><span class="p">.</span><span class="nf">respond_to?</span><span class="p">(</span><span class="ss">:sync_close</span><span class="o">=</span><span class="p">)</span> <span class="o">&&</span> <span class="vi">@config</span><span class="p">[</span><span class="ss">:SSLStartImmediately</span><span class="p">]</span>
<span class="no">WEBrick</span><span class="o">::</span><span class="no">Utils</span><span class="p">.</span><span class="nf">timeout</span><span class="p">(</span><span class="vi">@config</span><span class="p">[</span><span class="ss">:RequestTimeout</span><span class="p">])</span> <span class="k">do</span>
<span class="n">sock</span><span class="p">.</span><span class="nf">accept</span> <span class="c1"># OpenSSL::SSL::SSLSocket#accept</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>Commented out 2 & 4 (start from 1), still have the failure, then commented out all but 3, still the failure.</p>
<blockquote>
<p>Hopefully this gets figured out soon or usa or someone else<br>
can chime in. I'll be offline for a few days soonish.</p>
</blockquote>
<p>Offline is good sometimes... Thanks.</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=672102017-10-13T02:18:22ZMSP-Greg (Greg L)
<ul></ul><p>From reading normalperson's (Eric Wong's) description in <a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/60172" class="external">60172</a>, it sounds like this change may have shown an issue involving 'per-client threading' on Windows builds.</p>
<p>From my testing with Puma, this issue seems to exist in 2.3 and 2.4 also. Hence, if a fix is possible, please consider backporting to those versions.</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=672152017-10-13T04:14:33ZMSP-Greg (Greg L)
<ul><li><strong>ruby -v</strong> changed from <i>ruby 2.5.0dev (2017-10-12 trunk 60170) [x64-mingw32]</i> to <i>ruby 2.5.0dev (2017-10-13 trunk 60172) [x64-mingw32]</i></li></ul><p>My rolling Appveyor trunk build completed. It had the same failure as the mswin builds currently have. So the next builds pass, I may reverse this until it's resolved.</p>
<p>Summary:</p>
<pre><code>1 Total Failures/Errors Build No 44 Job Id e7s5dw24gghj56hb
ruby 2.5.0dev (2017-10-13 trunk 60175) [x64-mingw32]
2017-10-13 03:31:29 UTC
test-all 16954 tests, 2236388 assertions, 1 failures, 0 errors, 117 skips, 85 skips shown
test-spec 3554 files, 26057 examples, 203541 expectations, 0 failures, 0 errors, 0 tagged
mspec 3554 files, 26057 examples, 203471 expectations, 0 failures, 0 errors, 0 tagged
test-basic test succeeded
btest PASS all 1194 tests
</code></pre>
<p>Failure:</p>
<pre><code> 86) Failure:
TestNetHTTPS#test_certificate_verify_failure [C:/projects/ruby-loco/src/ruby/test/net/http/utils.rb:48]:
<[]> expected but was
<["[2017-10-13 03:22:01] ERROR Errno::ECONNRESET: An existing connection was forcibly closed by the remote host. - SSL_accept\n" +
"\tC:/projects/ruby-loco/src/ruby/lib/webrick/server.rb:298:in `accept'\n" +
"\tC:/projects/ruby-loco/src/ruby/lib/webrick/server.rb:298:in `block (2 levels) in start_thread'\n" +
"\tC:/projects/ruby-loco/src/ruby/lib/webrick/utils.rb:263:in `timeout'\n" +
"\tC:/projects/ruby-loco/src/ruby/lib/webrick/server.rb:297:in `block in start_thread'\n"]>.
</code></pre>
<p>Thanks, Greg</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=672192017-10-13T07:51:28Znagachika (Tomoyuki Chikanaga)nagachika00@gmail.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/14005">Bug #14005</a>: [PATCH] webrick: do not hang acceptor on slow TLS connections</i> added</li></ul> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=677542017-11-10T07:05:47Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>normalperson (Eric Wong)</i></li></ul> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=677692017-11-10T14:07:06ZMSP-Greg (Greg L)
<ul></ul><p>Hiroshi and Eric - I apologize for allowing this issue to remain open. I cannot find any indication that this is a problem with any current builds.</p>
<p>Okay to close.</p> Ruby master - Bug #14011: Appveyor failure - svn 60172 - ‘webrick: do not hang acceptor on slow TLS connections’https://bugs.ruby-lang.org/issues/14011?journal_id=677752017-11-10T23:55:57Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Rejected</i></li></ul>