https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-09-22T09:48:34ZRuby Issue Tracking SystemRuby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=992462022-09-22T09:48:34Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/99246/diff?detail_id=63156">diff</a>)</li></ul> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=993572022-09-27T03:49:18Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul></ul><p>I investigated this.</p>
<p><a href="https://github.com/ruby/syntax_suggest/blob/main/lib/syntax_suggest/api.rb#L67" class="external">https://github.com/ruby/syntax_suggest/blob/main/lib/syntax_suggest/api.rb#L67</a></p>
<pre><code> Timeout.timeout(timeout) do
record_dir ||= ENV["DEBUG"] ? "tmp" : nil
search = CodeSearch.new(source, record_dir: record_dir).call
end
</code></pre>
<p>This block raises <code>#<ThreadError: can't alloc thread></code> with Ruby 3.2.0-preview2 and master branch. When I commented-out this <code>Timeout.timeout</code>, syntax_suggest is working fine.</p>
<p>I bisected <code>timeout.rb</code>. After <a href="https://github.com/ruby/ruby/commit/89fbec224d8e1fa35e82bf2712c5a5fd3dc06b83" class="external">https://github.com/ruby/ruby/commit/89fbec224d8e1fa35e82bf2712c5a5fd3dc06b83</a> happens this <code>ThreadError</code> with syntax_suggest. I'm not sure why timeout.rb raises <code>ThreadError</code> when using syntax_suggest.</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=993582022-09-27T06:47:10Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul></ul><p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/18">@mame (Yusuke Endoh)</a> try to fix this <code>ThreadError</code> at <a href="https://github.com/ruby/ruby/pull/6452" class="external">https://github.com/ruby/ruby/pull/6452</a></p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=993612022-09-27T08:10:50Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>There are two different sub-issues in this issue.</p>
<a name="1-syntax_suggest-does-not-work-for-ruby-foorb"></a>
<h3 >1. syntax_suggest does not work for <code>ruby foo.rb</code>.<a href="#1-syntax_suggest-does-not-work-for-ruby-foorb" class="wiki-anchor">¶</a></h3>
<p>This is because a SyntaxError exception is not thrown even if there is a syntax error in foo.rb of <code>ruby foo.rb</code>. Instead, the interpreter prints the syntax error directly. So <code>Exception#detailed_message</code> is not called, which means syntax_suggest cannot intercept the exception handling.</p>
<p>This could be fixed by throwing a SyntaxError exception even if there is a parse error in foo.rb, but I don't know how hard this fix will be implemented.</p>
<a name="2-syntax_suggest-does-not-work-for-ruby-e-load-foorb"></a>
<h3 >2. syntax_suggest does not work for <code>ruby -e 'load "foo.rb"'</code>.<a href="#2-syntax_suggest-does-not-work-for-ruby-e-load-foorb" class="wiki-anchor">¶</a></h3>
<p>In this case, a SyntaxError exception is thrown, so <code>Exception#detailed_message</code> of syntax_suggest is actually called.<br>
However, <code>Exception#detailed_message</code> is called after the interpreter termination process has started.<br>
It is prohibited to create a new thread in this state, but syntax_suggest uses Timeout.timeout, which tries to create a new thread.<br>
This causes the failure <code>#<ThreadError: can't alloc thread></code> which <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/572">@hsbt (Hiroshi SHIBATA)</a> mentioned.</p>
<p><a href="https://github.com/ruby/ruby/pull/6452" class="external">My PR</a> changes the order so that <code>Exception#detailed_message</code> is called <em>before</em> the interpreter termination process.<br>
However, I am a little afraid that this change has unforeseen side effects because it will print an error message when other threads are still running.</p>
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/17">@ko1 (Koichi Sasada)</a> pointed that the following script prints <code>sub thread (RuntimeError)</code> and then <code>main thread (RuntimeError)</code> in Ruby 3.1, and that <code>main thread (RuntimeError)</code> and then <code>sub thread (RuntimeError)</code> with my PR applied.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="k">begin</span> <span class="nb">sleep</span><span class="p">;</span> <span class="k">ensure</span><span class="p">;</span> <span class="k">raise</span> <span class="s2">"sub thread"</span><span class="p">;</span> <span class="k">end</span> <span class="p">}</span>
<span class="k">raise</span> <span class="s2">"main thread"</span>
</code></pre>
<pre><code>$ ruby -v tt.rb
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux]
#<Thread:0x00007ffff3f0b2e8 tt.rb:1 aborting> terminated with exception (report_on_exception is true):
tt.rb:1:in `ensure in block in <main>': sub thread (RuntimeError)
from tt.rb:1:in `block in <main>'
tt.rb:2:in `<main>': main thread (RuntimeError)
</code></pre>
<pre><code>$ ./miniruby -v tt.rb
ruby 3.2.0dev (2022-09-27T06:08:48Z error_print-before.. 4e8618306b) [x86_64-linux]
tt.rb:2:in `<main>': main thread (RuntimeError)
#<Thread:0x00007ffff7408fb8 tt.rb:1 aborting> terminated with exception (report_on_exception is true):
tt.rb:1:in `ensure in block in <main>': sub thread (RuntimeError)
from tt.rb:1:in `block in <main>'
</code></pre> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=993692022-09-27T18:25:46ZEregon (Benoit Daloze)
<ul></ul><p>The current shutdown order in TruffleRuby is:</p>
<ol>
<li>execute main script (and save exception if any)</li>
<li>run at_exit hooks</li>
<li>print main script's exception with <code>.full_message</code>
</li>
<li>kill and wait all threads (which still runs some Ruby code in <code>ensure</code>)</li>
<li>internal cleanups which do not need to run Ruby code</li>
</ol>
<p>Which seems already the order with your change.<br>
I think we shouldn't run any Ruby code after "kill and wait all threads" (because the kill might have caused inconsistent state and triggered in the middle of an <code>ensure</code>, etc), so I think this change makes a lot of sense.</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=994932022-10-06T14:35:20Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>At the dev meeting, <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> proposed a slightly different order of termination process: (1) calls #detailed_message and keep the result string, (2) terminates all threads, and (3) prints the kept message.</p>
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> Do you create a patch?</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=994992022-10-06T16:02:38ZEregon (Benoit Daloze)
<ul></ul><p>Sounds OK-ish except I think <code>full_message</code> should be called instead of <code>detailed_message</code>.<br>
Also if terminating all threads takes a while it delays the printing significantly, that's not so nice.<br>
The order I posted above seems much cleaner.</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=995172022-10-07T21:44:37Zschneems (Richard Schneeman)
<ul></ul><p>Thank you all for looking at this issue. I have some more cycles now. I took a look at the explanation and the PR. I don't feel equipped to weigh in on the implementation. I understand the problem description and proposed solution(s).</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=1000742022-11-14T02:32:21Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Assignee</strong> changed from <i>hsbt (Hiroshi SHIBATA)</i> to <i>nobu (Nobuyoshi Nakada)</i></li></ul><p><code>syntax_suggest</code> is not working 3.2.0-preview3 too. <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> will resolve this until preview4.</p> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=1000802022-11-14T03:47:58Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Target version</strong> set to <i>3.2</i></li></ul> Ruby master - Bug #19016: syntax_suggest is not working with Ruby 3.2.0-preview2https://bugs.ruby-lang.org/issues/19016?journal_id=1001832022-11-20T13:44:08Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Closed</i></li></ul><p>Applied in changeset <a class="changeset" title="[Bug #19016] Handle syntax error in main script like other errors So that `SyntaxError#detailed_..." href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/701dfe4eb741628213e4d701c13ad6d76904ac4f">git|701dfe4eb741628213e4d701c13ad6d76904ac4f</a>.</p>
<hr>
<p>[Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: syntax_suggest is not working with Ruby 3.2.0-preview2 (Closed)" href="https://bugs.ruby-lang.org/issues/19016">#19016</a>] Handle syntax error in main script like other errors</p>
<p>So that <code>SyntaxError#detailed_message</code> will be used also in the case<br>
exiting by such syntax error.</p>