https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112018-05-08T03:12:57ZRuby Issue Tracking SystemRuby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=719032018-05-08T03:12:57Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>Investigating</p> Ruby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=719052018-05-08T03:42:33Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>Yes, this is an old bug and not caused by my changes, I guess.<br>
It requires reworking some internal data structures in<br>
variable.c to provide feature mapping to autoload_data_i<br>
so different autoloads referring to the same file (aka "feature")<br>
can safely wait on each other.</p>
<p>Will work on this tomorrow, I need sleep :<</p> Ruby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=719232018-05-10T01:19:26Znormalperson (Eric Wong)normalperson@yhbt.net
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>Applied in changeset trunk|r63387.</p>
<hr>
<p>variable.c: fix multiple autoload with identical file</p>
<p>We need to ensure autoload declarations pointing to the same<br>
feature (aka "file") can wait on each other to avoid deadlock<br>
situations.</p>
<p>So, reorganize autoload data structures to maintain a<br>
feature => autoload_data_i mapping, and have module constant<br>
tables point to the new autoload_const struct instead of<br>
directly to autoload_data_i. This allows multiple<br>
autoload_const structs to refer to the SAME autoload_data_i<br>
struct, and with it, the on-stack autoload_state.waitq.</p>
<p>The end result is different constants can share the same waitq<br>
(tied to the feature name), and not deadlock each other during<br>
loading.</p>
<p>Thanks to Eugene Kenny for the bug report and reproducible test case.</p>
<p>Reported-by: Eugene Kenny <a href="mailto:elkenny@gmail.com" class="email">elkenny@gmail.com</a></p>
<ul>
<li>variable.c (autoload_featuremap): new global<br>
(struct autoload_const): new per-const struct<br>
(struct autoload_state): reference autoload_const instead of autoload_data_i<br>
(struct autoload_data_i): remove per-const<br>
(autoload_i_mark): delete from autoload_featuremap if unreferenced<br>
(autoload_c_mark): new dmark callback<br>
(autoload_c_free): new dfree callback<br>
(autoload_c_memsize): new memsize callback<br>
(autoload_const_type): new data type<br>
(get_autoload_data): set autoload_const as well<br>
(rb_autoload_str): use new data structures<br>
(autoload_delete): cleanup from autoload_featuremap<br>
(check_autoload_required): adjust for new internals<br>
(rb_autoloading_value): ditto<br>
(struct autoload_const_set_args): remove, redundant with autoload_const<br>
(const_tbl_update): adjust for new internals<br>
(autoload_const_set): ditto<br>
(autoload_require): ditto<br>
(autoload_reset): ditto<br>
(rb_autoload_load): ditto<br>
(rb_const_set): ditto<br>
(current_autoload_data): ditto<br>
(set_const_visibility): ditto</li>
<li>test/ruby/test_autoload.rb (test_autoload_same_file): new test<br>
<a href="/issues/14742">[ruby-core:86935]</a> [Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Deadlock when autoloading different constants in the same file from multiple threads (Closed)" href="https://bugs.ruby-lang.org/issues/14742">#14742</a>]</li>
</ul> Ruby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=719242018-05-10T01:32:55Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>Sorry for the delay, r63387 should fix it.</p> Ruby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=719282018-05-10T05:12:48Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>Eric Wong <a href="mailto:normalperson@yhbt.net" class="email">normalperson@yhbt.net</a> wrote:</p>
<blockquote>
<p>Sorry for the delay, r63387 should fix it.</p>
</blockquote>
<p>Reverted for now. Some problems need fixing but I can't reproduce<br>
on my 32-bit system (and my 64-bit machines have problems).</p> Ruby master - Bug #14742: Deadlock when autoloading different constants in the same file from multiple threadshttps://bugs.ruby-lang.org/issues/14742?journal_id=722492018-05-25T16:09:03Zeugeneius (Eugene Kenny)
<ul></ul><p>It looks like the fix was un-reverted in r63392. Thank you for working on this, Eric!</p>