https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-01-19T11:14:40ZRuby Issue Tracking SystemRuby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=960512022-01-19T11:14:40Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/16038">Feature #16038</a>: Provide a public WeakMap that compares by equality rather than by identity </i> added</li></ul> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=960572022-01-19T15:36:10Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96057/diff?detail_id=61873">diff</a>)</li></ul><p>After another quick chat with <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> I removed one of the use cases because I made a mistake, it does require a <code>WeakValuesMap</code>. Leaving it in comment for posterity</p>
<a name="Deduplicating-constructors"></a>
<h4 >Deduplicating constructors.<a href="#Deduplicating-constructors" class="wiki-anchor">¶</a></h4>
<p>Can be used by large value object classes to automatically re-use existing instances:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">SomeValueObject</span>
<span class="no">CACHE</span> <span class="o">=</span> <span class="no">WeakKeysMap</span><span class="p">.</span><span class="nf">new</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">build</span><span class="p">(</span><span class="n">attributes</span><span class="p">)</span>
<span class="no">CACHE</span><span class="p">[</span><span class="n">attributes</span><span class="p">]</span> <span class="o">||=</span> <span class="n">new</span><span class="p">(</span><span class="n">attributes</span><span class="p">).</span><span class="nf">freeze</span> <span class="c1"># The instance hold the reference</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">attributes</span><span class="p">)</span>
<span class="vi">@attributes</span> <span class="o">=</span> <span class="n">attributes</span>
<span class="o">...</span> <span class="c1"># compute some expensive or large data</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=960602022-01-19T17:41:03ZEregon (Benoit Daloze)
<ul></ul><p>This looks great to me.</p>
<p>I prefer <code>WeakKeysMap</code>. <code>WeakRef::WeakKeysMap</code> sounds fine too.<br>
Anything with <code>Hash</code> in the name is IMHO not great because Hash implies ordering and Map no ordering, per well-established concurrent-ruby classes (Concurrent::Hash and Concurrent::Map).<br>
Having <code>WeakKeys</code> in the name seems best/useful, as there might be a <code>WeakValuesMap</code> in the future.</p>
<p>I believe <code>WeakKeysMap#dedup</code> should be a core method (i.e., predefined on WeakKeysMap), that way it can be guaranteed thread-safe on all Ruby implementations with minimal overhead.<br>
Doing it manually with Mutex when defined in Ruby would have a significant overhead.<br>
ObjectSpace::WeakMap is already thread-safe on CRuby, JRuby and TruffleRuby so it only makes sense WeakKeysMap is thread-safe as well.<br>
<code>WeakKeysMap#dedup</code> also makes that use-case much easier as the user does not need to define <code>dedup</code> themselves.</p>
<p>I would prefer no <code>compare_by_identity</code> support for simplicity.<br>
If <code>compare_by_identity</code> is included, such choice should be made when creating the WeakKeysMap (e.g. <code>WeakKeysMap.new(compare_by_identity: true)</code>), not later as that causes a whole lot of mess implementation-wise and makes the semantics unclear.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=964932022-02-15T04:46:55Zko1 (Koichi Sasada)
<ul></ul><p>Do you think this <code>Hash</code> compatible? For example, <code>default_proc</code> (I think it is too much)?</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=964942022-02-15T04:49:24Zko1 (Koichi Sasada)
<ul></ul><p>ko1 (Koichi Sasada) wrote in <a href="#note-4">#note-4</a>:</p>
<blockquote>
<p>Do you think this <code>Hash</code> compatible? For example, <code>default_proc</code> (I think it is too much)?</p>
</blockquote>
<p><code>ObjectSpace::WeakMap</code> is good starting point.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965002022-02-15T08:13:18Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>Do you think this Hash compatible? For example, default_proc (I think it is too much)?</p>
</blockquote>
<p>Yes, I'd rather not ask for methods that may cause difficulties in the future.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965032022-02-15T14:08:07ZDan0042 (Daniel DeLorme)
<ul></ul><p>If ObjectSpace::WeakMap had a #compare_by_identity that could be set to false, wouldn't this be equivalent to this WeakKeysMap? (as long as you use non-GC-able values only)</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965052022-02-15T14:42:26Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>as long as you use non-GC-able values only</p>
</blockquote>
<p>Then yes. But it's rarely convenient.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965092022-02-15T19:40:33ZDan0042 (Daniel DeLorme)
<ul></ul><p>byroot (Jean Boussier) wrote in <a href="#note-8">#note-8</a>:</p>
<blockquote>
<p>Then yes. But it's rarely convenient.</p>
</blockquote>
<p>Can you elaborate? Because the DeduplicationSet example has <code>@set[object] = true</code>, which is exactly a non-GC-able value.</p>
<p>And the METADATA example is not so convincing since it makes more sense to have metadata per object (identity); I can't imagine a use case where you'd want to share metadata for objects that are eql.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965102022-02-15T20:17:03Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>I can't imagine a use case where you'd want to share metadata for objects that are eql.</p>
</blockquote>
<p>As you certainly know, <code>Object#==</code> defaults to <code>Object#equal?</code>, so generally equality hashes are workable in scenarios where you actually need identity, the opposite isn't true. Hence why equality by default make sense.</p>
<p>Additionally <code>ObjectSpace::WeakMap</code> is not really made to be public and is specifically tailored to be the backup store for <code>WeakRef</code>.</p>
<blockquote>
<p>I can't imagine a use case where you'd want to share metadata for objects that are eql.</p>
</blockquote>
<p>A common case would be something like <code>CACHE = { "https://example.com/foo" => "HTML_BODY" }</code>. Various "workers" could share such "http cache" without worrying about the lifecycle of the objects etc.</p>
<p>Same thing if you want to safely store some thread local data: <code>DATA = { Thread.current => {key: "value"}}</code>.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965442022-02-17T13:16:59Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>I agree with introducing a map (hash) with each key referenced weakly. It should be in the core since weak references should be tightly integrated with the garbage collector.</p>
<p>Since the current <code>WeakMap</code> is hard to implement in JRuby, I also agree with making the existing <code>WeakMap</code> obsolete (gradually).</p>
<p>We need to determine the detail:</p>
<ul>
<li>
<strong>Where to place the class</strong>: we have several options. I vote for putting it under <code>ObjectSpace</code> as the current <code>WeakMap</code>
</li>
<li>
<strong>The name of the class</strong>: OP proposed <code>WeakKeysMap</code>. I prefer <code>WeakKeyMap</code> probably because my native language (Japanese) does not have noun plurality. But considering the class is "a map with each key referenced weakly", still <code>WeakKeyMap</code> looks better to me.</li>
<li>
<strong>Behavior</strong>: we should start with minimal set of methods. How about:
<ul>
<li><code>ObjectSpace::WeakKeyMap#[](key)</code></li>
<li><code>ObjectSpace::WeakKeyMap#[]=(key, val)</code></li>
<li><code>ObjectSpace::WeakKeyMap#getkey(key)</code></li>
<li><code>ObjectSpace::WeakKeyMap#inspect #=> #<ObjectSpace::WeakKeyMap size=100></code></li>
</ul>
</li>
</ul>
<p>Matz.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965472022-02-17T13:57:39Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>Thank you Matz!</p>
<blockquote>
<p>I vote for putting it under ObjectSpace</p>
</blockquote>
<p>I think it's fine. It's an advanced class, no need to pollute the main namespace with it.</p>
<blockquote>
<p>we should start with minimal set of methods.</p>
</blockquote>
<p>Agreed. A couple points though:</p>
<ul>
<li>I agree that <code>#each</code> isn't desirable, and tricky to implement correctly.</li>
<li>However <code>#keys</code> would be simpler to implement, and very useful for debugging purposes. If we can't introspect that is in the Hash, some bugs might be hard to resolve.</li>
</ul>
<p>From the meeting notes:</p>
<blockquote>
<p>mame: “Third party object extension” might be impossible without compare_by_identity. We need to confirm the OP if it is okay</p>
</blockquote>
<p>I don't think it's a problem in practice. Except for "value objects" <code>==</code> usually has identity semantic. I wouldn't object to a <code>compare_by_identity</code> option, but I don't think it's a dealbreaker if it's not there.</p>
<blockquote>
<p>ko1: key(key)?</p>
</blockquote>
<p>That would have been my preferred name too, but it's already defined on <code>Hash</code> with a different semantic, so I think it would be confusing. <code>#getkey(key)</code> is a good compromise I think.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965722022-02-18T14:01:07Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>I took a stab at implementing the current spec: <a href="https://github.com/ruby/ruby/pull/5570" class="external">https://github.com/ruby/ruby/pull/5570</a></p>
<a name="Methods"></a>
<h3 >Methods<a href="#Methods" class="wiki-anchor">¶</a></h3>
<ul>
<li><code>ObjectSpace::WeakKeyMap#[](key)</code></li>
<li><code>ObjectSpace::WeakKeyMap#[]=(key, val)</code></li>
<li><code>ObjectSpace::WeakKeyMap#getkey(key)</code></li>
<li><code>ObjectSpace::WeakKeyMap#size</code></li>
<li><code>ObjectSpace::WeakKeyMap#clear</code></li>
<li><code>ObjectSpace::WeakKeyMap#inspect #=> #<ObjectSpace::WeakKeyMap size=100></code></li>
</ul>
<a name="For-"></a>
<h3 >For <code>#[]=</code>:<a href="#For-" class="wiki-anchor">¶</a></h3>
<ul>
<li>Trying to use an immediate as a key fail with <code>ArgumentError</code>
</li>
<li>For consistency, <code>BigNum</code> and dynamic Symbol also raise <code>ArgumentError</code>
</li>
<li>Using an object without a <code>#hash</code> method raise <code>NoMethodError</code> like a regular hash.</li>
</ul>
<a name="Notes"></a>
<h3 >Notes<a href="#Notes" class="wiki-anchor">¶</a></h3>
<p>For simplicity, I took a shortcut. When a key is finalized we can no longer hash it, so I traverse the table in <code>O(N)</code> and delete entries pointing to the same object.</p>
<p>I'm not certain yet what's the best solution for this, but I assume we can record the hash inside a secondary <code>table</code> with identity semantic.</p>
<p>But this is good enough to experiment with the defined interface and see if there is problems with the spec itself.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965932022-02-20T15:56:26Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>What is <code>#size</code> method needed for? It may be error-prune to depend on a return value of the method because the size may change immediately after the method is called. For debugging, <code>#inspect</code> includes the size information. For memory usage analysis, <code>ObjectSpace.memsize_of</code> would be a better API.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=965942022-02-20T16:06:14Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/18">@mame (Yusuke Endoh)</a>, I think you convinced me, I'll remove it.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=1006562022-12-15T07:23:46Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>LGTM. Probably too late for 3.2. For 3.3 maybe?</p>
<p>Matz.</p> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=1006752022-12-15T09:02:56Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul><li><strong>Target version</strong> set to <i>3.3</i></li></ul> Ruby master - Feature #18498: Introduce a public WeakKeysMap that compares by equalityhttps://bugs.ruby-lang.org/issues/18498?journal_id=1020012023-02-23T15:02:18Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>Applied in changeset <a class="changeset" title="Implement ObjectSpace::WeakKeyMap basic allocator [Feature #18498]" href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/2a5354e59324cb296a423c73ec15ff9191086964">git|2a5354e59324cb296a423c73ec15ff9191086964</a>.</p>
<hr>
<p>Implement ObjectSpace::WeakKeyMap basic allocator</p>
<p>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Introduce a public WeakKeysMap that compares by equality (Closed)" href="https://bugs.ruby-lang.org/issues/18498">#18498</a>]</p>