https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-02-16T18:45:32ZRuby Issue Tracking SystemRuby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965162022-02-16T18:45:32Zko1 (Koichi Sasada)
<ul></ul><p>Current design is the global counter doesn't change frequently.<br>
Do you have measurements about it on some apps?</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965172022-02-16T19:10:04Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul></ul><p>At the moment on Shopify's core monolith we're seeing around 1 in 30 requests invalidate the global cache. We're still working out the source of the invalidations. But at the moment with the current design if anything changes anywhere everything is invalidated. So unfortunately it happens quite frequently.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965182022-02-16T19:34:14ZEregon (Benoit Daloze)
<ul></ul><p>During startup, global invalidation for constants also causes a lot of extra lookups, and with a JIT it throws away a lot of code during startup (or the JIT can't inline the value of the constant).</p>
<p>Global per-name constant invalidation is also what JRuby does IIRC.</p>
<p>This is what we did in TruffleRuby, it's per class and constant/method name so it's quite precise (same general approach for method & constant lookup):<br>
<a href="https://medium.com/graalvm/precise-method-and-constant-invalidation-in-truffleruby-4dd56c6bac1a" class="external">https://medium.com/graalvm/precise-method-and-constant-invalidation-in-truffleruby-4dd56c6bac1a</a><br>
It has the advantage to not invalidate needlessly when e.g. two modules have a constant with the same name.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965402022-02-17T12:30:15Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>Amusingly enough, this discussion led me to instrument our production environment to see what is bumping the cache, and one of the big offenders is <code>open-uri</code>: <a href="https://github.com/ruby/open-uri/blob/174a8eb7de357fc04c0675dd30073c0218f401a5/lib/open-uri.rb#L415-L417" class="external">https://github.com/ruby/open-uri/blob/174a8eb7de357fc04c0675dd30073c0218f401a5/lib/open-uri.rb#L415-L417</a></p>
<p>Another big offender is <code>tzinfo</code>: <a href="https://github.com/tzinfo/tzinfo/pull/129" class="external">https://github.com/tzinfo/tzinfo/pull/129</a></p>
<p>(I'm only mentioning sources that aren't specific to our application).</p>
<p>Here's the modified Ruby I used to find the source of the bumps: <a href="https://github.com/Shopify/ruby/commit/7aad79590dd62c05ba3b65d1964dc80f147441b6" class="external">https://github.com/Shopify/ruby/commit/7aad79590dd62c05ba3b65d1964dc80f147441b6</a></p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965712022-02-18T08:47:46Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>I have a fix for <code>open-uri</code>: <a href="https://github.com/ruby/open-uri/pull/8" class="external">https://github.com/ruby/open-uri/pull/8</a></p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=965732022-02-18T14:10:14ZDan0042 (Daniel DeLorme)
<ul></ul><blockquote>
<p>It increased total retained memory from 1.23Gb to 1.3Gb (about a 0.7% increase).</p>
</blockquote>
<p>1.3 / 1.23 is a 5.7% increase, not 0.7%</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=966862022-02-28T14:46:33Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>So I've been digging more on the constant cache busts happening in our app, beside what I already mentioned, what I found is:</p>
<p>A large majority of the busts are caused by autoloaded constants in gems. A few shouldn't autoload at all, but in many case is because the gem provide multiple implementation of an interface, and doesn't want to load them all. So there isn't really a good fix for this. Even the <code>rack</code> gem is fully autoloaded. This means that any web application is bound to invalidate the constant cache on the first few requests it processes.</p>
<p>Then I found some more APIs using <code>Object#extend</code> like <code>open-uri</code>. These are more of a problem, because they bust the cache every time they're called, not just the first time.</p>
<p>And since currently Ruby doesn't offer much API to diagnose this problem, it's unlikely to get better any time soon. So maybe the extra memory usage is worth it?</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967002022-03-04T15:05:49Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>Do you have measurements about it on some apps?</p>
</blockquote>
<p>So here's some metrics from the app I investigated. The metric is not directly <code>RubyVM.stat(:global_constant_state)</code>, instead we emit an increment if the <code>global_constant_state</code> changed during a request cycle.</p>
<p>The straight line is the last 24h, and the dotted line is the same days 4 weeks prior:</p>
<p><img src="https://user-images.githubusercontent.com/19192189/156726006-75aab77a-7fdf-47cf-88cb-1175f193c18a.png" alt="Constant Caches"></p>
<p>The number one cause by far was <code>Object#extend</code> like in <a href="https://github.com/ruby/open-uri/pull/8" class="external"><code>open-uri</code></a>, <a href="https://github.com/aws/aws-sdk-ruby/pull/2670" class="external"><code>aws-sdk</code></a> as well as <code>ActiveRecord::Reation#extending</code>.</p>
<p>The second most common cause was gems using a lot of <code>autoload</code>, or having memoized class attribute like <a href="https://github.com/tzinfo/tzinfo/pull/129" class="external">tzinfo</a>.</p>
<p>Overall fixing these had a very significant impact on the service latency</p>
<p><img src="https://user-images.githubusercontent.com/19192189/156727814-adb0f8b5-9012-4d2c-ab9c-b29d80748a5c.png" alt="Latency"></p>
<p>I'm however worried that this situation will degrade over time, as there's very little way to actively prevent this kind of problems from cropping up.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967032022-03-04T17:49:24Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96703/diff?detail_id=62108">diff</a>)</li></ul> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967042022-03-04T18:17:49Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96704/diff?detail_id=62109">diff</a>)</li></ul> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967392022-03-10T02:17:28Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96739/diff?detail_id=62123">diff</a>)</li></ul> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967402022-03-10T02:18:27Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul></ul><p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/11019">@Dan0042 (Daniel DeLorme)</a> yeah sorry, I was looking at different numbers and got wires crossed.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967412022-03-10T02:47:06Zkddnewton (Kevin Newton)kddnewton@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96741/diff?detail_id=62124">diff</a>)</li></ul> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967552022-03-10T13:40:35ZEregon (Benoit Daloze)
<ul></ul><p>What's the memory overhead of this? (probably the biggest concern from CRuby's side)</p>
<p>A 5.7% increase does sound like a lot for this.<br>
But it seems the description now says 1% for the monolith?<br>
What's the percentage for railsbench?</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967562022-03-10T13:45:59Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><p>@kddeisz is away for a few days, so I'll take the liberty to answer even though he may correct me later.</p>
<blockquote>
<p>A 5.7% increase does sound like a lot for this. But it seems the description now says 1% for the monolith?</p>
</blockquote>
<p>The initial measurement that was showing the 5.7% increase was flawed, it was actually much less. Sorry about that, it's not that trivial to measure.<br>
Also thanks to <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/11657">@jhawthorn (John Hawthorn)</a> the patch memory usage was reduced even further, hence the ~1%.</p>
<p>Also after chatting a bit yesterday, we believe that in practice this will likely save memory in forking environments, because the finer grained cache will mean less ISeq being written into when a single constant change, so less CoW invalidations. But of course that's heavily dependent on the app.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=967662022-03-10T17:29:57Zjhawthorn (John Hawthorn)
<ul></ul><p>Tested this patch out on GitHub's largest app and the size of the additional constant cache bookkeeping was only ~3MB (as measured by <code>vm_memsize_constant_cache</code>) for our ~950MB application.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=968452022-03-15T07:39:08Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>I'm not against the proposal, but for the record, the change makes <code>Object#extend</code> and <code>Module#include</code> slow.</p>
<p>Before the patch:</p>
<pre><code>$ time ./miniruby -e 'module M; A=B=C=D=E=F=G=H=I=J=K=L=M=N=O=P=Q=R=S=T=U=V=W=X=Y=Z=1; end; 1000000.times { Object.new.extend(M) }'
real 0m1.002s
user 0m0.985s
sys 0m0.016s
</code></pre>
<p>After the patch:</p>
<pre><code>$ time ./miniruby -e 'module M; A=B=C=D=E=F=G=H=I=J=K=L=M=N=O=P=Q=R=S=T=U=V=W=X=Y=Z=1; end; 1000000.times { Object.new.extend(M) }'
real 0m1.560s
user 0m1.543s
sys 0m0.016s
</code></pre>
<p>After the patch, <code>Object#extend</code> invalidates all constant names defined in the module of the arguments, which takes O(n). I think it's an acceptable trade-off, though.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=969032022-03-17T14:01:23Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>I am positive introducing this proposal.</p>
<p>Matz.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=969042022-03-17T14:11:03Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>I'm not against the proposal, but for the record, the change makes Object#extend and Module#include slow.</p>
</blockquote>
<p>I think it's acceptable, because the same code previously would have busted the global constant cache, so it would have made the whole application slower, now to pay the cost upfront, which is positive in my book.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=970282022-03-25T11:30:05Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>Applied in changeset <a class="changeset" title="Revert "Finer-grained inline constant cache invalidation" This reverts commits for [Feature #185..." href="https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/69967ee64eac9ce65b83533a566d69d12a6046d0">git|69967ee64eac9ce65b83533a566d69d12a6046d0</a>.</p>
<hr>
<p>Revert "Finer-grained inline constant cache invalidation"</p>
<p>This reverts commits for [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Finer-grained constant invalidation (Closed)" href="https://bugs.ruby-lang.org/issues/18589">#18589</a>]:</p>
<ul>
<li>8008fb7352abc6fba433b99bf20763cf0d4adb38<br>
"Update formatting per feedback"</li>
<li>8f6eaca2e19828e92ecdb28b0fe693d606a03f96<br>
"Delete ID from constant cache table if it becomes empty on ISEQ free"</li>
<li>629908586b4bead1103267652f8b96b1083573a8<br>
"Finer-grained inline constant cache invalidation"</li>
</ul>
<p>MSWin builds on AppVeyor have been crashing since the merger.</p> Ruby master - Feature #18589: Finer-grained constant invalidationhttps://bugs.ruby-lang.org/issues/18589?journal_id=994852022-10-06T08:01:08Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul></ul><p>Note: This proposal merged at <a href="https://github.com/ruby/ruby/pull/5716" class="external">https://github.com/ruby/ruby/pull/5716</a> again.</p>