https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-01-17T01:36:20ZRuby Issue Tracking SystemRuby master - Feature #18494: [RFC] ENV["RUBY_GC_..."]= changes GC parameters dynamicallyhttps://bugs.ruby-lang.org/issues/18494?journal_id=960022022-01-17T01:36:20Zko1 (Koichi Sasada)
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/96002/diff?detail_id=61851">diff</a>)</li></ul> Ruby master - Feature #18494: [RFC] ENV["RUBY_GC_..."]= changes GC parameters dynamicallyhttps://bugs.ruby-lang.org/issues/18494?journal_id=960032022-01-17T01:38:57Zko1 (Koichi Sasada)
<ul></ul><p>Some <code>RUBY_GC_...</code> vars do not affect correctly because they are used only on setup.<br>
We need to document which vars can be modified dynamically.</p> Ruby master - Feature #18494: [RFC] ENV["RUBY_GC_..."]= changes GC parameters dynamicallyhttps://bugs.ruby-lang.org/issues/18494?journal_id=960122022-01-17T09:11:39Zbyroot (Jean Boussier)byroot@ruby-lang.org
<ul></ul><blockquote>
<p>Some RUBY_GC_... vars do not affect correctly because they are used only on setup.</p>
</blockquote>
<p>Should we allow to change them at runtime through some <code>::GC</code> API? I could see some automatic/dynamic GC tuning gems being implemented using these APIs. e.g. monitor how often and how long the GC runs, and tweak some values in response.</p> Ruby master - Feature #18494: [RFC] ENV["RUBY_GC_..."]= changes GC parameters dynamicallyhttps://bugs.ruby-lang.org/issues/18494?journal_id=960302022-01-17T23:15:17Znormalperson (Eric Wong)normalperson@yhbt.net
<ul><li><strong>File</strong> <a href="/attachments/9168">0003-ruby_gc_set_params-do-not-set-initial-pages-on-dynam.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/9168/0003-ruby_gc_set_params-do-not-set-initial-pages-on-dynam.patch">0003-ruby_gc_set_params-do-not-set-initial-pages-on-dynam.patch</a> added</li><li><strong>File</strong> <a href="/attachments/9166">0001-ruby_gc_set_params-update-malloc_limit-when-env-is-s.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/9166/0001-ruby_gc_set_params-update-malloc_limit-when-env-is-s.patch">0001-ruby_gc_set_params-update-malloc_limit-when-env-is-s.patch</a> added</li><li><strong>File</strong> <a href="/attachments/9167">0002-ENV-RUBY_GC_.-changes-GC-parameters-dynamically.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/9167/0002-ENV-RUBY_GC_.-changes-GC-parameters-dynamically.patch">0002-ENV-RUBY_GC_.-changes-GC-parameters-dynamically.patch</a> added</li></ul><blockquote>
<p><a href="https://bugs.ruby-lang.org/issues/18494" class="external">https://bugs.ruby-lang.org/issues/18494</a></p>
</blockquote>
<p>Thanks both for the comments.</p>
<p>"ko1 (Koichi Sasada)" wrote:</p>
<blockquote>
<p>Some <code>RUBY_GC_...</code> vars do not affect correctly because they are used only on setup.<br>
We need to document which vars can be modified dynamically.</p>
</blockquote>
<p>Updated patch series attached, patch 3/3 adds partial guard to<br>
RUBY_GC_INIT_HEAP_SLOTS to prevent gc_set_initial_pages().</p>
<p>Patch 1/3 also fixes an existing bug in how RUBY_GC_MALLOC_LIMIT<br>
that is independent of this new feature.</p>
<p>All the other parameters seem fine, however I'm not sure about<br>
interactions w.r.t. Ractors and double-precision floats.</p>
<p>"byroot (Jean Boussier)" wrote:</p>
<blockquote>
<p>Should we allow to change them at runtime through some <code>::GC</code><br>
API? I could see some automatic/dynamic GC tuning gems being<br>
implemented using these APIs. e.g. monitor how often and how<br>
long the GC runs, and tweak some values in response.</p>
</blockquote>
<p>I'm rather strongly against this. I envision use would be:</p>
<ol>
<li>load a bunch of code, read configs, etc...</li>
<li>set GC params</li>
<li>run main loop until exit</li>
</ol>
<p>The main loop in reasonably-written applications should be<br>
highly predictable in terms of memory use and have minimal<br>
outliers and spikes. IOW, a web server should be serving<br>
reasonably-sized HTML pages/chunks that clients can render w/o<br>
falling over; and reading/sending large files in chunks rather<br>
than slurping hundreds of MB into RAM.</p>
<p>For a rare memory spikes inside the main loop, I'd much rather<br>
trigger GC ASAP than be saddled with a larger heap for the<br>
remainder of process lifetime (because larger heaps increase<br>
potential for fragmentation).</p>
<p>I see the presence of tuning knobs to be an admission of the<br>
shortcomings of the GC and something that could/should<br>
eventually be eliminated via GC improvements. (Fwiw, I've<br>
mostly given up on Ruby and it's GC; but I made this patch since<br>
it's too time-consuming to rewrite some Ruby in a scripting<br>
language with auto ref-counting and faster startup time).</p>
<p>Anyways, I've updated the commit message in my original patch<br>
(now patch 2/3) to further explain my decision. Here's the<br>
relevant snippet:</p>
<p>This has no extra API footprint, and will silently be a no-op for other<br>
Ruby implementations. I tried to make this change as non-intrusive as<br>
possible to minimize the growth in executable and icache size. It is<br>
not optimized for repeated changes and (IMHO) should not be. IMHO,<br>
tuning knobs should be last resorts and used sparingly to minimize the<br>
learning curve and cognitive overhead required to run applications.</p>
<p>Thus there is no "GC.foo=" API to reduce documentation and support<br>
overhead, since GC parameters are implementation-dependent and may<br>
change over time. Developers can make ENV changes freely without<br>
worrying about forward, backwards, nor cross-engine incompatibilities.<br>
These parameters are already documented in the manpage and other<br>
sources, so there's less cognitive overhead required to learn them.</p>
<p>Increasing the Ruby API footprint would also hurt startup time and<br>
increases memory usage.</p>
<p>Available as a self-hosted pull request (generated via "git request-pull"):</p>
<p>The following changes since commit eb98275c967d8938526966fe53e52f5a10249492:</p>
<ul>
<li>2022-01-18 [ci skip] (2022-01-18 05:39:51 +0900)</li>
</ul>
<p>are available in the Git repository at:</p>
<p><a href="https://yhbt.net/ruby.git" class="external">https://yhbt.net/ruby.git</a> gc-param-dyn-v2</p>
<p>for you to fetch changes up to ffb336a8ddb0e21d8f3bb4ce1801e90ea78af42d:</p>
<p>ruby_gc_set_params: do not set initial pages on dynamic update (2022-01-17 22:59:58 +0000)</p>
<hr>
<p>Eric Wong (3):<br>
ruby_gc_set_params: update malloc_limit when env is set<br>
ENV["RUBY_GC_..."]= changes GC parameters dynamically<br>
ruby_gc_set_params: do not set initial pages on dynamic update</p>
<p>gc.c | 11 +++++++----<br>
hash.c | 5 +++++<br>
internal/gc.h | 2 +-<br>
ruby.c | 2 +-<br>
test/ruby/test_gc.rb | 4 ++++<br>
5 files changed, 18 insertions(+), 6 deletions(-)</p>
<p>Thanks for reading this far.</p>