https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112013-08-17T06:40:31ZRuby Issue Tracking SystemRuby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=412012013-08-17T06:40:31Znormalperson (Eric Wong)normalperson@yhbt.net
<ul><li><strong>File</strong> <a href="/attachments/3896">0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3896/0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch">0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch</a> added</li></ul><p>[PATCH] thread_pthread: use eventfd under Linux for timer thread</p>
<p>The timer thread is an ideal use case for eventfd because it is<br>
only used to signal wakeups and not transfer data.</p>
<p>From eventfd(2) manpage:</p>
<p>Applications can use an eventfd file descriptor instead of a pipe (see<br>
pipe(2)) in all cases where a pipe is used simply to signal events.<br>
The kernel overhead of an eventfd file descriptor is much lower than<br>
that of a pipe, and only one file descriptor is required (versus the<br>
two required for a pipe).</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=412022013-08-17T06:53:24Znormalperson (Eric Wong)normalperson@yhbt.net
<ul><li><strong>File</strong> <a href="/attachments/3897">0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3897/0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch">0001-thread_pthread-use-eventfd-under-Linux-for-timer-thr.patch</a> added</li></ul><p>Ignore my first patch, eventfd_compat function should have void return, not int.<br>
Sorry for the confusion</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=412152013-08-17T20:23:11Zko1 (Koichi Sasada)
<ul></ul><p>(2013/08/16 10:47), normalperson (Eric Wong) wrote:</p>
<blockquote>
<p>eventfd is a cheaper alternative to pipe for self-notification (signals) on Linux</p>
<p>I will submit patches in the next few days/weeks unless there are objections<br>
(or somebody else wants to do it sooner). I'd also like to cleanup some of the existing #ifdefs in that area while I'm at it.</p>
</blockquote>
<p>Can we see the performance comparison?<br>
If we can see the clear difference, it can be acceptable.</p>
<p>(If we can't see any difference, it only increase the source code<br>
complexity).</p>
<p>--<br>
// SASADA Koichi at atdot dot net</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=412322013-08-18T04:53:20Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>SASADA Koichi <a href="mailto:ko1@atdot.net" class="email">ko1@atdot.net</a> wrote:</p>
<blockquote>
<p>(2013/08/16 10:47), normalperson (Eric Wong) wrote:</p>
<blockquote>
<p>eventfd is a cheaper alternative to pipe for self-notification (signals) on Linux</p>
<p>I will submit patches in the next few days/weeks unless there are objections<br>
(or somebody else wants to do it sooner). I'd also like to cleanup some of the existing #ifdefs in that area while I'm at it.</p>
</blockquote>
<p>Can we see the performance comparison?<br>
If we can see the clear difference, it can be acceptable.</p>
</blockquote>
<p>It's not for speed (signal handling performance should not be a<br>
bottleneck), but halve FD use in userspace and reduce memory use inside<br>
the kernel.</p>
<p>AFAIK, writing to a empty pipe still allocates a 4K page, eventfd avoids<br>
that allocation/deallocation. Since Ruby is CoW/fork-friendly, this<br>
should allow running more Ruby processes on a system.</p>
<p>I also thought my own code had an FD leak when timer_thread_pipe_low was<br>
introduced. Maybe this will reduce confusion for users who lsof Ruby<br>
processes, since there are more pipe users than eventfd users.</p>
<blockquote>
<p>(If we can't see any difference, it only increase the source code<br>
complexity).</p>
</blockquote>
<p>I've tried to minimize the impact of my patch and keep the eventfd/pipe<br>
difference minimal.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=413052013-08-21T04:53:18Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul></ul><p>Hi</p>
<p>On Sat, Aug 17, 2013 at 3:37 PM, Eric Wong <a href="mailto:normalperson@yhbt.net" class="email">normalperson@yhbt.net</a> wrote:</p>
<blockquote>
<p>SASADA Koichi <a href="mailto:ko1@atdot.net" class="email">ko1@atdot.net</a> wrote:</p>
<blockquote>
<p>(2013/08/16 10:47), normalperson (Eric Wong) wrote:</p>
<blockquote>
<p>eventfd is a cheaper alternative to pipe for self-notification (signals) on Linux</p>
<p>I will submit patches in the next few days/weeks unless there are objections<br>
(or somebody else wants to do it sooner). I'd also like to cleanup some of the existing #ifdefs in that area while I'm at it.</p>
</blockquote>
<p>Can we see the performance comparison?<br>
If we can see the clear difference, it can be acceptable.</p>
</blockquote>
<p>It's not for speed (signal handling performance should not be a<br>
bottleneck), but halve FD use in userspace and reduce memory use inside<br>
the kernel.</p>
</blockquote>
<p>How much increase number of maximum ruby processes? Can you measure it?<br>
I bet the difference is very small.</p>
<blockquote>
<p>AFAIK, writing to a empty pipe still allocates a 4K page, eventfd avoids<br>
that allocation/deallocation. Since Ruby is CoW/fork-friendly, this<br>
should allow running more Ruby processes on a system.</p>
<p>I also thought my own code had an FD leak when timer_thread_pipe_low was<br>
introduced. Maybe this will reduce confusion for users who lsof Ruby<br>
processes, since there are more pipe users than eventfd users.</p>
</blockquote>
<p>Well, that's not a good reason. You said your patch decrease your confusion<br>
but increase a confusion of other eventfd users.</p>
<blockquote>
<blockquote>
<p>(If we can't see any difference, it only increase the source code<br>
complexity).</p>
</blockquote>
<p>I've tried to minimize the impact of my patch and keep the eventfd/pipe<br>
difference minimal.</p>
</blockquote>
<p>Anyway, I haven't seen any bugs in your patch. I would see a measurement<br>
result.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=413062013-08-21T07:29:16Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>KOSAKI Motohiro <a href="mailto:kosaki.motohiro@gmail.com" class="email">kosaki.motohiro@gmail.com</a> wrote:</p>
<blockquote>
<p>Hi</p>
<p>On Sat, Aug 17, 2013 at 3:37 PM, Eric Wong <a href="mailto:normalperson@yhbt.net" class="email">normalperson@yhbt.net</a> wrote:</p>
<blockquote>
<p>SASADA Koichi <a href="mailto:ko1@atdot.net" class="email">ko1@atdot.net</a> wrote:</p>
<blockquote>
<p>(2013/08/16 10:47), normalperson (Eric Wong) wrote:</p>
<blockquote>
<p>eventfd is a cheaper alternative to pipe for self-notification (signals) on Linux</p>
<p>I will submit patches in the next few days/weeks unless there are objections<br>
(or somebody else wants to do it sooner). I'd also like to cleanup some of the existing #ifdefs in that area while I'm at it.</p>
</blockquote>
<p>Can we see the performance comparison?<br>
If we can see the clear difference, it can be acceptable.</p>
</blockquote>
<p>It's not for speed (signal handling performance should not be a<br>
bottleneck), but halve FD use in userspace and reduce memory use inside<br>
the kernel.</p>
</blockquote>
<p>How much increase number of maximum ruby processes? Can you measure it?<br>
I bet the difference is very small.</p>
</blockquote>
<p>On Linux 3.10 on x86_64, 64-byte L1 cache line size</p>
<p>file->private_data:<br>
sizeof(struct eventfd_ctx) == 48 bytes<br>
sizeof(struct pipe_inode_info) == 136 bytes</p>
<p>So 176 bytes and 2 FDs saved for every Ruby process. Fwiw, I often have<br>
hundreds of (mostly idle) Ruby processes on my systems running random<br>
scripts/daemons.</p>
<p>I doubt most users will notice the difference. But maybe it will make a<br>
tiny difference somewhere (fewer cache lines touched, smaller select()<br>
footprint).</p>
<p>I don't have a machine to forkbomb with Ruby, but overall size of Ruby<br>
is probably the limiting factor anyways.</p>
<blockquote>
<blockquote>
<p>AFAIK, writing to a empty pipe still allocates a 4K page, eventfd avoids<br>
that allocation/deallocation. Since Ruby is CoW/fork-friendly, this<br>
should allow running more Ruby processes on a system.</p>
<p>I also thought my own code had an FD leak when timer_thread_pipe_low was<br>
introduced. Maybe this will reduce confusion for users who lsof Ruby<br>
processes, since there are more pipe users than eventfd users.</p>
</blockquote>
<p>Well, that's not a good reason. You said your patch decrease your confusion<br>
but increase a confusion of other eventfd users.</p>
</blockquote>
<p>I suppose it depends on the user. I'm don't know of anybody using<br>
eventfd with Ruby right now (but I'll be updating some of my projects to<br>
do so).</p>
<blockquote>
<blockquote>
<blockquote>
<p>(If we can't see any difference, it only increase the source code<br>
complexity).</p>
</blockquote>
<p>I've tried to minimize the impact of my patch and keep the eventfd/pipe<br>
difference minimal.</p>
</blockquote>
<p>Anyway, I haven't seen any bugs in your patch. I would see a measurement<br>
result.</p>
</blockquote>
<p>Thanks for looking. Sorry I cannot provide real-world measurement/use<br>
case.</p>
<p>Ideally, we wouldn't even need a timer thread and we could just use<br>
ppoll/pselect. But that would be a very intrusive change (and maybe too<br>
incompatible with C extensions).</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=413102013-08-21T18:53:15Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul></ul><blockquote>
<p>Ideally, we wouldn't even need a timer thread and we could just use<br>
ppoll/pselect. But that would be a very intrusive change (and maybe too<br>
incompatible with C extensions).</p>
</blockquote>
<p>Ideally?<br>
syscall is much slower than current flag check in VM loop. That's why<br>
now VM event loop doesn't handle thread runtime expire directly.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=413152013-08-22T02:23:11Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>KOSAKI Motohiro <a href="mailto:kosaki.motohiro@gmail.com" class="email">kosaki.motohiro@gmail.com</a> wrote:</p>
<blockquote>
<blockquote>
<p>Ideally, we wouldn't even need a timer thread and we could just use<br>
ppoll/pselect. But that would be a very intrusive change (and maybe too<br>
incompatible with C extensions).</p>
</blockquote>
<p>Ideally?<br>
syscall is much slower than current flag check in VM loop. That's why<br>
now VM event loop doesn't handle thread runtime expire directly.</p>
</blockquote>
<p>Ah, I forget about thread runtime expiry. Maybe that's why I gave up<br>
on the idea originally, I had this idea a long while (years?) back.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=421452013-10-01T17:10:15Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Feedback</i></li><li><strong>Target version</strong> changed from <i>2.1.0</i> to <i>2.6</i></li></ul><p>I'm negative because it causes code complex unless it has performance improvement.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=476482014-07-08T22:38:35Znormalperson (Eric Wong)normalperson@yhbt.net
<ul><li><strong>File</strong> <a href="/attachments/4530">tt_efd_v2.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/4530/tt_efd_v2.patch">tt_efd_v2.patch</a> added</li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/47648/diff?detail_id=34436">diff</a>)</li></ul><p>Updated patch (from testing for <a class="issue tracker-1 status-1 priority-4 priority-default" title="Bug: IO operation is 10x slower in multi-thread environment (Open)" href="https://bugs.ruby-lang.org/issues/10009">#10009</a>).</p>
<p>Uploading for archival purposes. This version is probably less intrusive and<br>
falls back to pipe in case of ENOSYS (in case glibc supports eventfd and the<br>
kernel has eventfd disabled).</p>
<p>This has no measurable performance improvement for me, but saves two FDs<br>
and a few bytes of kernel memory for every process.</p> Ruby master - Feature #8788: use eventfd on newer Linux instead of pipe for timer threadhttps://bugs.ruby-lang.org/issues/8788?journal_id=689132017-12-25T18:15:12Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Target version</strong> deleted (<del><i>2.6</i></del>)</li></ul>