Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112024-01-05T22:14:24ZRuby Issue Tracking System
Redmine Ruby master - Bug #20155 (Assigned): Using value of rb_fiber_scheduler_current() crashes Rubyhttps://bugs.ruby-lang.org/issues/201552024-01-05T22:14:24Zpaddor (Patrik Wenger)paddor@gmail.com
<p>While trying to manually block/unblock fibers from an extension using the Fiber Scheduler,<br>
I noticed that using the return value of <code>rb_fiber_scheduler_current()</code> crashes Ruby.</p>
<p>I've created a minimal extension gem called "fiber_blocker". Its test suite shows the behavior. See <a href="https://github.com/paddor/fiber_blocker" class="external">https://github.com/paddor/fiber_blocker</a>, especially the lines containing <code>FIXME</code>.</p>
<p>Passing <code>Fiber.scheduler</code> to the extension functions works. But letting it get the current scheduler itself does not seem to work.</p>
<p>Is <code>rb_fiber_scheduler_current()</code>(within a non-blocking Fiber) not the equivalent to <code>Fiber.scheduler</code>?<br>
Even just printing the its return value with <code>#p</code> will crash Ruby.</p>
<p>Ruby either crashes like this:</p>
<pre><code># Running:
T1 BEGIN
T2 BEGIN
T1 END
..T1 BEGIN
ext: blocking fiber
passed scheduler = #<Scheduler:0x00007fc5f22d39e8 @readable={}, @writable={}, @waiting={}, @closed=false, @lock=#<Thread::Mutex:0x00007fc5f22ec8d0>, @blocking={}, @ready=[], @urgent=[#<IO:fd 5>, #<IO:fd 6>]>
T2 BEGIN
ext: unblocking fiber
T1 END
.E
Finished in 1.007014s, 3.9721 runs/s, 2.9791 assertions/s.
1) Error:
TestFiberBlocker#test_fiber_blocker_current_fiber:
fatal: machine stack overflow in critical region
No backtrace
</code></pre>
<p>Or with a segfault:</p>
<pre><code># Running:
FiberBlocker.test works.
.T1 BEGIN
T2 BEGIN
T1 END
.T1 BEGIN
ext: blocking fiber
/home/user/dev/oss/async_ruby_test/rbnng/fiber_blocker/test/test_fiber_blocker.rb:40: [BUG] Segmentation fault at 0x00000000390d8f98
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux]
-- Control frame information -----------------------------------------------
c:0003 p:---- s:0012 e:000011 CFUNC :block_fiber
c:0002 p:0014 s:0006 e:000005 BLOCK /home/user/dev/oss/async_ruby_test/rbnng/fiber_blocker/test/test_fiber_blocker.rb:40 [FINISH]
c:0001 p:---- s:0003 e:000002 DUMMY [FINISH]
-- Ruby level backtrace information ----------------------------------------
/home/user/dev/oss/async_ruby_test/rbnng/fiber_blocker/test/test_fiber_blocker.rb:40:in `block in test_fiber_blocking_in_ext'
/home/user/dev/oss/async_ruby_test/rbnng/fiber_blocker/test/test_fiber_blocker.rb:40:in `block_fiber'
-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 4
-- Machine register context ------------------------------------------------
RIP: 0x00007f1554f17ad8 RBP: 0x00000000390d8f90 RSP: 0x00007f153a79e280
RAX: 0x00007f1554addba8 RBX: 0x00007f153a79eab0 RCX: 0x0000000000000000
RDX: 0x00007f1554ade600 RDI: 0x00007f15551e8788 RSI: 0x0000000000000ae1
R8: 0x000000000000002b R9: 0x00007f153a79f038 R10: 0x00007f1554c0b9b0
R11: 0x00007f153a79e490 R12: 0x0000000000000ae1 R13: 0x0000000000000000
R14: 0x0000000000000000 R15: 0x000055ab732d7df0 EFL: 0x0000000000010206
-- C level backtrace information -------------------------------------------
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_print_backtrace+0x14) [0x7f1554f24961] /home/user/src/ruby-3.3.0/vm_dump.c:820
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_vm_bugreport) /home/user/src/ruby-3.3.0/vm_dump.c:1151
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_bug_for_fatal_signal+0x104) [0x7f1554d1c214] /home/user/src/ruby-3.3.0/error.c:1065
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(sigsegv+0x4f) [0x7f1554e700df] /home/user/src/ruby-3.3.0/signal.c:926
/lib/x86_64-linux-gnu/libc.so.6(0x7f1554842520) [0x7f1554842520]
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(RBASIC_CLASS+0x0) [0x7f1554f17ad8] ./include/ruby/internal/globals.h:178
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(gccct_method_search) /home/user/src/ruby-3.3.0/vm_eval.c:475
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_funcallv_scope) /home/user/src/ruby-3.3.0/vm_eval.c:1063
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_funcallv) /home/user/src/ruby-3.3.0/vm_eval.c:1084
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_inspect+0x19) [0x7f1554dc1569] /home/user/src/ruby-3.3.0/object.c:697
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(ruby__sfvextra+0x11a) [0x7f1554e7223a] /home/user/src/ruby-3.3.0/sprintf.c:1119
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(BSD_vfprintf+0xa69) [0x7f1554e73059] /home/user/src/ruby-3.3.0/vsnprintf.c:830
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(RBASIC_SET_CLASS_RAW+0x0) [0x7f1554e75b56] /home/user/src/ruby-3.3.0/sprintf.c:1168
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(ruby_vsprintf0) /home/user/src/ruby-3.3.0/sprintf.c:1169
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_enc_vsprintf+0x5d) [0x7f1554e75ecd] /home/user/src/ruby-3.3.0/sprintf.c:1195
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_sprintf+0x9d) [0x7f1554e7607d] /home/user/src/ruby-3.3.0/sprintf.c:1225
/home/user/dev/oss/async_ruby_test/rbnng/fiber_blocker/lib/fiber_blocker/fiber_blocker.so(block_fiber+0x4a) [0x7f1554ad430a] ../../../../ext/fiber_blocker/fiber_blocker.c:29
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_cfp_consistent_p+0x0) [0x7f1554ef64b4] /home/user/src/ruby-3.3.0/vm_insnhelper.c:3490
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_call_cfunc_with_frame_) /home/user/src/ruby-3.3.0/vm_insnhelper.c:3492
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_call_cfunc_with_frame) /home/user/src/ruby-3.3.0/vm_insnhelper.c:3518
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_call_cfunc_other) /home/user/src/ruby-3.3.0/vm_insnhelper.c:3544
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_sendish+0x9e) [0x7f1554f06f87] /home/user/src/ruby-3.3.0/vm_insnhelper.c:5581
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(vm_exec_core) /home/user/src/ruby-3.3.0/insns.def:834
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_vm_exec+0x19a) [0x7f1554f0d1fa] /home/user/src/ruby-3.3.0/vm.c:2486
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_vm_invoke_proc+0x5f) [0x7f1554f12e0f] /home/user/src/ruby-3.3.0/vm.c:1728
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_fiber_start+0x1ba) [0x7f1554cf098a] /home/user/src/ruby-3.3.0/cont.c:2536
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(fiber_entry+0x20) [0x7f1554cf0d00] /home/user/src/ruby-3.3.0/cont.c:847
/home/user/.rubies/ruby-3.3.0/lib/libruby.so.3.3(rb_threadptr_root_fiber_setup) (null):0
</code></pre>
<p>This happens with the Async scheduler as well as with Ruby’s test scheduler. My minimal extension uses Ruby’s.</p>
<p>I hope I'm not missing something obvious. My C isn't very good.</p> Ruby master - Feature #20105 (Open): Introduce `IO::Stream` or something similar.https://bugs.ruby-lang.org/issues/201052024-01-01T07:43:50Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>Ruby's IO class has a general model for streaming IO, including some hidden classes like <code>IO::generic_readable</code> and <code>IO::generic_writable</code>.</p>
<p>As Ruby's core IO classes evolve, gems like <code>openssl</code> (see <code>OpenSSL::Buffering</code>) need to be updated to support changes to the interface.</p>
<p>As it stands, there are changes in <code>IO</code> which are not copied to <code>OpenSSL::Buffering</code>. I'd like to propose we introduce some shared interface for streams that is used by <code>IO</code>, <code>Socket</code>, and <code>OpenSSL</code> to start with. The general interface would be similar to <code>IO</code> and allow code like <code>OpenSSL</code> to avoid re-implementing the <code>IO</code> interface.</p>
<p>I don't have a strong idea for the interface yet, but it would probably look something like this:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">IO::Stream</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">io</span><span class="p">,</span> <span class="ss">buffered: </span><span class="kp">true</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">read</span><span class="p">(</span><span class="n">size</span><span class="p">,</span> <span class="n">buffer</span><span class="o">=</span><span class="kp">nil</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">write</span><span class="p">(</span><span class="n">size</span><span class="p">,</span> <span class="n">buffer</span><span class="o">=</span><span class="kp">nil</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># Include general operations from IO, like gets, puts, etc</span>
<span class="k">end</span>
</code></pre>
<p>I think ideally we'd try implement with pure Ruby and a first goal would be to replace <code>OpenSSL::Buffering</code>.</p> Ruby master - Feature #20102 (Open): Introduce `Fiber#resuming?`https://bugs.ruby-lang.org/issues/201022023-12-28T07:25:59Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>There are some tricky edge cases when using <code>Fibre#raise</code> and <code>Fiber#kill</code>, e.g.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">fiber</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="n">killer</span> <span class="o">=</span> <span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">fiber</span><span class="p">.</span><span class="nf">raise</span><span class="p">(</span><span class="s2">"Stop"</span><span class="p">)</span>
<span class="k">end</span>
<span class="n">fiber</span> <span class="o">=</span> <span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">killer</span><span class="p">.</span><span class="nf">resume</span>
<span class="k">end</span>
<span class="n">fiber</span><span class="p">.</span><span class="nf">resume</span>
<span class="c1"># 4:in `raise': attempt to raise a resuming fiber (FiberError)</span>
<span class="c1"># 4:in `block in <main>'</span>
</code></pre>
<p>Async has to deal with this edge case explicitly by rescuing the exception:</p>
<p><a href="https://github.com/socketry/async/blob/ffd019d9c1d547926a28fe8f36bf7bfe91d8a168/lib/async/task.rb#L226-L233" class="external">https://github.com/socketry/async/blob/ffd019d9c1d547926a28fe8f36bf7bfe91d8a168/lib/async/task.rb#L226-L233</a></p>
<p>I'd like to avoid doing that and instead just ask "Can I kill/raise on this fiber right now?" which is determined by whether the fiber itself can be resumed or transferred to.</p>
<p>To address this, I'd like to introduce <code>Fiber#resuming?</code>:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="cm">/*
* call-seq: fiber.resumed? -> true or false
*
* Whether the fiber is currently resumed.
*/</span>
<span class="n">VALUE</span>
<span class="nf">rb_fiber_resuming_p</span><span class="p">(</span><span class="n">VALUE</span> <span class="n">fiber_value</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">struct</span> <span class="n">rb_fiber_struct</span> <span class="o">*</span><span class="n">fiber</span> <span class="o">=</span> <span class="n">fiber_ptr</span><span class="p">(</span><span class="n">fiber_value</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">FIBER_TERMINATED_P</span><span class="p">(</span><span class="n">fiber</span><span class="p">))</span> <span class="k">return</span> <span class="n">RUBY_Qfalse</span><span class="p">;</span>
<span class="k">return</span> <span class="n">RBOOL</span><span class="p">(</span><span class="n">fiber</span><span class="o">-></span><span class="n">resuming_fiber</span><span class="p">);</span>
<span class="p">}</span>
</code></pre>
<p>See the PR: <a href="https://github.com/ruby/ruby/pull/9382" class="external">https://github.com/ruby/ruby/pull/9382</a></p> Ruby master - Bug #20082 (Open): Killing fibers across threads: unexpected exceptionhttps://bugs.ruby-lang.org/issues/200822023-12-23T21:15:27Zzverok (Victor Shepelev)zverok.offline@gmail.com
<p>For providing the example in a changelog, I tried to imitate killing fibers belonging to other threads.<br>
Documentation <a href="https://docs.ruby-lang.org/en/master/Fiber.html#method-i-kill" class="external">claims</a></p>
<blockquote>
<p>Raises FiberError if called on a fiber belonging to another thread.</p>
</blockquote>
<p>So, I created this artificial example to check how it works:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">fibers</span> <span class="o">=</span> <span class="p">[]</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="n">f</span> <span class="o">=</span> <span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">each</span> <span class="p">{</span> <span class="nb">sleep</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">fibers</span> <span class="o"><<</span> <span class="n">f</span>
<span class="n">f</span><span class="p">.</span><span class="nf">resume</span>
<span class="p">}</span>
<span class="nb">sleep</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span> <span class="c1"># to make sure the thread has started</span>
<span class="n">fibers</span><span class="p">.</span><span class="nf">last</span><span class="p">.</span><span class="nf">kill</span>
</code></pre>
<p>The example indeed fails with <code>FiberError</code>, but the error message is confusing:</p>
<pre><code>in `kill': attempt to resume a resumed fiber (double resume) (FiberError)
</code></pre>
<p>Is this an expected message for such case? Or am I misunderstanding something?</p> Ruby master - Feature #19642 (Open): Remove vectored read/write from `io.c`.https://bugs.ruby-lang.org/issues/196422023-05-15T06:33:11Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p><a href="https://github.com/socketry/async/issues/228#issuecomment-1546789910" class="external">https://github.com/socketry/async/issues/228#issuecomment-1546789910</a> is a comment from the kernel developer who tells us that <code>writev</code> is always worse than <code>write</code> system call.</p>
<p>A large amount of complexity in <code>io.c</code> comes from optional support from <code>writev</code>.</p>
<p>So, I'd like to remove support for <code>writev</code>.</p>
<p>I may try to measure the performance before/after. However it may not show much difference, except that the implementation in <code>io.c</code> can be much simpler.</p> Ruby master - Misc #19122 (Open): Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's s...https://bugs.ruby-lang.org/issues/191222022-11-11T16:06:08Zsmcgivern (Sean McGivern)
<p>I'd like to propose that Ruby stops using MADV_FREE when freeing a Fiber's stack, and switches to using MADV_DONTNEED even when MADV_FREE is supported.</p>
<p>MADV_FREE is used in one place in the Ruby codebase, when freeing the stack of a freed Fiber: <a href="https://git.ruby-lang.org/ruby.git/tree/cont.c#n683" class="external">https://git.ruby-lang.org/ruby.git/tree/cont.c#n683</a></p>
<p>The comment for <code>fiber_pool_stack_free</code> says:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">// We advise the operating system that the stack memory pages are no longer being used.</span>
<span class="c1">// This introduce some performance overhead but allows system to relaim memory when there is pressure.</span>
</code></pre>
<p>Where possible (i.e. on Linux 4.5 and later), <code>fiber_pool_stack_free</code> uses <code>MADV_FREE</code> over <code>MADV_DONTNEED</code>. This has the side effect that memory statistics such as RSS will not reduce until and unless the OS actually reclaims that memory. If that doesn't happen, then the reported memory usage via RSS will be much higher than the 'real' memory usage.</p>
<p>If this was pervasive throughtout the Ruby codebase then that would be one thing, but currently this is just for Fiber. This means that:</p>
<ol>
<li>A program that doesn't use Fiber will have somewhat reliable RSS statistics on recent Linux.</li>
<li>A program that heavily uses Fiber (such as something using Async::HTTP) will see an inflated RSS statistic.</li>
</ol>
<p>Go made a similar change to the one I'm proposing here for similar reasons: <a href="https://github.com/golang/go/issues/42330" class="external">https://github.com/golang/go/issues/42330</a></p>
<blockquote>
<p>While <code>MADV_FREE</code> is somewhat faster than <code>MADV_DONTNEED</code>, it doesn't affect many of the statistics that <code>MADV_DONTNEED</code> does until the memory is actually reclaimed. This generally leads to poor user experience, like confusing stats in <code>top</code> and other monitoring tools; and bad integration with management systems that respond to memory usage.<br>
[...]<br>
I propose we change the default to prefer <code>MADV_DONTNEED</code> over <code>MADV_FREE</code>, to favor user-friendliness and minimal surprise over performance. I think it's become clear that Linux's implementation of <code>MADV_FREE</code> ultimately doesn't meet our needs.</p>
</blockquote>
<p>As an aside, MADV_FREE was not used in Ruby 3.1 (<a href="https://bugs.ruby-lang.org/issues/19101" class="external">https://bugs.ruby-lang.org/issues/19101</a>), and I haven't found any bugs filed about this behaviour other than that one.</p> Ruby master - Feature #19077 (Open): Introduce `Hash#dup` copy on write.https://bugs.ruby-lang.org/issues/190772022-10-22T21:55:25Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>Pull Request: <a href="https://github.com/ruby/ruby/pull/6613" class="external">https://github.com/ruby/ruby/pull/6613</a>.</p> Ruby master - Feature #19057 (Assigned): Hide implementation of `rb_io_t`.https://bugs.ruby-lang.org/issues/190572022-10-15T01:54:57Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>In order to make improvements to the IO implementation like <a href="https://bugs.ruby-lang.org/issues/18455" class="external">https://bugs.ruby-lang.org/issues/18455</a>, we need to add new fields to <code>struct rb_io_t</code>.</p>
<p>By the way, ending types in <code>_t</code> is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop <code>_t</code> where possible during this conversion.</p>
<p>Anyway, we should try to hide the implementation of <code>struct rb_io</code>. Ideally, we don't expose any of it, but the problem is backwards compatibility.</p>
<p>So, in order to remain backwards compatibility, we should expose some fields of <code>struct rb_io</code>, the most commonly used one is <code>fd</code> and <code>mode</code>, but several others are commonly used.</p>
<p>There are many fields which should not be exposed because they are implementation details.</p>
<a name="Current-proposal"></a>
<h2 >Current proposal<a href="#Current-proposal" class="wiki-anchor">¶</a></h2>
<p>The current proposed change <a href="https://github.com/ruby/ruby/pull/6511" class="external">https://github.com/ruby/ruby/pull/6511</a> creates two structs:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">// include/ruby/io.h</span>
<span class="cp">#ifndef RB_IO_T
</span><span class="k">struct</span> <span class="n">rb_io</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">fd</span><span class="p">;</span>
<span class="c1">// ... public fields ...</span>
<span class="p">};</span>
<span class="cp">#else
</span><span class="k">struct</span> <span class="n">rb_io</span><span class="p">;</span>
<span class="cp">#endif
</span>
<span class="c1">// internal/io.h</span>
<span class="cp">#define RB_IO_T
</span><span class="k">struct</span> <span class="n">rb_io</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">fd</span><span class="p">;</span>
<span class="c1">// ... public fields ...</span>
<span class="c1">// ... private fields ...</span>
<span class="p">};</span>
</code></pre>
<p>However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> + <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/73">@tenderlovemaking (Aaron Patterson)</a> have both given some kind of approval.</p>
<p>That being said, maybe it's not safe.</p>
<p>There are two alternatives:</p>
<a name="Hide-all-details"></a>
<h2 >Hide all details<a href="#Hide-all-details" class="wiki-anchor">¶</a></h2>
<p>We can make public <code>struct rb_io</code> completely invisible.</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">// include/ruby/io.h</span>
<span class="cp">#define RB_IO_HIDDEN
</span><span class="k">struct</span> <span class="n">rb_io</span><span class="p">;</span>
<span class="kt">int</span> <span class="nf">rb_ioptr_descriptor</span><span class="p">(</span><span class="k">struct</span> <span class="n">rb_io</span> <span class="o">*</span><span class="n">ioptr</span><span class="p">);</span> <span class="c1">// accessor for previously visible state.</span>
<span class="c1">// internal/io.h</span>
<span class="k">struct</span> <span class="n">rb_io</span> <span class="p">{</span>
<span class="c1">// ... all fields ...</span>
<span class="p">};</span>
</code></pre>
<p>This would only be forwards compatible, and code would need to feature detect like this:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="cp">#ifdef RB_IO_HIDDEN
#define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor
#else
#define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr)
#endif
</span></code></pre>
<a name="Nested-public-interface"></a>
<h2 >Nested public interface<a href="#Nested-public-interface" class="wiki-anchor">¶</a></h2>
<p>Alternatively, we can nest the public fields into the private struct:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">// include/ruby/io.h</span>
<span class="k">struct</span> <span class="n">rb_io_public</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">fd</span><span class="p">;</span>
<span class="c1">// ... public fields ...</span>
<span class="p">};</span>
<span class="c1">// internal/io.h</span>
<span class="cp">#define RB_IO_T
</span><span class="k">struct</span> <span class="n">rb_io</span> <span class="p">{</span>
<span class="k">struct</span> <span class="n">rb_io_public</span> <span class="n">public</span><span class="p">;</span>
<span class="c1">// ... private fields ...</span>
<span class="p">};</span>
</code></pre>
<a name="Considerations"></a>
<h2 >Considerations<a href="#Considerations" class="wiki-anchor">¶</a></h2>
<p>I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us.</p>
<p>I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing.</p> Ruby master - Feature #19056 (Open): Introduce `Fiber.annotation` for attaching messages to fibers.https://bugs.ruby-lang.org/issues/190562022-10-14T23:09:31Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>It's useful to know what a fiber is doing especially when they have a temporal execution (i.e. sockets connecting vs connected, binding vs accepting, queue popping, etc)</p>
<p>Let's introduce <code>Fiber.annotate</code> and <code>Fiber#annotation</code> for logging a short message attached to Fibers.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Fiber</span><span class="p">.</span><span class="nf">annotate</span> <span class="s2">"Counting to 10"</span>
<span class="mi">10</span><span class="p">.</span><span class="nf">times</span><span class="p">{</span><span class="o">|</span><span class="no">I</span><span class="o">|</span> <span class="nb">puts</span> <span class="no">I</span><span class="p">}</span>
<span class="c1"># Fiber.current.annotation => "Counting to 10"</span>
</code></pre>
<p>Pull Request: <a href="https://github.com/ruby/ruby/pull/6554" class="external">https://github.com/ruby/ruby/pull/6554</a></p> Ruby master - Bug #18818 (Open): Thread waitq does not retain referenced objects, can lead to use...https://bugs.ruby-lang.org/issues/188182022-06-05T21:39:16Znevans (Nicholas Evans)
<p>The attached script (and/or others like it) can cause SEGV in 3.0, 3.1, and master. It has always behaved as expected when I use <code>optflags=-O0</code>.</p>
<p>When I use it with <code>make run</code> on <code>master</code>:</p>
<pre><code>./miniruby -I../lib -I. -I.ext/common -r./x86_64-linux-fake ../test.rb
========================================================================
fiber_queue
completed in 0.00031349004711955786
========================================================================
fiber_sized_queue
../test.rb:62: [BUG] Segmentation fault at 0x0000000000000000
ruby 3.2.0dev (2022-06-05T06:18:26Z master 5ce0be022f) [x86_64-linux]
-- Control frame information -----------------------------------------------
c:0005 p:---- s:0023 e:000022 CFUNC :%
c:0004 p:0031 s:0018 e:000015 METHOD ../test.rb:62 [FINISH]
c:0003 p:---- s:0010 e:000009 CFUNC :pop
c:0002 p:0009 s:0006 e:000005 BLOCK ../test.rb:154 [FINISH]
c:0001 p:---- s:0003 e:000002 (none) [FINISH]
-- Ruby level backtrace information ----------------------------------------
../test.rb:154:in `block (2 levels) in <main>'
../test.rb:154:in `pop'
../test.rb:62:in `unblock'
../test.rb:62:in `%'
-- Machine register context ------------------------------------------------
RIP: 0x000055eae9ffa417 RBP: 0x00007f80aba855d8 RSP: 0x00007f80a9789598
RAX: 0x000000000000009b RBX: 0x00007f80a9789628 RCX: 0x00007f80ab9c37a0
RDX: 0x00007f80a97895c0 RDI: 0x0000000000000000 RSI: 0x000000000000009b
R8: 0x0000000000000000 R9: 0x00007f80a97895c0 R10: 0x0000000055550083
R11: 0x00007f80ac32ace0 R12: 0x00007f80aba855d8 R13: 0x00007f80ab9c3780
R14: 0x00007f80a97895c0 R15: 0x000000000000009b EFL: 0x0000000000010202
-- C level backtrace information -------------------------------------------
./miniruby(rb_vm_bugreport+0x5cf) [0x55eaea06b0ef]
./miniruby(rb_bug_for_fatal_signal+0xec) [0x55eae9e4fc2c]
./miniruby(sigsegv+0x4d) [0x55eae9fba30d]
[0x7f80ac153520]
./miniruby(rb_id_table_lookup+0x7) [0x55eae9ffa417]
./miniruby(callable_method_entry+0x103) [0x55eaea046bd3]
./miniruby(vm_respond_to+0x3f) [0x55eaea056c1f]
./miniruby(rb_check_funcall_default_kw+0x19c) [0x55eaea05788c]
./miniruby(rb_check_convert_type_with_id+0x8e) [0x55eae9f1b85e]
./miniruby(rb_str_format_m+0x1a) [0x55eae9fce82a]
./miniruby(vm_call_cfunc_with_frame+0x127) [0x55eaea041ac7]
./miniruby(vm_exec_core+0x114) [0x55eaea05d684]
./miniruby(rb_vm_exec+0x187) [0x55eaea04e747]
./miniruby(rb_funcallv_scope+0x1b0) [0x55eaea05a770]
./miniruby(rb_fiber_scheduler_unblock+0x3e) [0x55eae9fb979e]
./miniruby(sync_wakeup+0x10d) [0x55eae9ffd45d]
./miniruby(rb_szqueue_pop+0xf5) [0x55eae9ffefd5]
./miniruby(vm_call_cfunc_with_frame+0x127) [0x55eaea041ac7]
./miniruby(vm_exec_core+0x114) [0x55eaea05d684]
./miniruby(rb_vm_exec+0x187) [0x55eaea04e747]
./miniruby(rb_vm_invoke_proc+0x5f) [0x55eaea05584f]
./miniruby(rb_fiber_start+0x1da) [0x55eae9e1e24a]
./miniruby(fiber_entry+0x0) [0x55eae9e1e550]
</code></pre>
<p>I've attached the rest of the VM dump. <code>make runruby</code> gives a nearly identical dump. I can post a core dump or <code>rr</code> recording, if needed.<br>
_<br>
I'm sorry I didn't simplify the script more; small, seemingly irrelevant changes can change the failure or allow it to pass. Sometimes it raises a bizarre exception instead of SEGV, most commonly a NoMethodError which seemingly indicates that the local vars have been shifted or scrambled. For example, this particular SEGV was caused by a guard clause checking that <code>unblock(blocker, fiber)</code> was given a Fiber object. Here, that object is invalid, but I've seen it be a string or some other object from elsewhere in the process.</p>
<p>For comparison, this is what the script output should look like:</p>
<pre><code>========================================================================
fiber_queue
completed in 0.00031569297425448895
========================================================================
fiber_sized_queue
completed in 0.1176840600091964
========================================================================
fiber_sized_queue2
completed in 0.19209402799606323
========================================================================
fiber_sized_queue3
completed in 0.21404067997355014
========================================================================
fiber_sized_queue4
completed in 0.30277197097893804
</code></pre>
<p>I was attempting to create some simple benchmarks for <code>Queue</code> and <code>SizedQueue</code> with fibers, to mimic <code>benchmark/vm_thread_*queue*.rb</code>. I never completed the benchmarks because of this SEGV. :)</p> Ruby master - Feature #18227 (Open): Static class initialization.https://bugs.ruby-lang.org/issues/182272021-09-27T03:49:35Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>As a follow on from <a href="https://bugs.ruby-lang.org/issues/18189" class="external">https://bugs.ruby-lang.org/issues/18189</a> I would like to propose some kind of static class initialization. I'll investigate whether it's possible and create a PR.</p> Ruby master - Feature #16833 (Open): Add Enumerable#empty?https://bugs.ruby-lang.org/issues/168332020-05-06T14:09:02Zf3ndot (Justin Bull)
<p>It was surprising to me that Enumerator, something mixed into Array, does not include <code>#empty?</code>. I think it is reasonable to assume people may have to guard iterating and other logic based on the emptiness of an enumerator, such was my case.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="c1"># pretend there's convoluted enumerator logic to produce this structure</span>
<span class="n">table_rows</span> <span class="o">=</span> <span class="p">[{</span> <span class="ss">data: </span><span class="p">[</span><span class="s1">'First'</span><span class="p">,</span> <span class="s1">'Second'</span><span class="p">,</span> <span class="s1">'Third'</span><span class="p">],</span> <span class="ss">config: </span><span class="p">{}</span> <span class="p">},</span> <span class="p">{</span> <span class="ss">data: </span><span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">config: </span><span class="p">{</span> <span class="ss">color: </span><span class="s1">'red'</span> <span class="p">}</span> <span class="p">}].</span><span class="nf">to_enum</span>
<span class="k">return</span> <span class="k">if</span> <span class="n">table_rows</span><span class="p">.</span><span class="nf">empty?</span>
<span class="n">table_header</span> <span class="o">=</span> <span class="n">table_rows</span><span class="p">.</span><span class="nf">first</span><span class="p">[</span><span class="ss">:data</span><span class="p">]</span> <span class="c1"># requires an empty guard</span>
<span class="c1"># ...</span>
</code></pre>
<p>I propose that it simply behaves as <code>#take(1).to_a.empty?</code> instead of aliasing to something like <code>#none?</code> because of falsey elements or <code>#size == 0</code> because of potential <code>nil</code> returns:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">[].</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">empty?</span> <span class="c1"># => true</span>
<span class="p">[</span><span class="kp">false</span><span class="p">].</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">empty?</span> <span class="c1"># => false</span>
<span class="p">[</span><span class="kp">nil</span><span class="p">].</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">empty?</span> <span class="c1"># => false</span>
<span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">empty?</span> <span class="c1"># => false</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">to_enum</span><span class="p">.</span><span class="nf">empty?</span> <span class="c1"># => false</span>
</code></pre>