Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112024-03-22T01:58:03ZRuby Issue Tracking System
Redmine Ruby master - Misc #20387 (Open): Meta-ticket for ASAN supporthttps://bugs.ruby-lang.org/issues/203872024-03-22T01:58:03Zkjtsanaktsidis (KJ Tsanaktsidis)kjtsanaktsidis@gmail.com
<p>I was asked to provide a bit of information about the current status of ASAN in CRuby, so I thought I'd open this meta-ticket to track all of the work I've been performing on fixing up address sanitizer support.</p>
<p>So far, I have fixed the following issues related to ASAN support:</p>
<ul>
<li>
<a href="https://bugs.ruby-lang.org/issues/20001" class="external">https://bugs.ruby-lang.org/issues/20001</a> + <a href="https://github.com/ruby/ruby/pull/9505" class="external">https://github.com/ruby/ruby/pull/9505</a>, which dealt with two main themes:
<ul>
<li>Pushing the logic for capturing the start of the machine stack much closer to the top of the call stack, so that VALUEs stored close to the top of the machine stack get marked properly</li>
<li>Marking VALUEs stored on ASAN fake stacks during machine stack marking</li>
</ul>
</li>
<li>
<a href="https://bugs.ruby-lang.org/issues/20220" class="external">https://bugs.ruby-lang.org/issues/20220</a> + <a href="https://github.com/ruby/ruby/pull/9734" class="external">https://github.com/ruby/ruby/pull/9734</a>, which made M:N threading notify ASAN about stack switches in the same way that fibers do
<ul>
<li>Note: ASAN still doesn't work with M:N threading, but that actually has nothing to do with ASAN; it's because the most recent versions of Clang which are needed for ASAN just don't work with M:N threading either. See <a href="https://bugs.ruby-lang.org/issues/20243" class="external">https://bugs.ruby-lang.org/issues/20243</a> for more info about that.</li>
</ul>
</li>
<li>
<a href="https://bugs.ruby-lang.org/issues/20273" class="external">https://bugs.ruby-lang.org/issues/20273</a> + <a href="https://github.com/ruby/ruby/pull/10012" class="external">https://github.com/ruby/ruby/pull/10012</a>, which disables <code>callcc</code> (and the associated tests) when ASAN is enabled
<ul>
<li>callcc is very rarely used in real code and the way it works is just fundamentally incompatible with ASAN (it performs longjmp's which I think are technically undefined behaviour)</li>
</ul>
</li>
<li>
<a href="https://bugs.ruby-lang.org/issues/20221" class="external">https://bugs.ruby-lang.org/issues/20221</a> + <a href="https://github.com/ruby/ruby/pull/9865" class="external">https://github.com/ruby/ruby/pull/9865</a>, which ignore some global symbols that ASAN defines from the global symbol leak checker</li>
<li>
<a href="https://bugs.ruby-lang.org/issues/20274" class="external">https://bugs.ruby-lang.org/issues/20274</a> + <a href="https://github.com/ruby/ruby/pull/10087" class="external">https://github.com/ruby/ruby/pull/10087</a>, which ignores some false positive tests about memory leaks when ASAN is enabl</li>
<li>I updated the ASAN docs in <a href="https://github.com/ruby/ruby/pull/9922" class="external">https://github.com/ruby/ruby/pull/9922</a> to more closely reflect current reality</li>
</ul>
<p>The current state of things is that, by following the instructions in <a href="https://github.com/ruby/ruby/blob/master/doc/contributing/building_ruby.md" class="external">https://github.com/ruby/ruby/blob/master/doc/contributing/building_ruby.md</a>, you can successfully build Ruby with ASAN enabled, however, the test suite has several failures. I'm currently working on addressing these:</p>
<p>The next step is to merge <a href="https://github.com/ruby/ruby/pull/10122" class="external">https://github.com/ruby/ruby/pull/10122</a> (<a href="https://bugs.ruby-lang.org/issues/20310" class="external">https://bugs.ruby-lang.org/issues/20310</a>) which I plan to do next week (I'm currently away on a work trip). That makes sure that VALUEs stored in ASAN fake stacks from threads other than the currently running thread get marked during GC.</p>
<p>After that, I need to push up patches for the remaining few issues. I mostly have these patches ready to go already; in fact, last week I got the full <code>make check</code> suite passing all tests with ASAN enabled!</p>
<p>Once that's working, I'd like to investigate how ASAN can fit into CRuby's CI matrix somewhere so that it <em>stays</em> working, although I have not thought too deeply about this yet.</p>
<p>I will provide further updates on this ticket so anybody interested can stay in the loop.</p> Ruby master - Misc #20336 (Open): DevMeeting-2024-04-17https://bugs.ruby-lang.org/issues/203362024-03-14T10:17:44Zmame (Yusuke Endoh)mame@ruby-lang.org
<a name="The-next-dev-meeting"></a>
<h1 >The next dev meeting<a href="#The-next-dev-meeting" class="wiki-anchor">¶</a></h1>
<p><strong>Date: 2024/04/17 13:00-17:00</strong> (JST)<br>
Log: <em>TBD</em></p>
<ul>
<li>Dev meeting <em>IS NOT</em> a decision-making place. All decisions should be done at the bug tracker.</li>
<li>Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly.</li>
<li>Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue).</li>
<li>We will write a record of the discussion in the file or to each ticket in English.</li>
<li>All activities are best-effort (keep in mind that most of us are volunteer developers).</li>
<li>The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time.</li>
<li>
<em>DO NOT</em> discuss then on this ticket, please.</li>
</ul>
<a name="Call-for-agenda-items"></a>
<h1 >Call for agenda items<a href="#Call-for-agenda-items" class="wiki-anchor">¶</a></h1>
<p>If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format:</p>
<pre><code>* [Ticket ref] Ticket title (your name)
* Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.)
</code></pre>
<p>Example:</p>
<pre><code>* [Feature #14609] `Kernel#p` without args shows the receiver (ko1)
* I feel this feature is very useful and some people say :+1: so let discuss this feature.
</code></pre>
<ul>
<li>It is recommended to add a comment by 2024/04/14. We hold a preparatory meeting to create an agenda a few days before the dev-meeting.</li>
<li>The format is strict. We'll use <a href="https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86" class="external">this script to automatically create an markdown-style agenda</a>. We may ignore a comment that does not follow the format.</li>
<li>Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion.</li>
</ul> Ruby master - Misc #20320 (Open): Using OSU Open Source Lab native ppc64le/s390x CI services trig...https://bugs.ruby-lang.org/issues/203202024-03-01T16:26:28Zjaruga (Jun Aruga)
<p>We have been using Travis CI to run unit tests the native arm64/ppc64le/s390x in the ruby/ruby, ruby/zlib and ruby/prism repositories in the Ruby project.</p>
<p>One of the challenges is Travis CI's chronic unstable infra issues. To be fair, folks at Travis CI support are helpful, usually responding quickly, sometimes not. And I hope Travis CI will find the root causes of the issues. However, I would like to find alternative way to run the native ppc64le/s390x pipelines on pull-requests to change the current challenge fundamentally.</p>
<p>And I heard from folks at IBM that the there were CI services that we could run on the pull-requests. The CI services are Jenkins services provided by Oregon State University (OSU) Open Source Lab below. The CI services are the only alternative of running the native ppc64le/s390x pipelines on pull-requests as far as I know. As a note, we are already using SSH-accessing servers provided by OSU Open Source Lab.</p>
<ul>
<li>PowerPC64 (ppc64le): <a href="https://osuosl.org/services/powerdev/" class="external">https://osuosl.org/services/powerdev/</a> - POWER Continuous Integration (POWER CI)</li>
<li>IBM Z (s390x): <a href="https://osuosl.org/services/ibm-z/" class="external">https://osuosl.org/services/ibm-z/</a> - IBM Z Continuous Integration (IBM Z CI)</li>
</ul>
<p>Today I requested the CI services from the request form pages above. And I will try the services by myself first. Then if it looks good to us, I want to migrate the Travis CI ppc64le/s390x pipelines to the OSU Open Source Lab's ones. I will comment on this ticket when there are updates.</p>
<p>Note:<br>
Oregon State University Open Source Lab (OSUOSL) support email: <a href="mailto:support@osuosl.org" class="email">support@osuosl.org</a> and <a href="https://osuosl.org/contact/" class="external">other contacts</a>.</p> Ruby master - Misc #20287 (Open): DevMeeting before or after RubyKaigihttps://bugs.ruby-lang.org/issues/202872024-02-21T08:29:56Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<p>RubyKaigi itself runs from May 15-17 (Wed-Fri) in Naha.</p>
<p>It would be nice to have a DevMeeting before or afterwards. We should try to plan ahead, so that people can make the right flight and hotel reservations early.</p>
<p>I have never been to Okinawa before and don't know any facilities that would be available.</p> Ruby master - Misc #20272 (Open): Ambiguity around Ractor message sending FIFO semanticshttps://bugs.ruby-lang.org/issues/202722024-02-16T18:26:09Zforthoney (Seong-Heon Jung)castlehoneyjung@gmail.com
<p>The docs should explicitly state the semantics/properties of Ractor message, especially when it comes to FIFO.<br>
For example, assume I have two Ractors, Ractor A and Ractor B. Ractor A sends two messages <code>"hello"</code> and <code>"world"</code> in this order to Ractor B.<br>
If I call <code>Ractor.receive</code> on Ractor B, am I guaranteed to see <code>"hello"</code> and then <code>"world"</code> or is it possible the messages are delivered out of order?</p> Ruby master - Misc #20259 (Open): Proposal to add "ruby --irb" and / or "ruby --start-irb" to sta...https://bugs.ruby-lang.org/issues/202592024-02-12T18:48:24ZrubyFeedback (robert heiler)
<p>This issue proposes to add:</p>
<p>a) a commandline flag to bin/ruby that allows the user to start irb,</p>
<p>and</p>
<p>b) allow the user to make use of ruby to start irb, via the commandline (thus, the<br>
proposal's a) and b) are naturally connected to one another, meaning that we would<br>
have a commandline flag for the binary "ruby", and this flag, when invoked, will<br>
start irb, in the current terminal, just as if we would have typed "irb")</p>
<p>Background to this proposal, including prior proposals:</p>
<p>I believe there are prior proposals that are related, such as this one here from<br>
~3 years ago:</p>
<p><a href="https://bugs.ruby-lang.org/issues/17859" class="external">https://bugs.ruby-lang.org/issues/17859</a></p>
<p>by deivid.</p>
<p>Note that my proposal is a bit different: I don't propose that "ruby" would behave<br>
like python here, but instead I would propose that bin/ruby gets a specific flag<br>
such as "--irb" and/or "--start-irb" (the latter is a bit longer to type, so the<br>
former is more convenient, but I also like that the latter, aka --start-irb is more<br>
specific, showing the intent more clearly; and we can add both flags anyway, so<br>
people can just pick what they prefer. I would probably just make an alias such<br>
as "rirb" for "ruby --start-irb" on my local system).</p>
<p>The way I came yesterday to a somewhat similar conclusion as deivid did, shall be<br>
explained next, even though my path is an indirect one.</p>
<p>As some may know, Tim (and others) are creating natalie, a ruby "dialect" (well,<br>
it'll be ruby in the end I believe but right now it does not satisfy the full<br>
ruby specification):</p>
<p><a href="https://github.com/natalie-lang/natalie" class="external">https://github.com/natalie-lang/natalie</a></p>
<p>Although there are some issues natalie currently has (it's a bit slow right now, for<br>
instance), they did invest quite a lot of time into it and it is slowly shaping up -<br>
at the least presently. More work to be done on satisfying ruby's specification.</p>
<p>Yesterday I compiled natalie from a git checkout and it compiled and installed fine.<br>
I was confused that under bin/ there was only "natalie", but no irb. I may be wrong<br>
but I think in the past I had bin/irb there too. Anyway, that then got me thinking<br>
that I may have made a mistake - but, also, why can't we just use ruby to start<br>
irb, as-is? That is, the executable called "ruby", under bin/ruby.</p>
<p>Yes, this is natalie, not ruby, I understand that. I also fully understand that MRI<br>
ruby having --irb and --start-irb does not change the situation for natalie, jruby,<br>
truffleruby - you name it. I get that, although I should like to point out that they<br>
may often follow suit what MRI ruby decides, so if they have a variant of irb, it<br>
may be likely that they would add the same or similar commandline flags to support<br>
changes in MRI, even if it may take a while (see jruby issue tracker having tons<br>
of todo-entries on github, to keep track of new ruby releases and what changed<br>
between version).</p>
<p>But I was then also wondering why ruby itself would not allow us to start irb in such<br>
a way, via "ruby". I understand that we have bin/irb, so this is not a real problem -<br>
people can just start "irb" normally. And I think we can require irb from a ruby .rb<br>
file just fine, as well, and use irb that way, all fine too. But even then I still<br>
wondered why we can not run irb via "ruby". Is there any reason that speaks against<br>
"ruby --start-irb" starting irb? I don't think the stdlib idea for ruby was to prevent<br>
ruby itself from providing convenience to the user. I also don't propose other gems<br>
to be started like that either, supporting many different commandline flags or<br>
--start=name_of_gem_here; irb is more important, in my opinion, than many other<br>
gems. The key idea for irb is to have "interactive ruby". pry pushed that idea<br>
further, and I think the recent irb releases included some ideas from pry too. ALL<br>
of these focus on an "interactive ruby", no matter the real name (be it irb,<br>
pry or anything else) - just as "python" without arguments focuses on "interactive<br>
python".</p>
<p>If we look at python, python kind of provides two ways to run things: "python foobar.py",<br>
and without argument, the interactive python. Again, I am not proposing to join that<br>
together, different to deivid's proposal, but indeed, it made me question ruby's current<br>
behaviour.</p>
<p>I think python's behaviour to be able to select either way, via "python" itself, is better;<br>
that we are unable to select which behaviour to use via commandline flag, in python, is a<br>
drawback, as jeremy pointed out since he prefers the current ruby heaviour, but via an<br>
additional commandline flag I think we can at the least get python's interactive behaviour<br>
into ruby (that is, IRB; although it should be mentioned that the new IRB is a bit different<br>
to the older IRB, but I leave it at that, since the proposal here is about starting IRB,<br>
not about IRB's behaviour - the latter is for the github project of IRB, rather than MRI's<br>
issue tracker here).</p>
<p>deivid reasoned that python's behaviour is useful for newbies, which I do not doubt. jeremy<br>
said that he prefers the current behaviour of ruby, which is also fine. An explicit commandline<br>
flag for starting irb may be a compromise, but I should like to say that my rationale today<br>
was different from deivid's one, in that I actually really want bin/ruby to give us a SIMPLE<br>
means to start irb, rather than merely "ruby" showing the same behaviour as "python" does,<br>
in regarsd to starting its interactive variant.</p>
<p>Again, I can work around this just fine, using a .rb file to start IRB and so forth, but I<br>
really think having this supported via a simple commandline flag may be more convenient. Of<br>
course, a secondary proposal to this may then be to add another commandline flag to get<br>
to support deivid's suggestion :) - but that would also be for another issue request on the<br>
bugtracker here. For this proposal, I really merely suggest to add --irb and/or --start-irb<br>
(or any other variant that may be more fitting, perhaps --interactive; the name is not so<br>
important, although it should be a convenient name and ideally short. The functionality is<br>
more important than the name.)</p>
<p>Implementation details:</p>
<p>Let's for a moment assume people may find this useful. Then the question is, which irb should<br>
be started when using "ruby --irb"?</p>
<p>Probably the most sensible default option here is to simply use whatever irb version has been<br>
installed on the target computer. Hiroshi Shibata and others worked towards gemifying ruby's<br>
stdlib, so it makes sense that "ruby --start-irb" would use the currently installed irb version,<br>
whatever that is. For most ruby users this is probably the one that came distributed with<br>
whatever ruby version they are using, at the least for modern ruby versions.</p>
<p>Otherwise they may just install a new irb version from e. g.</p>
<pre><code>https://github.com/ruby/irb
</code></pre>
<p>or whatever:</p>
<pre><code>gem install irb
</code></pre>
<p>installs. And this is then started when they do "ruby --start-irb". So, "ruby --start-irb"<br>
would be exactly the same as the user typing "irb" on the commandline, without any further<br>
arguments.</p>
<p>Of course the user can also start irb via "irb", as-is. So they'd only get an additional<br>
way to start irb. They can continue to use "irb" just fine. All that would change is that<br>
"ruby" itself would get an additional commandline argument, one that starts its "interactive<br>
part".</p>
<p>Anyway - I have no idea how much support the above would find; perhaps if jeremy is active<br>
he could compare it towards deivid's suggestion and comment on it too. I thought about<br>
appending to deivid's suggestion, but the suggestion here is a bit different, so I think<br>
it may be better to propose it as a separate one, even though it is somewhat close to his<br>
proposal, even though it is unrelated (see how I reached that conclusion, which I think is<br>
different than deivid's proposal).</p>
<p>I am not sure if there are even older proposals or not, but there may be. It's also fine<br>
to close this issue here, by the way, to keep the issue tracker list shorter (and perhaps<br>
link it towards deivid's above issue too, if this is closed, which is fine). I kind of<br>
wanted to contribute a bit to the discussion from a different point of view. Perhaps to<br>
avoid having too many open issues, this issue here could be left open for a month or<br>
two, to allow for a bit of more discussion, then it can be closed and e. g. referred to<br>
deivids' issue; and if deivid has time he may comment on the one here or his older<br>
proposal. At any rate, thank you for reading this proposal.</p>
<p>(PS: I filed it under Misc because it did not seem to necessarily be a Feature of the<br>
ruby language itself. Of course it could also fit under Feature, so I was not sure<br>
where to categorize it.)</p> Ruby master - Misc #20240 (Open): Unable to build ruby 3.1.0 on macOS when shared due to dylibs (...https://bugs.ruby-lang.org/issues/202402024-02-06T10:53:26Zjmarrec (Julien Marrec)
<p>I am trying to develop a conan (the C/C++ package manager) recipe for Ruby. The recipe would allow downstream users to 1) get a runnable ruby executable, and 2) be able to link to ruby, or embbed it in a C/C++ program if built statically, in an easy way.</p>
<p>Currently there is an existing ruby 3.1.0 recipe that I'm trying to adapt, so I have to support this version.</p>
<p>First off, let me say that I can succesfully build with 3.3.0, so I know something has changed for the better since then. I'm just at a lost when figuring out what I need to backport to make 3.1.0 work.</p>
<p>The original issue is that it appears miniruby is looking for some dylibs and not finding them. Even if I do define <code>LD_LIBRARY_PATH</code>, <code>DYLD_LIBRARY_PATH</code> or <code>DYLD_FALLBACK_LIBRARY_PATH</code> (any combinations of these three) in my env.</p>
<pre><code class="shell syntaxhl" data-language="shell">dsymutil exe/ruby<span class="p">;</span> <span class="o">{</span> <span class="nb">test</span> <span class="nt">-z</span> <span class="s1">''</span> <span class="o">||</span> codesign <span class="nt">-s</span> <span class="s1">''</span> <span class="nt">-f</span> exe/ruby<span class="p">;</span> <span class="o">}</span>
./miniruby <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">'prog, dest, inst = ARGV; dest += "/ruby"'</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">'exit unless prog==inst'</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">'unless prog=="ruby"'</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">' begin File.unlink(dest); rescue Errno::ENOENT; end'</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">' File.symlink(prog, dest)'</span> <span class="se">\</span>
<span class="nt">-e</span> <span class="s1">'end'</span> <span class="se">\</span>
ruby exe ruby
dyld[59344]: Library not loaded: @rpath/libgmp.10.dylib
Referenced from: <356E0011-6223-321A-9179-D55618D248D0> /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby
Reason: no LC_RPATH<span class="s1">'s found
make: *** [exe/ruby] Abort trap: 6
make: *** Deleting file `exe/ruby'</span>
</code></pre>
<p>It seems that something is unsetting the variables, because this for eg works fine</p>
<pre><code class="shell syntaxhl" data-language="shell"><span class="nv">DYLD_LIBRARY_PATH</span><span class="o">=</span>/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/lib:/Users/julien/.conan2/p/b/opense854e464e8ff6/p/lib:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/lib:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/lib:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/lib:/Users/julien/.conan2/p/b/termc22b5bb1515971/p/lib:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p/lib: /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby <span class="nt">-e</span> <span class="s2">"puts 'Hello, world'"</span>
</code></pre>
<p>My configure call is like this:</p>
<pre><code class="shell syntaxhl" data-language="shell">./configure <span class="nt">--enable-shared</span> <span class="nt">--disable-static</span> <span class="nt">--prefix</span><span class="o">=</span>/ <span class="s1">'--bindir=${prefix}/bin'</span> <span class="s1">'--sbindir=${prefix}/bin'</span> <span class="s1">'--libdir=${prefix}/lib'</span> <span class="s1">'--includedir=${prefix}/include'</span> <span class="s1">'--oldincludedir=${prefix}/include'</span> <span class="nt">--disable-install-doc</span> <span class="nt">--enable-load-relative</span> <span class="nt">--with-zlib-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p <span class="nt">--with-openssl-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/opense854e464e8ff6/p <span class="nt">--with-libffi-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p <span class="nt">--with-libyaml-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p <span class="nt">--with-readline-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/readl0d0041a63fa03/p <span class="nt">--with-gmp-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p <span class="nt">--with-opt-dir</span><span class="o">=</span>/Users/julien/.conan2/p/b/opense854e464e8ff6/p:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p <span class="nt">--disable-jit-support</span>
</code></pre>
<p>I have tried to backport <a href="https://github.com/ruby/ruby/pull/6296/files" class="external">https://github.com/ruby/ruby/pull/6296/files</a> and <a href="https://github.com/ruby/ruby/commit/48644e71096c70132be9dfdcbfb414ec2e68d18b" class="external">https://github.com/ruby/ruby/commit/48644e71096c70132be9dfdcbfb414ec2e68d18b</a> and <a href="https://github.com/ruby/ruby/pull/8730" class="external">https://github.com/ruby/ruby/pull/8730</a> amongst other things but I can't make it work. (I even tried a more brute force approach patching a lot of files by diffing 3.3.0 with 3.1.0, but please note I don't know what I'm doing... and I can get to the install step but then I get some errors about Psych / libymal and undefined Gem::Install:Zlib).</p>
<p>I would <strong>greatly</strong> appreciate if someone can spare some time to help me wrap this up (I've been trying to make the recipe work for so long that I'm about to give up...)</p> Ruby master - Misc #20238 (Open): Use prism for mk_builtin_loader.rbhttps://bugs.ruby-lang.org/issues/202382024-02-05T20:52:02Zkddnewton (Kevin Newton)kddnewton@gmail.com
<p>I would like to propose that we use prism for mk_builtin_loader.rb.</p>
<p>Right now the Ruby syntax that you can use in builtin classes is restricted to the base Ruby version (2.7). This means you can't use a lot of the nicer syntax that Ruby has shipped in the last couple of years.</p>
<p>If we switch to using prism to parse the builtin files instead of using ripper, then we can always use the latest version of Ruby syntax. A pull request for this is here: <a href="https://github.com/kddnewton/ruby/pull/65" class="external">https://github.com/kddnewton/ruby/pull/65</a>. The approach for the PR is taken from how RJIT bindgen works.</p> Ruby master - Misc #20232 (Open): Document Kernel#require and Module#autoload concurrency guaranteeshttps://bugs.ruby-lang.org/issues/202322024-02-01T05:20:04Zfxn (Xavier Noria)fxn@hashref.com
<p>I'd like to document <code>Kernel#require</code> and <code>Module#autoload</code> concurrency guarantees.</p>
<p>In the case of multiple threads loading the same file concurrently, <code>Kernel#require</code> will succeed in just one of them and the rest will wait and return false. If a constant that has an autoload is concurrently referenced, the same can be said. Assuming no errors, only one thread will succeed, and the rest wait. There will be no context switching in the middle of an autoload that will result in a <code>NameError</code> in other threads waiting for that constant.</p>
<p>Now, I'd like to have a discussion about those guarantees with fibers.</p>
<p>In the case of manually managed fibers, users can enter a deadlock by hand:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># This produces a deadlock.</span>
<span class="no">File</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="s1">'/tmp/bar.rb'</span><span class="p">,</span> <span class="o"><<~</span><span class="no">RUBY</span><span class="p">)</span><span class="sh">
Fiber.yield
</span><span class="no">RUBY</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="nb">require</span> <span class="s1">'/tmp/bar.rb'</span> <span class="p">}.</span><span class="nf">resume</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="nb">require</span> <span class="s1">'/tmp/bar.rb'</span> <span class="p">}.</span><span class="nf">resume</span>
</code></pre>
<p>If this is expected, I guess users should be told this is a possibility in the API dosc? Because from a user perspective, you don't really have elements to anticipate a deadlock there if the docs don't warn you.</p>
<p>A similar deadlock can be triggered with an <code>autoload</code> instead of a <code>require</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># This produces a deadlock.</span>
<span class="no">File</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="s1">'/tmp/bar.rb'</span><span class="p">,</span> <span class="o"><<~</span><span class="no">RUBY</span><span class="p">)</span><span class="sh">
Fiber.yield
Bar = 1
</span><span class="no">RUBY</span>
<span class="nb">autoload</span> <span class="ss">:Bar</span><span class="p">,</span> <span class="s1">'/tmp/bar.rb'</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="no">Bar</span> <span class="p">}.</span><span class="nf">resume</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="no">Bar</span> <span class="p">}.</span><span class="nf">resume</span>
</code></pre>
<p>A different matter is fibers managed by fiber schedulers. I have not been able to enter a deadlock with the fiber schedulers I have tried, but from the point of view of the user, doing something like I/O or sleeping at the top-level is not unlike that manual <code>Fiber.yield</code> above. The contract for fiber schedulers is mostly an interface, but it does not address this, at least in an explicit way. Do fiber schedulers guarantee anything about this with the current contract?</p>
<p>I'd be glad to volunteer docs with the conclusions of this thread.</p> Ruby master - Misc #20170 (Open): Drop support for GCC < 11https://bugs.ruby-lang.org/issues/201702024-01-09T13:40:59Zkddnewton (Kevin Newton)kddnewton@gmail.com
<p>Right now, CI compiles everything from GCC 7+. However, GCC 7-10 are all end-of-life and no longer supported. We should drop support for the end-of-life compilers.</p> Ruby master - Misc #20156 (Open): C99 updateshttps://bugs.ruby-lang.org/issues/201562024-01-05T23:11:12Zkddnewton (Kevin Newton)kddnewton@gmail.com
<p>Now that we require Visual Studio 2015 and Oracle Developer Studio 12.5, we can do a couple of things to clean up:</p>
<ul>
<li>We should be able to remove our custom <code>stdbool.h</code> (added in VS2015 and Oracle SS 12.3)</li>
<li>Remove our usage of <code>__restrict</code> for Windows (added in VS2015)</li>
<li>Remove our custom <code>va_copy</code> (added in VS2015)</li>
</ul>
<p>Just notating, a couple of things that we <em>can</em> use now:</p>
<ul>
<li>snprintf</li>
<li>size_t printf format specifier (%z)</li>
</ul>
<p>I'm happy to do this work, but I wanted to open this ticket in case there is something besides Visual Studio or Oracle Solaris Studio that I would have missed for one of our compilation targets. Are there any other targets that do not fully implement C99?</p> Ruby master - Misc #20013 (Open): Travis CI statushttps://bugs.ruby-lang.org/issues/200132023-11-21T14:10:56Zjaruga (Jun Aruga)
<p>I would like to use this ticket to manage our activities to report Travis CI status.</p>
<p>Because there is Travis CI status page provided by Travis CI. However, even when the page shows ok, I actually see infra issues.<br>
<a href="https://www.traviscistatus.com/" class="external">https://www.traviscistatus.com/</a></p>
<p>I would share my activities and report the Travis CI status on the ticket.<br>
The ticket's status is not closed until we stop using Travis CI.</p>
<p>The easiest option to fix the Travis infra issue is to email Travis CI support <code>support _AT_ travis-ci.com</code>.</p>
<p>You can check <a href="https://github.com/ruby/ruby/wiki/CI-Servers#travis-ci" class="external">this ruby/ruby Travis CI wiki page</a> for details.</p> Ruby master - Misc #19971 (Open): Confusing arity of a Proc with implicit rest parameterhttps://bugs.ruby-lang.org/issues/199712023-10-24T13:07:51Zandrykonchin (Andrew Konchin)
<p>I've noticed that such proc <code>proc { |a,| }</code> has arity <code>1</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="p">,</span><span class="o">|</span> <span class="p">}.</span><span class="nf">arity</span> <span class="c1"># => 1</span>
</code></pre>
<p>that means only one required parameter, but the proc behaves similar to a proc with explicit rest parameter (<code>proc { |a, *| }</code>) that has arity <code>-2</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="o">*|</span> <span class="p">}.</span><span class="nf">arity</span> <span class="c1"># => -2</span>
</code></pre>
<p>that means one required parameter and rest parameter.</p>
<p>So I am wondering whether it's intentional behaviour and what the rational is behind it.</p> Ruby master - Misc #19912 (Open): [Small potential for slight improvement in regards to trailing ...https://bugs.ruby-lang.org/issues/199122023-10-06T05:57:07ZrubyFeedback (robert heiler)
<p>Not sure if I should file this under bug or not; it is probably not a bug per se<br>
and just the current behaviour. So I filed it under Misc here.</p>
<p>So let me copy/paste the code next:</p>
<pre><code>alias e puts
i = 'abc'
case i
when /help
e 'No help options are currently documented.'
when /-?-?rotate=(\d+)$/i
pp $1.to_s
exit
end
end
</code></pre>
<p>I isolated this from a larger .rb file. I was adding support for<br>
--rotate= for a class tasked with commandline stuff in regards to<br>
.pdf files, e. g. rotating wrong .pdf files.</p>
<p>When I run this .rb file, that is the one I showed above, I get<br>
the following error, as-is (the extra "end" is kind of deliberate,<br>
I had a much larger case/when structure before).</p>
<pre><code>foo.rb: --> foo.rb
Unmatched `end', missing keyword (`do', `def`, `if`, etc.) ?
> 4 i = 'abc'
> 6 case i
> 7 when /help
> 8 e 'No help options are currently documented.'
> 9 when /-?-?rotate=(\d+)$/i
> 12 end
> 14 end
foo.rb:9: syntax error, unexpected backslash (SyntaxError)
when /-?-?rotate=(\d+)$/i
^
</code></pre>
<p>Because there are at the least two problems in this .rb file,<br>
it appears to be that the ruby parser focuses on the last<br>
error found. To those who can not instantly find the error,<br>
I simply forgot a trailing / in the case/when menu, at<br>
"when /help". That actually was the real bug too. But I first<br>
looked down, where ruby alerted me that there is an issue<br>
with "when /-?-?rotate=(\d+)$/i". The real problem was the<br>
missing "/".</p>
<p>Is there potential to improve handling of such small mistakes,<br>
specifically someone forgetting a trailing / in a case/when<br>
menu? I assume ruby has to decide what is going on, but usually<br>
when I use a case/when menu, I have multiple such entries, so<br>
perhaps this gives the ruby parser additional information it<br>
could use, such as: "look at the immediate surrounding and<br>
then determine whether this may be a missing /".</p>
<p>I don't know how difficult it is to add that, and if it is too<br>
much work then I think the time may be spent elsewhere. But on<br>
the other hand, if the ruby parser could detect such small<br>
mistakes more easily (a forgotten /), this would be rather nice<br>
to have.</p>
<p>This may also have to do with how ruby reports errors; I guess<br>
the part "Unmatched <code>end', missing keyword (</code>do', <code>def</code>, <code>if</code>,<br>
etc.) ?" is ruby's attempt to give useful information to the<br>
user, and in many cases this should work. But in case of simple<br>
errors such as the above it may not work, and secondary errors<br>
may disguise or confuse the user momentarily. I usually look at<br>
the last part of any long trace shown by the ruby parser to me,<br>
so that is why I sometimes miss prior errors.</p> Ruby master - Misc #19767 (Open): [Not really a bug, but more a not ideal notification] "historic...https://bugs.ruby-lang.org/issues/197672023-07-14T08:10:20ZrubyFeedback (robert heiler)
<p>To get my knowledge about ruby regexes up-to-date I have been<br>
going through this tutorial/book here at:</p>
<p><a href="https://learnbyexample.github.io/Ruby_Regexp/unicode.html" class="external">https://learnbyexample.github.io/Ruby_Regexp/unicode.html</a></p>
<p>One example they provide is this, with some odd characters:</p>
<pre><code>'fox:αλεπού'.scan(/\w+/n)
</code></pre>
<p>This will match the found word ("fox"), but it also reports<br>
the following warning:</p>
<pre><code>warning: historical binary regexp match /.../n against UTF-8 string
</code></pre>
<p>Now: this may be obvious to others, but to me personally I am not<br>
sure what a "historical" binary regexp match actually is. I assume<br>
it may have meant that this was more used in the past, and may be<br>
discouraged now? Or is something else meant? What does "historical"<br>
mean in this context?</p>
<p>I may not be the only one who does not fully understand the term<br>
historical. Most of ruby's warnings are fairly easy to understand,<br>
but this one seems odd. Right now I do not know whether we can use<br>
the "n" modifier in a regex - not that I really have a good use<br>
case for it (I am using UTF-8 these days, so I don't seem to need<br>
ASCII-8BIT anyway), but perhaps the warning could be changed a little.</p>
<p>I have no good alternative suggestion how it can be changed, largely<br>
because I do not know what it actually means, e. g. what is "historical"<br>
about it (but, even then, I'd actually recommend against using the<br>
word "historical" because I don't understand what it means; deprecated<br>
is easy to understand, historical does not tell me anything).</p>
<p>Perhaps it could be expressed somewhat differently and we could get<br>
rid of the word "historical" there? Either way, it's a tiny issue so<br>
I was not even sure whether to report it. But, from the point of view<br>
of other warnings, I believe the term "historical" does not tell the<br>
user enough about what the issue is here.</p>
<p>(irb):1: warning: historical binary regexp match /.../n against UTF-8 string<br>
=> ["fox"]</p> Ruby master - Misc #19758 (Open): Statically link ext/jsonhttps://bugs.ruby-lang.org/issues/197582023-07-05T13:24:24ZMyCo (Maik Menz)
<p>Hi,</p>
<p>I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version.<br>
In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place.</p>
<p>I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib.<br>
With my limited knowledge of the building processes, my first attempt was to add both of those libs into <code>LOCAL_LIBS</code> and now they apear in the linking process.<br>
But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build.</p>
<p>What am I missing? What do I have to do, to get those linked into the static lib?</p>
<p>Regards<br>
Maik</p> Ruby master - Misc #19692 (Open): Net::HTTP Performance Workstreamhttps://bugs.ruby-lang.org/issues/196922023-05-24T20:21:17Zbaweaver (Brandon Weaver)keystonelemur@gmail.com
<p>While working on identifying causes behind Capybara test slowness I had noticed some memory profiles pointed at a few slow code paths in <code>Net::HTTP</code>, one of which I've already opened a PR for here:</p>
<p><a href="https://github.com/ruby/net-http/pull/140" class="external">https://github.com/ruby/net-http/pull/140</a></p>
<p>The reason I'm opening this ticket is that I had found some additional areas which warrant investigation for some decent performance gains in frequently run sections of code in <code>Net::HTTP</code> and to ask if someone would be willing to work with me on opening likely several PRs against the code base in the near future.</p>
<p>I will hold myself to the standard of qualifying these changes with performance data on which parts are frequently run and are slow, but would appreciate working with someone from core through this.</p> Ruby master - Misc #19691 (Open): File.realpath on Windows does not return actual case of the filehttps://bugs.ruby-lang.org/issues/196912023-05-24T15:22:53ZMSP-Greg (Greg L)
<p>I don't know what I think of this (expected or bug), but recently in a forum I saw an issue that I don't recall accidentally doing myself. An equivalent example:</p>
<pre><code>ruby -rsingleton -rSingleton -e "puts 'case issue'"
</code></pre>
<p>Note the change of casing with <code>Singleton</code>.</p>
<p>On case insensitive file systems, the above will double load the file, which errors.</p>
<p>Obviously, on case sensitive file systems, it errors with <code>cannot load such file -- Singleton</code></p>
<p>So, options are:</p>
<p>A. No change to current behavior, which I'm fine with.</p>
<p>B. If the OS is case insensitive, don't reload the file, and maybe a warning that <code>Singleton</code> and <code>singleton</code> are equivalent.</p> Ruby master - Misc #19421 (Open): Distribution documentationhttps://bugs.ruby-lang.org/issues/194212023-02-07T05:03:49Zioquatix (Samuel Williams)samuel@oriontransfer.net
<p>I use Ruby a lot, on a lot of different systems, and help people and companies use it, including developers who install it on their systems.</p>
<p>Over time, I found that installing Ruby isn't always easy. Part of this is due to package management. There are many systems, and Ruby has had some tricky migrations (e.g. OpenSSL is probably one of the most painful ones that lots of developers have trouble with).</p>
<p>Arch Linux has been stuck on Ruby 3.0 for a long time, which could be considered surprising given that Arch Linux is often on the bleeding edge of releases. I personally use Arch too. So I decided to ask, what is holding them up from making a release?</p>
<p>I found out they had many questions about how to distribute Ruby correctly. When I listened to those questions I felt that there are many ambiguities in how we build and package Ruby for operating system packages. This isn't to say that there isn't a good way to do it, just that we as a core team might be able to improve our communication about how Ruby is evolving and the implications for package managers (if any).</p>
<p>I've introduced <code>doc/distribution.md</code> as an effort to start having better documentation for people distributing Ruby. There are many ways which people distribute Ruby, and many "partial" documentation or assumptions being made about how to distribute Ruby and I'd like to provide a convenient standard location that can help people build package for Ruby distribution. Ultimately this makes my job easier because the latest versions of Ruby will be easier to install, so that's what I care about, but I don't care about what specifically is in the document, except that I think we should listen to the kinds of questions being asked and, in the best interest of Ruby, provide guidance.</p>
<p>There was a lot of good discussion on the PR, but my goal is not to make a finished document, but instead plant the seed so it can grow.</p>
<p><a href="https://github.com/ruby/ruby/pull/6856" class="external">https://github.com/ruby/ruby/pull/6856</a></p>
<p>Some follow up discussion is required:</p>
<ul>
<li>
<p>What is the best practice for building source packages. The documentation I wrote from this was removed as "out of scope" but I disagree with that (<a href="https://github.com/ruby/ruby/commit/c35ebed895e1a3f7bced3db50ea0db8f284744e8" class="external">https://github.com/ruby/ruby/commit/c35ebed895e1a3f7bced3db50ea0db8f284744e8</a>). I don't have a strong opinion about what it should look like, but I think we should give a clear example of how to build source packages like what I wrote.</p>
</li>
<li>
<p>Related to the above, what is the official location for source tarballs?</p>
</li>
<li>
<p>What optional dependencies are required for building vs distributing Ruby. Arch Linux currently lists: <code>doxygen gdbm graphviz libffi libyaml openssl ttf-dejavu tk</code> as dependencies but it's not clear if this list is up to date, or what the expectations are. When someone installs Ruby (e.g. <code>apt-get install ruby</code>) what dependencies should be installed? I think it would be helpful to list expected dependencies (from a system package POV), etc.</p>
</li>
<li>
<p>Is Ruby needed for building Ruby? Should source packages install Ruby before building from source? If so, what versions are supported?</p>
</li>
<li>
<p>Clear guidance on gems that are distributed an alongside Ruby, and how security changes to gems are managed.</p>
</li>
</ul>
<p>Even if we have more detailed documentation elsewhere, let's summarise it and then cross-reference it. People who build packages to distribute and install Ruby should feel supported, they are very important to our community. To this end, I established <code>#distribution</code> channel on Slack for discussion. We should listen to the questions being asked and use those questions to drive improvements to documentation.</p> Ruby master - Misc #19391 (Open): IO #write/#read behaviour when binmode and encoding are explici...https://bugs.ruby-lang.org/issues/193912023-01-30T10:19:05Zandrykonchin (Andrew Konchin)
<p>I have a question about how <code>IO</code> operations are supposed to behave when file is opened in binary mode but encoding is specified too.</p>
<p>Documentation of <code>b</code> option of <code>IO.new</code> method and of <code>IO#binmode</code> method is a bit not synchronised and confusing:</p>
<p><code>IO.open</code>:</p>
<pre><code>"b" Binary file mode
Suppresses EOL <-> CRLF conversion on Windows. And
sets external encoding to ASCII-8BIT unless explicitly
specified.
</code></pre>
<p><code>IO#binmode</code>:</p>
<pre><code>Puts ios into binary mode. Once a stream is in binary mode, it cannot be reset to nonbinary mode.
newline conversion disabled
encoding conversion disabled
content is treated as ASCII-8BIT
</code></pre>
<p>So <code>unless explicitly specified</code> and <code>encoding conversion disabled</code> are a bit contradictory.</p>
<p>Right now I observe that <code>IO.write</code> still encodes an input string in explicitly specified encoding (with <code>encoding</code> option) even if binary mode is set (with <code>binmode</code> option)</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="vi">@filename</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">,</span> <span class="ss">encoding: </span><span class="no">Encoding</span><span class="o">::</span><span class="no">UTF_32LE</span><span class="p">,</span> <span class="ss">binmode: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">file</span><span class="o">|</span>
<span class="n">file</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="s2">"hi"</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">File</span><span class="p">.</span><span class="nf">binread</span><span class="p">(</span><span class="vi">@filename</span><span class="p">)</span> <span class="c1"># => "h\u0000\u0000\u0000i\u0000\u0000\u0000"</span>
</code></pre> Ruby master - Misc #19304 (Open): Kernel vs Object documentationhttps://bugs.ruby-lang.org/issues/193042023-01-03T14:00:26Zzverok (Victor Shepelev)zverok.offline@gmail.com
<p>The restating of the problems raised in <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Move public methods from Kernel to Object (Rejected)" href="https://bugs.ruby-lang.org/issues/19300">#19300</a>, from scratch.</p>
<p>I believe it is rather a topic of <strong>community significance</strong> and would be happy if it is possible to <strong>discuss on dev-meeting</strong>, even if the outcome would probably "let's change RDoc in this or that way".</p>
<p>So, the problem statement:</p>
<ol>
<li>
<code>Kernel</code> module defines two "types" of methods: private ones, that pretend to be global (like <code>puts</code>), but actually available from inside every object; and public ones (like <code>#class</code> or <code>#frozen?</code>) that are present in every object including <code>Kernel</code>.</li>
<li>Since the beginning of times, docs provided an illusion that the second type of the methods belongs to <code>Object</code> class, which is, in fact, devoid of its own definitions, just includes <code>Kernel</code>. This was handled by special RDoc hack (which, basically, forcefully <a href="https://github.com/ruby/rdoc/blob/017bb9fa496ee0e0959facba694a053008d1ecb0/lib/rdoc/parser/c.rb#L477" class="external">reattached definition</a> of public methods if they are defined in <code>Kernel</code>, to <code>Object</code>)</li>
<li>The RDoc hack was working in C code only, so after introduction of <code>kernel.rb</code> the illusion started to crumble: methods like <code>#tap</code>, <code>#then</code>, <code>#class</code> <a href="https://docs.ruby-lang.org/en/3.2/Kernel.html" class="external">are now documented</a> as belonging to <code>Kernel</code> (breaking the tradition of public methods being documented in <code>Object</code>)</li>
<li>This is all inconsistent and confusing, and I believe, adds to friction both for newcomers and curious investigators of the language!</li>
</ol>
<p>Additionally, I believe:</p>
<ol>
<li>That the distinction of "two kinds of methods" is useful (saying from a practical experience with explaining Ruby while mentoring, writing articles, and discussing with colleagues)</li>
<li>But, considering everything, I agree with what <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> and <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> say in <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Move public methods from Kernel to Object (Rejected)" href="https://bugs.ruby-lang.org/issues/19300">#19300</a>: pretending that methods belong not to the module they really belong to is not the optimal way to document things!</li>
</ol>
<p>So, the options I see (save for "do nothing and let things sort themselves somehow"):</p>
<ol>
<li>Change Ruby's implementation so public methods would really belong to <code>Object</code>. Actually, I don't believe anybody would agree to experiment on this scale, so just listing it here for completeness.</li>
<li>Just remove the RDoc hack; all methods would belong to <code>Kernel</code>, and maybe some introductory free-form text will instruct that they are different. <strong>Pro:</strong> Easy to do. <strong>Contra:</strong> The <code>Kernel</code> docs would be super-confusing.</li>
<li>Continue to maintain the hack in RDoc and extend it to <code>kernel.rb</code> (so methods like <code>#clone</code> and <code>#frozen?</code> would be still in <code>Object</code>). <strong>Pro:</strong> Relatively easy to do, will maintain structure many people used to. <strong>Contra:</strong> This is still a lie, methods belong to <code>Kernel</code>, not <code>Object</code>.</li>
<li>Adjust RDoc to a) be able to separately list <strong>public</strong> and <strong>private</strong> methods in two different lists, and add intro for <code>Kernel</code> explaining how to treat those. <strong>Pro:</strong> Probably the best correspondence to reality? <strong>Contra:</strong> Not sure how easy it is to change RDoc this way; and other renderers (like ruby-doc.com and rubyapi.org) would need to adjust themselves.</li>
</ol>
<p>So far, (4) looks the most reasonable (if (1) is 100% impossible, that is!)</p> Ruby master - Misc #19176 (Open): Missing insns_info for iseqs in ruby 3.2 built with debug optionshttps://bugs.ruby-lang.org/issues/191762022-12-03T13:25:59Zhurricup (Alexandr Evstigneev)hurricup@gmail.com
<p>I'm not quite sure it is a proper place to ask questions, correct me if I'm wrong.<br>
But I have a problem with ruby-3.2.0 previews. Looks like iseq is missing <code>insns_info</code>s.</p>
<p>It works fine in with pre-3.2 rubies but seems something has changed and I can't figure out - what exactly.</p>
<p>It looks like this:<br>
<img src="https://bugs.ruby-lang.org/attachments/download/9375/clipboard-202212031723-don1h.png" alt="" loading="lazy"></p>
<p>You may see that body, supposed to be a pointer to <code>iseq_insn_info_entry</code> contains some 0x7 pointing nowhere. And <code>size</code> for <code>insns_info</code> in all <code>iseq</code> I can find with iterating objspace is zero.</p> Ruby master - Misc #19142 (Open): Run test suites against 'install', based on ENV variable?https://bugs.ruby-lang.org/issues/191422022-11-22T22:39:07ZMSP-Greg (Greg L)
<p>Some time ago I believe there was discussion about:</p>
<pre><code>make
make test
make install
</code></pre>
<p>vs</p>
<pre><code>make
make install
make test
</code></pre>
<p>Some people preferred to not have to run install before test (disk space, time, etc).</p>
<p>Might an environment variable like <code>RUBY_TEST_FROM_INSTALL</code> or <code>RUBY_TEST_FROM_PATH</code> be<br>
added that would trigger the test suite to test against the Ruby install folder or the<br>
Ruby in PATH?</p>
<p>I believe a handful of files in the test system would need changes, as they load files<br>
using <code>require_relative</code>...</p>
<p>I think spec/mspec is already setup to do so.</p> Ruby master - Misc #19131 (Open): MatchData#values_at(): addressing with Range and Integer index ...https://bugs.ruby-lang.org/issues/191312022-11-15T15:33:18Zandrykonchin (Andrew Konchin)
<p>I've noticed a strange nuance and not sure whether it's intentional or not. In case it's intentional - it seems to me inconsistent.</p>
<p>I will illustrate it with the following example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sr">/(.)(.)(.)(.)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="s2">"abcde"</span><span class="p">).</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="o">..-</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># => ["abcd", "a", "b", "c", "d"]</span>
<span class="sr">/(.)(.)(.)(.)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="s2">"abcde"</span><span class="p">).</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># => [nil]</span>
</code></pre>
<p>So with a negative index we can address the whole matched string (<code>0</code> element) with a Range argument but cannot with an Integer index.</p>
<p>We can index with negative values only captured values:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="sr">/(.)(.)(.)(.)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="s2">"abcde"</span><span class="p">).</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => ["a"]</span>
<span class="sr">/(.)(.)(.)(.)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="s2">"abcde"</span><span class="p">).</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">3</span><span class="p">)</span> <span class="c1"># => ["b"]</span>
<span class="sr">/(.)(.)(.)(.)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="s2">"abcde"</span><span class="p">).</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># => ["c"]</span>
<span class="c1"># ...</span>
</code></pre>
<p>I would expect <code>Range</code> and <code>Integer</code> arguments are handled consistently and either both allow to address the first element with negative index or both disallow it.</p>
<hr>
<pre><code>ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin21]
</code></pre> Ruby master - Misc #19123 (Open): Error handling of Struct#values_at and Array#values_at is sligh...https://bugs.ruby-lang.org/issues/191232022-11-11T17:29:41Zandrykonchin (Andrew Konchin)
<p><code>Struct#values_at</code> and `Array#values_at look pretty similar and handle all the complex cases of arguments (integer Ranges, list of Integers and mixing Ranges and Integers) in the same way.</p>
<p>Error handling is similar as well. In case of invalid Range argument they behave identically:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">clazz</span> <span class="o">=</span> <span class="no">Struct</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:name</span><span class="p">,</span> <span class="ss">:director</span><span class="p">,</span> <span class="ss">:year</span><span class="p">)</span>
<span class="n">movie</span> <span class="o">=</span> <span class="n">clazz</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Sympathy for Mr. Vengeance'</span><span class="p">,</span> <span class="s1">'Chan-wook Park'</span><span class="p">,</span> <span class="mi">2002</span><span class="p">)</span>
<span class="n">array</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="c1"># end is out of range </span>
<span class="n">movie</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => ["Sympathy for Mr. Vengeance", "Chan-wook Park", 2002, nil, nil]</span>
<span class="n">array</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => [0, 1, 2, nil, nil]</span>
<span class="c1"># beginning is out of range</span>
<span class="n">movie</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="o">..</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># -5..4 out of range (RangeError)</span>
<span class="n">array</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="o">..</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># -5..4 out of range (RangeError)</span>
</code></pre>
<p>But when Integer argument is passed - they handle out of range cases differently:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># end is out of range </span>
<span class="n">movie</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => offset 4 too large for struct(size:3) (IndexError)</span>
<span class="n">array</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => [nil]</span>
<span class="c1"># beginning is out of range</span>
<span class="n">movie</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># => offset -5 too small for struct(size:3) (IndexError)</span>
<span class="n">array</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># => [nil]</span>
</code></pre>
<p>So I am wondering what is the reason of such inconvenience. I suppose there may be a reason of this difference. But I see some benefits in consistent error handling.</p> Ruby master - Misc #19122 (Open): Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's s...https://bugs.ruby-lang.org/issues/191222022-11-11T16:06:08Zsmcgivern (Sean McGivern)
<p>I'd like to propose that Ruby stops using MADV_FREE when freeing a Fiber's stack, and switches to using MADV_DONTNEED even when MADV_FREE is supported.</p>
<p>MADV_FREE is used in one place in the Ruby codebase, when freeing the stack of a freed Fiber: <a href="https://git.ruby-lang.org/ruby.git/tree/cont.c#n683" class="external">https://git.ruby-lang.org/ruby.git/tree/cont.c#n683</a></p>
<p>The comment for <code>fiber_pool_stack_free</code> says:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">// We advise the operating system that the stack memory pages are no longer being used.</span>
<span class="c1">// This introduce some performance overhead but allows system to relaim memory when there is pressure.</span>
</code></pre>
<p>Where possible (i.e. on Linux 4.5 and later), <code>fiber_pool_stack_free</code> uses <code>MADV_FREE</code> over <code>MADV_DONTNEED</code>. This has the side effect that memory statistics such as RSS will not reduce until and unless the OS actually reclaims that memory. If that doesn't happen, then the reported memory usage via RSS will be much higher than the 'real' memory usage.</p>
<p>If this was pervasive throughtout the Ruby codebase then that would be one thing, but currently this is just for Fiber. This means that:</p>
<ol>
<li>A program that doesn't use Fiber will have somewhat reliable RSS statistics on recent Linux.</li>
<li>A program that heavily uses Fiber (such as something using Async::HTTP) will see an inflated RSS statistic.</li>
</ol>
<p>Go made a similar change to the one I'm proposing here for similar reasons: <a href="https://github.com/golang/go/issues/42330" class="external">https://github.com/golang/go/issues/42330</a></p>
<blockquote>
<p>While <code>MADV_FREE</code> is somewhat faster than <code>MADV_DONTNEED</code>, it doesn't affect many of the statistics that <code>MADV_DONTNEED</code> does until the memory is actually reclaimed. This generally leads to poor user experience, like confusing stats in <code>top</code> and other monitoring tools; and bad integration with management systems that respond to memory usage.<br>
[...]<br>
I propose we change the default to prefer <code>MADV_DONTNEED</code> over <code>MADV_FREE</code>, to favor user-friendliness and minimal surprise over performance. I think it's become clear that Linux's implementation of <code>MADV_FREE</code> ultimately doesn't meet our needs.</p>
</blockquote>
<p>As an aside, MADV_FREE was not used in Ruby 3.1 (<a href="https://bugs.ruby-lang.org/issues/19101" class="external">https://bugs.ruby-lang.org/issues/19101</a>), and I haven't found any bugs filed about this behaviour other than that one.</p> Ruby master - Misc #19121 (Open): Time: utc offset argument's formats "+/-HH", "+/-HHMM", "+/-HHM...https://bugs.ruby-lang.org/issues/191212022-11-11T14:50:11Zandrykonchin (Andrew Konchin)
<p>I've noticed a strange difference between Ruby versions 2.7, 3.0 and 3.1.</p>
<p>The mentioned above formats are supported in Ruby 2.7 and Ruby 3.1 but aren't supported in Ruby 3.0.</p>
<p>So I am wondering whether supporting these formats is expected behaviour or a bug. In any case I would expect consistent behaviour in all the supported Ruby versions.</p>
<p>Ruby 2.7.4</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+05"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +0500</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+0530"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +0530</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+053037"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +0530</span>
</code></pre>
<p>Ruby 3.0.4</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+05"</span><span class="p">)</span>
<span class="c1">#(irb):1:in `initialize': "+HH:MM", "-HH:MM", "UTC" or "A".."I","K".."Z" expected for utc_offset (ArgumentError)</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+0530"</span><span class="p">)</span>
<span class="c1">#(irb):2:in `initialize': "+HH:MM", "-HH:MM", "UTC" or "A".."I","K".."Z" expected for utc_offset (ArgumentError)</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+053037"</span><span class="p">)</span>
<span class="c1">#(irb):3:in `initialize': "+HH:MM", "-HH:MM", "UTC" or "A".."I","K".."Z" expected for utc_offset (ArgumentError)</span>
</code></pre>
<p>Ruby 3.1.2</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+05"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +0500</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+0530"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +0530</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="s2">"+053037"</span><span class="p">)</span> <span class="c1"># => 2000-01-01 00:00:00 +053037</span>
</code></pre> Ruby master - Misc #19098 (Open): Time#strftime: %z and widthhttps://bugs.ruby-lang.org/issues/190982022-11-01T17:49:20Zandrykonchin (Andrew Konchin)
<p>It seems <code>%z</code> behaves in some surprising way when it is combined with a width - sign <code>+</code> is placed at the beginning of the result string:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%10z"</span><span class="p">)</span>
<span class="o">=></span> <span class="s2">"+000000200"</span>
<span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%_10z"</span><span class="p">)</span>
<span class="o">=></span> <span class="s2">" +200"</span>
</code></pre>
<p>It seems a time zone offset is treated as a number. It probably makes sense with default format but it looks strange with <code>:</code> separators:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s2">"%10::z"</span><span class="p">)</span>
<span class="c1"># => "+002:00:00"</span>
</code></pre>
<p>So I would expect that <code>%z</code> directive output to be treated as a non-numerical and padded by default with spaces.</p> Ruby master - Misc #19052 (Open): Increased memory usage (RSS) for Ruby when compiled by gcchttps://bugs.ruby-lang.org/issues/190522022-10-14T00:16:31Zbosh-ecosystem-vmware (BOSH Ecosystem Team @ VMware)
<p>Hello! We have seen a large increase in memory usage as measured by resident set size (RSS) in our Ruby 3.1.2 programs when testing on Ubuntu Jammy (22.04). After further testing, we have narrowed down the problem as starting with Ubuntu Eoan (19.10). We first noticed this increase coming from a gem called <code>eventmachine v1.2.7</code>, which is a concurrency library that uses threads internally. That led us to try threads directly, and we came up with a simple Ruby program that reproduces the memory usage issue by spawning multiple threads through <code>Thread.new</code>.</p>
<p>We are unsure whether the problem extends beyond just <code>Thread.new</code>; it was the easiest problem to spot within our applications.</p>
<p>We are building Ruby from source using <code>ruby-install</code>, though have seen the problem in versions installed via <code>apt</code>.</p>
<p>We are seeking to understand the underlying cause of this memory bloat, and what we can do to prevent it.</p>
<p>Generally, we've been seeing similar total virtual memory allocated, only a difference in RSS usage.</p>
<p>On Ubuntu Disco (19.04) and prior, we see a relatively small increase in RSS upon spawning the threads, on the order of 20 kilobytes per thread. On Ubuntu Eoan and later, we see that RSS increases by 1 megabyte per thread.</p>
<p>When Ruby is compiled using <code>clang</code> instead of <code>gcc</code> on Ubuntu Eoan and later, the memory usage is comparable to older versions of Ubuntu.</p>
<p>We do not believe it is related to gcc version <em>by itself</em>, as we tried compiling Ruby using gcc 9 on Ubuntu Disco, and did not encounter the memory problem. On the theory that it may be a memory fragmentation issue, we tried both using the <code>MALLOC_ARENA_MAX</code> runtime environment variable, and compiling Ruby against <code>jemalloc</code>. Neither of these reduced RSS usage.</p>
<p>Here are the memory differences when simply calling <code>Thread.new</code> 10 times:</p>
<table>
<thead>
<tr>
<th>Configuration</th>
<th>RSS before spawning threads</th>
<th>RSS after spawning threads</th>
<th>Diff</th>
</tr>
</thead>
<tbody>
<tr>
<td>Disco + gcc 8</td>
<td>13596 kB</td>
<td>13800 kB</td>
<td>204 kB</td>
</tr>
<tr>
<td>Disco + gcc 9</td>
<td>13880 kB</td>
<td>14088 kB</td>
<td>208 kB</td>
</tr>
<tr>
<td>Eoan + gcc 9</td>
<td>22268 kB</td>
<td>32672 kB</td>
<td>10404 kB</td>
</tr>
<tr>
<td>Eoan + clang</td>
<td>13976 kB</td>
<td>14180 kB</td>
<td>204 kB</td>
</tr>
<tr>
<td>Jammy + gcc 11</td>
<td>22208 kB</td>
<td>32608 kB</td>
<td>10400 kB</td>
</tr>
<tr>
<td>Jammy + clang</td>
<td>13668 kB</td>
<td>13872 kB</td>
<td>204 kB</td>
</tr>
<tr>
<td>Jammy + jemalloc</td>
<td>24212 kB</td>
<td>35216 kB</td>
<td>11004 kB</td>
</tr>
<tr>
<td>Jammy + apt</td>
<td>21308 kB</td>
<td>31712 kB</td>
<td>10404 kB</td>
</tr>
</tbody>
</table>
<p>We have attached the program and a series of Dockerfiles that replicate the test environments. While the reproduction is using Docker files, we have seen this same behavior on actual VMs. To run a particular Docker config, use <code>./run.sh <DIRECTORY></code>, e.g. <code>./run.sh jammy-clang</code>.</p>
<p>Does anyone understand why there is an increase in resident set size (RSS) when Ruby is compiled with gcc vs clang?<br>
Is there additional configuration (flags, etc) that could be sent to gcc to get equivalent memory usage on Jammy that is seen when using Disco or clang?</p> Ruby master - Misc #18984 (Open): Doc for Range#size for Float/Rational does not make sensehttps://bugs.ruby-lang.org/issues/189842022-08-29T14:39:13Zmasasakano (Masa Sakano)
<p>When <code>Range</code> consists of any Numeric, according to <a href="https://ruby-doc.org/core-3.1.2/Range.html#method-i-size" class="external">Official docs for Ruby-3.1.2</a>, <code>Range#size</code> should,</p>
<blockquote>
<p>Returns the count of elements in self if both begin and end values are numeric;</p>
</blockquote>
<p>Indeed, when <code>Range</code> consists of only <code>Integer</code>, this makes sense.<br>
However, when the begin value of <code>Range</code> is a <code>Float</code> or <code>Rational</code>, "<em>the count of elements</em>" does not make sense. Such Ranges are not iteratable, suggesting there are no such things as "elements":</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mf">0.51</span><span class="o">..</span><span class="mi">5</span><span class="p">.</span><span class="nf">quo</span><span class="p">(</span><span class="mi">2</span><span class="p">)).</span><span class="nf">each</span><span class="p">{}</span> <span class="c1"># => TypeError</span>
</code></pre>
<p>Yet, <code>Range#size</code> of such Ranges returns an Integer</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mf">0.51</span><span class="o">..</span><span class="mi">5</span><span class="p">.</span><span class="nf">quo</span><span class="p">(</span><span class="mi">2</span><span class="p">)).</span><span class="nf">size</span> <span class="c1"># => 2</span>
</code></pre>
<p>It seems both begin and end values of a Range are rounded (<code>Numeric#round</code>) to the nearest Integer before <code>Range#size</code> is calculated.</p>
<p>If this is the specification, I suggest it should be clearly stated in the <a href="https://ruby-doc.org/core-3.1.2/Range.html#method-i-size" class="external">Official docs</a>, avoiding the confusing expression "the count of elements", because "elements" are not unambiguously defined for Range of Float/Rational.</p> Ruby master - Misc #18924 (Open): lstrlen and lstrcat still in usehttps://bugs.ruby-lang.org/issues/189242022-07-17T20:42:52Ztest35965@gmail.com (Alexander Riccio)
<p>In practice, as-used, this is probably benign, but is a Terrible Idea.</p>
<p>There are several places where someone's still using lstrcat and lstrlen in ruby.</p>
<p>lstrcat catches and <em><strong>suppresses</strong></em> access violations. I have been burned by this in the wild as a user of other software, corrupting data and causing mysterious crashes. There's no good reason to use these functions. IIRC, they may even date back to before the standard library was standard (on windows?)?</p>
<p>Swapping them for standard functions like strcpy is not great, but can only break code that was incorrect and memory-corrupting already.</p> Ruby master - Misc #18921 (Open): Remove workaround for some fixed bug (llvm.4898 & 38095)?https://bugs.ruby-lang.org/issues/189212022-07-16T15:52:40ZChandler (Chandler Hen)
<p>I notice a workaround for llvm.4898 in constant_p.h:<br>
<a href="https://github.com/ruby/ruby/blob/master/include/ruby/internal/constant_p.h" class="external">https://github.com/ruby/ruby/blob/master/include/ruby/internal/constant_p.h</a></p>
<pre><code class="c syntaxhl" data-language="c"><span class="o">*</span> <span class="n">Note</span> <span class="n">that</span> <span class="n">__builtin_constant_p</span> <span class="n">can</span> <span class="n">be</span> <span class="n">applicable</span> <span class="n">inside</span> <span class="n">of</span> <span class="kr">inline</span> <span class="n">functions</span><span class="p">,</span>
<span class="o">*</span> <span class="n">according</span> <span class="n">to</span> <span class="n">GCC</span> <span class="n">manual</span><span class="p">.</span> <span class="n">Clang</span> <span class="n">lacks</span> <span class="n">that</span> <span class="n">feature</span><span class="p">,</span> <span class="n">though</span><span class="p">.</span>
<span class="o">*</span>
<span class="o">*</span> <span class="err">@</span><span class="n">see</span> <span class="n">https</span><span class="o">:</span><span class="c1">//bugs.llvm.org/show_bug.cgi?id=4898</span>
<span class="o">*</span> <span class="err">@</span><span class="n">see</span> <span class="n">https</span><span class="o">:</span><span class="c1">//gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html</span>
<span class="err">*/</span>
<span class="cp">#include</span> <span class="cpf">"ruby/internal/has/builtin.h"</span><span class="cp">
</span>
<span class="cm">/** Wraps (or simulates) `__builtin_constant_p` */</span>
<span class="cp">#if RBIMPL_HAS_BUILTIN(__builtin_constant_p)
# define RBIMPL_CONSTANT_P(expr) __builtin_constant_p(expr)
#else
# define RBIMPL_CONSTANT_P(expr) 0
#endif
</span>
<span class="cp">#endif </span><span class="cm">/* RBIMPL_CONSTANT_P_H */</span><span class="cp">
</span></code></pre>
<p>and a workaround for llvm.38095 in scan_args.h:<br>
<a href="https://github.com/ruby/ruby/blob/master/include/ruby/internal/scan_args.h" class="external">https://github.com/ruby/ruby/blob/master/include/ruby/internal/scan_args.h</a></p>
<pre><code class="c syntaxhl" data-language="c"><span class="cm">/* NOTE: Use `char *fmt` instead of `const char *fmt` because of clang's bug*/</span>
<span class="cm">/* https://bugs.llvm.org/show_bug.cgi?id=38095 */</span>
<span class="cp"># define rb_scan_args0(argc, argv, fmt, varc, vars) \
rb_scan_args_set(RB_SCAN_ARGS_PASS_CALLED_KEYWORDS, argc, argv, \
rb_scan_args_n_lead(fmt), \
rb_scan_args_n_opt(fmt), \
rb_scan_args_n_trail(fmt), \
rb_scan_args_f_var(fmt), \
rb_scan_args_f_hash(fmt), \
rb_scan_args_f_block(fmt), \
(rb_scan_args_verify(fmt, varc), vars), (char *)fmt, varc)
</span><span class="p">...</span>
</code></pre>
<p>These bugs are already marked as fixed:<br>
bugs.llvm.org/show_bug.cgi?id=4898<br>
bugs.llvm.org/show_bug.cgi?id=38095<br>
Shall they be removed?</p> Ruby master - Misc #18840 (Open): Top-level #using and other methods docshttps://bugs.ruby-lang.org/issues/188402022-06-18T19:12:56Zzverok (Victor Shepelev)zverok.offline@gmail.com
<p>I was looking into some docs problems, and the question I have is that we don't have any place where <code>main</code>'s methods documentation is rendered?</p>
<p>The <code>#using</code>, for example, is <a href="https://github.com/ruby/ruby/blob/ruby_3_1/eval.c#L1960" class="external">defined</a> on <code>main</code>'s singleton class (if I am reading the code correctly), and it has <a href="https://github.com/ruby/ruby/blob/ruby_3_1/eval.c#L1687" class="external">RDoc defined</a> in <code>*.c</code>, but for all I can tell it is rendered nowhere.</p>
<p>Theoretically, it would've been nice to have a place where <code>main</code> object's concept would be explained and the methods available in it, listed, right?..</p>
<p>Or am I missing something obvious?</p> Ruby master - Misc #18761 (Open): provide an example wasm projecthttps://bugs.ruby-lang.org/issues/187612022-04-30T19:25:46Zgrosser (Michael Grosser)michael@grosser.it
<p>Neither the release notes nor the wasm/README.md include any runnable examples.<br>
Please provide a docker image that can be used to generate a wasm binary (even if it's just "hello world"),<br>
to show how this is supposed to work.</p> Ruby master - Misc #18725 (Open): IO#write and IO#wait_writable block for write pipe if read pipe...https://bugs.ruby-lang.org/issues/187252022-04-13T23:20:52Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<p>I'm not sure whether this is a Ruby issue, an OpenBSD issue, or something else, but <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/3344">@ioquatix (Samuel Williams)</a> asked me to post this here. The following program hangs on OpenBSD:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'io/wait'</span>
<span class="n">rd</span><span class="p">,</span> <span class="n">wr</span> <span class="o">=</span> <span class="no">IO</span><span class="p">.</span><span class="nf">pipe</span>
<span class="n">thread_pass</span> <span class="o">=</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'pass'</span>
<span class="n">write</span> <span class="o">=</span> <span class="no">ARGV</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'write'</span>
<span class="n">thread</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">longer</span> <span class="o">=</span> <span class="s2">"0"</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span>
<span class="p">(</span><span class="mi">1024</span> <span class="o">*</span> <span class="mi">4</span><span class="p">).</span><span class="nf">times</span> <span class="k">do</span>
<span class="k">if</span> <span class="n">write</span>
<span class="n">wr</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="n">longer</span><span class="p">)</span>
<span class="k">else</span>
<span class="k">while</span> <span class="n">wr</span><span class="p">.</span><span class="nf">write_nonblock</span><span class="p">(</span><span class="n">longer</span><span class="p">,</span> <span class="ss">:exception</span><span class="o">=></span><span class="kp">false</span><span class="p">)</span> <span class="o">==</span> <span class="ss">:wait_writable</span>
<span class="n">thread_pass</span> <span class="p">?</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">pass</span> <span class="p">:</span> <span class="n">wr</span><span class="p">.</span><span class="nf">wait_writable</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="ss">:finished</span>
<span class="k">rescue</span> <span class="o">=></span> <span class="n">e</span>
<span class="n">e</span>
<span class="k">end</span>
<span class="nb">sleep</span> <span class="mi">1</span>
<span class="n">rd</span><span class="p">.</span><span class="nf">close</span>
<span class="nb">puts</span> <span class="ss">:rd_close</span>
<span class="nb">puts</span> <span class="n">thread</span><span class="p">.</span><span class="nf">value</span>
</code></pre>
<p>This program will also hang if <code>write</code> is given as the argument, using <code>wr.write</code> instead of <code>wr.write_nonblock</code>/<code>wr.wait_writable</code>. However, if <code>pass</code> is given as the argument, using <code>Thread.pass</code> instead of <code>wr.wait_writable</code>, the program will not hang.</p>
<p>From testing, the hang when called without an argument is in <code>wait_writable</code>, and the hang with a <code>write</code> argument is in <code>write</code>.</p>
<p>Is Ruby supposed to guarantee that a closing the read end of a pipe in one thread causes a raise of EPIPE to the write end of the pipe in a different thread if already inside <code>IO#write</code>/<code>IO#wait_writable</code>? Or is this platform-specific behavior?</p>
<p>This example was extracted from one of Rack's tests, which was causing non-deterministic hangs on OpenBSD.</p> Ruby master - Misc #18587 (Open): What was the reason behind Ruby choosing SipHash for Hash?https://bugs.ruby-lang.org/issues/185872022-02-16T15:42:48Zmidnight (Sarun R)
<p>Hello</p>
<p>I am digging into the history behind Ruby using SipHash for its Hash.<br>
I found that in 2012 there were CVE-2012-5371 showing up;<br>
the Ruby maintainers went with the decision to switch algorithms, probably, because we wanted something quick to implement at the time.<br>
The change went live in late 2012.</p>
<p>Fast forward with the Ruby 3x3 initiative, we now seem to care about the performance again.<br>
And hash DoSing does not seem to be an urgent threat now; we have time to be deliberate about Hash again.</p>
<p>I can't find the old discussion related to Ruby's SipHash decision.<br>
I just found that SipHash is not the only solution to prevent hashtable DoSing.<br>
There is an interesting discussion on golang side in late 2015:<br>
<a href="https://github.com/golang/go/issues/9365" class="external">https://github.com/golang/go/issues/9365</a></p>
<p>Just to recap, Go's authors argue that:</p>
<ul>
<li>Cryptographic hash is not needed to construct a DoS-resistant hashtable.</li>
<li>If the random seed is per-hashtable bases, the attack vector exploitable from a remote adversary seems unlikely.</li>
<li>If we want to be extra careful about it, and since the collision is unlikely, when collision actually does occur despite the per-hashtable seed, we can handle that as a special case where we can rerandom the seed and rehash the key.</li>
<li>The way random seeds are folded into the hash does matter, for example, CityHash does f(g(msg), seed); in such case, collision in g will cause a collision in f because the output of g is independent of the seed.</li>
<li>Slowing down hashtable for everyone to prevent hard-to-exploit DoS doesn't seem to be a good trade-off.</li>
</ul>
<p>On the actual implementation, they use AES-NI to achieve good pseudo-random functions' properties. And use some fallback non-cryptographic hashing function on the platform without AES-NI.</p>
<p>Now, I read the rationale on golang side, I want to understand the rationale on the Ruby side too.<br>
I am not there 10-years-ago, and I can't find records or discussions at the time. There might be some Ruby limitations that the approach described by go's authors does not apply.</p>
<p>So, I asked in the hope of someone still remembering what was happening, the situation we are in 10 years ago, or the limitation of Ruby that prevents per-Hash seeds.</p> Ruby master - Misc #18420 (Open): Question about how to handle IO.pipe reader and writer when for...https://bugs.ruby-lang.org/issues/184202021-12-22T00:04:00Zestolfo (Emily S)
<p>I don't think this is a bug but I'm not sure. It could be just a limitation of forking with a IO.pipe reader and writer.</p>
<p>I work on the Elastic Ruby APM agent (<a href="https://github.com/elastic/apm-agent-ruby" class="external">https://github.com/elastic/apm-agent-ruby</a>). The Ruby agent is a singleton and uses a IO.pipe to send data to the APM server. The writer we wrap in a GzipWriter. When the parent process is forked, the child inherits the readers and (gzip)writers. I detect the forking by comparing PIDs when an event is created and create new readers and writers in the child process so that I don't interfere with the parent's streams.<br>
When the fork exits, I close the child's readers and writers but I see the follow error</p>
<p>zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.<br>
zlib(finalizer): the stream was freed prematurely.</p>
<p>because the parent's writer is still considered living in the child process.<br>
One option to address this error is to create a finalizer on the writer that closes it. But then I get intermittent errors from the APM server saying the gzipped data's header is invalid. I'm assuming this is because the parent's writer is closed by the child before the compression is complete in the parent.</p>
<p>Is there a proper way to handle forking when the parent is using a pipe? Both options I have to handle the writer result in an error/warning: I can't close the GzipWriter in the child because of the corrupted data issue but if I don't close it, I get the "Zlib::GzipWriter object must be closed explicitly" warning.</p>
<p>If I close the reader and writer in a "before_fork" hook run in the parent, I avoid both issues. But I don't want to recreate the reader/writer objects every time the parent is forked.</p>
<p>I've been testing with ruby 2.7.0 and Resque (<a href="https://github.com/resque/resque" class="external">https://github.com/resque/resque</a>) primarily but this issue is observed with Puma and other frameworks that use forking.</p> Ruby master - Misc #18371 (Assigned): Release branches (release information in general)https://bugs.ruby-lang.org/issues/183712021-11-30T22:52:11Ztenderlovemaking (Aaron Patterson)tenderlove@ruby-lang.org
<p>Hi,</p>
<p>I was trying to learn about Ruby's release process. I noticed that we don't create a release branch until the final version is shipped. Is there a reason we don't create the release branch when the first preview is shipped? The reason I'm asking is because I'm worried about merging things to master after the first preview. Do we have any documentation on the release process? (I was searching and couldn't find much info, but maybe I didn't search correctly)</p>
<p>Thanks!</p> Ruby master - Misc #18352 (Open): What is the Hash#grep expected?https://bugs.ruby-lang.org/issues/183522021-11-20T09:50:09Zzw963 (Wei Zheng)
<p>Current ruby implement, When use Array#grep, the method name means is expected.</p>
<pre><code>[19] pry(#<App>)> [:foo1, :foo2, :bar].grep /foo/
[
:foo1,
:foo2
]
</code></pre>
<p>But when use with hash, the result is really confusing ...</p>
<pre><code class="rb syntaxhl" data-language="rb"><span class="p">[</span><span class="mi">12</span><span class="p">]</span> <span class="n">pry</span><span class="p">(</span><span class="c1">#<App>)> {foo: '100', bar: '200'}.grep /foo/ </span>
<span class="p">[]</span>
</code></pre>
<p>This result almost make Include Enumerable#grep into Hash is totally meaningless, right?</p>
<p>so, i consider if we should introduce a <code>Hash#grep</code> method instead.</p>
<p>Following is what is expected. (=== is matching on hash key, as Hash#slice)</p>
<pre><code class="rb syntaxhl" data-language="rb"><span class="p">[</span><span class="mi">20</span><span class="p">]</span> <span class="n">pry</span><span class="p">(</span><span class="c1">#<App>)> {foo1: '100', foo2: '200', bar: '200'}.grep /foo/ </span>
<span class="p">{</span>
<span class="ss">:foo1</span> <span class="o">=></span> <span class="s2">"100"</span><span class="p">,</span>
<span class="ss">:foo2</span> <span class="o">=></span> <span class="s2">"200"</span>
<span class="p">}</span>
</code></pre> Ruby master - Misc #18248 (Open): Add Feature Triaging Guidehttps://bugs.ruby-lang.org/issues/182482021-10-11T16:14:14Zjeremyevans0 (Jeremy Evans)merch-redmine@jeremyevans.net
<p>Ruby added a bug triaging guide in June 2019, and since then I've used it to reduce the number of open bugs in the tracker from over 1400 to about 300. Ruby currently has over 1200 open feature requests on the issue tracker. From a cursory review, many of these have already been implemented, and many are unlikely to be desired. I would like to add a feature triaging guide to Ruby and start triaging feature requests, with the goal of having open feature requests be features not yet implemented that the Ruby core team would like implemented or would consider patches for. By doing so, we can make it easier for potential contributors to Ruby to easily see possible contributions they could make.</p>
<p>I've added a pull request with a draft of the proposed guide: <a href="https://github.com/ruby/ruby/pull/4953" class="external">https://github.com/ruby/ruby/pull/4953</a></p> Ruby master - Misc #18150 (Open): Proposal: Deprecate leading zero syntax to declare octals, sinc...https://bugs.ruby-lang.org/issues/181502021-09-03T17:10:05ZProGM (Piero Dotti)
<p>Hi there,<br>
I'd like to open a discussion about the leading zeros syntax to declare octal numbers.</p>
<p>Let me give you a little bit of context.</p>
<p>It seems like ruby considers all integers with leading zeros as octal.<br>
For instance, if you write 012, ruby reads it as 10 in decimal.<br>
There is an alternative syntax for this, using an "o" character after the zero, i.e. 0o12</p>
<p>I've discovered this behavior by chance a couple of days ago.<br>
I was declaring a new Date object:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">START_DATE</span> <span class="o">=</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">09</span><span class="p">,</span> <span class="mo">01</span><span class="p">)</span>
</code></pre>
<p>In my mind, I was thinking of ISO 8601 ("2021-09-01") and I wrote it in that way without even thinking about it.</p>
<p>I immediately got a weird error:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">SyntaxError</span> <span class="p">((</span><span class="n">irb</span><span class="p">):</span><span class="mi">2</span><span class="p">:</span> <span class="no">Invalid</span> <span class="n">octal</span> <span class="n">digit</span><span class="p">)</span>
</code></pre>
<p>That was astonishing. "What the heck does this error mean?", I thought.</p>
<p>After a brief research, I've discovered that many languages that come from C have this "weird" behavior.<br>
Javascript, Go, Java, and ruby itself all treat leading zeros like this.<br>
Rust and Elixir, instead, are parsing 00123 like 123, as math notation suggests.<br>
Finally, Python 3+ raises an error.</p>
<hr>
<p>My proposal is: throw a deprecation warning in ruby 3.x when using this syntax and treat 0011 as 11 from ruby 4+.</p>
<p>Why?</p>
<ul>
<li>Because it's extremely confusing</li>
<li>There is an alternative syntax that is much more explicit (0o11 vs 011)</li>
<li>It leads to weird behaviors (like 011 != 11)</li>
<li>Because Python 2.x was treating leading zeros numbers as octals like ruby does, but this has been changed in python 3+: <a href="https://medium.com/techmacademy/leading-zeros-in-python-a-hidden-mystery-revealed-ee3e3065634d" class="external">https://medium.com/techmacademy/leading-zeros-in-python-a-hidden-mystery-revealed-ee3e3065634d</a>
</li>
</ul>
<p>Ruby's claim is "A PROGRAMMER'S BEST FRIEND". As a programmer, I don't consider this behavior very friendly. :(</p>
<p>Let me know what you think about this!</p> Ruby master - Misc #18082 (Open): FileUtils.remove_entry_secure has inconsistent documenthttps://bugs.ruby-lang.org/issues/180822021-08-17T16:26:38Zpocke (Masataka Kuwabara)
<p><code>FileUtils.remove_entry_secure</code> and <code>FileUtils.rm_r</code> have inconsistency about the vulnerability condition the in the documentations.</p>
<p><code>remove_entry_secure</code> document:<br>
<a href="https://github.com/ruby/ruby/blob/6a9bfa4d9387b9d8f07f43f4546437be57f9e27c/lib/fileutils.rb#L660-L664" class="external">https://github.com/ruby/ruby/blob/6a9bfa4d9387b9d8f07f43f4546437be57f9e27c/lib/fileutils.rb#L660-L664</a></p>
<blockquote>
<p>#rm_r causes security hole when:</p>
<ul>
<li>Parent directory is world writable (including /tmp).</li>
<li>Removing directory tree includes world writable directory.</li>
<li>The system has symbolic link.</li>
</ul>
</blockquote>
<p><code>rm_r</code> document:<br>
<a href="https://github.com/ruby/ruby/blob/6a9bfa4d9387b9d8f07f43f4546437be57f9e27c/lib/fileutils.rb#L614-L618" class="external">https://github.com/ruby/ruby/blob/6a9bfa4d9387b9d8f07f43f4546437be57f9e27c/lib/fileutils.rb#L614-L618</a></p>
<blockquote>
<p>WARNING: This method causes local vulnerability<br>
if one of parent directories or removing directory tree are world<br>
writable (including /tmp, whose permission is 1777), and the current<br>
process has strong privilege such as Unix super user (root), and the<br>
system has symbolic link.</p>
</blockquote>
<p>The differences are following.</p>
<ul>
<li>
<code>rm_r</code> describes about strong privilege, but <code>remove_entry_secure</code> doesn't.</li>
<li>
<code>rm_r</code> describes "one of parent directories <strong>OR</strong> removing directory tree are world writable", but the conditions are just listed in <code>remove_entry_secure</code> doc, it seems <strong>AND</strong> condition.</li>
</ul>
<p>I couldn't understand the prerequisites of the vulnerability from the documents.<br>
I think both documents should describe the same prerequisites.</p>
<p>I don't know what is the right prerequisites, so I didn't make a patch.</p> Ruby master - Misc #17829 (Open): Clang/LLVM correctness of x64-mingw32 build (`shorten-64-to-32`...https://bugs.ruby-lang.org/issues/178292021-04-26T16:23:26Zxtkoba (Tee KOBAYASHI)
<p>The attached log is from <code>make miniruby</code> for x64-mingw32 with Clang/LLVM. Warnings are silenced except for <code>-Wshorten-64-to-32</code>.</p>
<p>This might not be an issue solely for Clang/LLVM, because the size of each type is the same as that in GCC.</p>
<p>I believe most of them are false positive. It would be easy to silence these warnings by explicit type casting. But before doing so, it must be checked whether each shortening is legitimate or not.</p>
<p>Version: <code>ruby 3.1.0dev (2021-04-26T13:46:51Z master 203eeeefdd) [x64-mingw32]</code></p> Ruby master - Misc #17720 (Assigned): Cirrus CI to check non-x86_64 architecture cases by own mac...https://bugs.ruby-lang.org/issues/177202021-03-12T17:50:05Zjaruga (Jun Aruga)
<p>Hello!</p>
<p>This ticket is related to the tickets <a class="issue tracker-5 status-5 priority-4 priority-default closed" title="Misc: Enabling ARM 64/32-bit cases by Drone CI (Closed)" href="https://bugs.ruby-lang.org/issues/16234">#16234</a> <a class="issue tracker-5 status-5 priority-4 priority-default closed" title="Misc: Enabling IBM PowerPC/Z cases in Travis CI (Closed)" href="https://bugs.ruby-lang.org/issues/16360">#16360</a>. But I opened a new ticket because it is related to general non-x86_64 architecture CI cases.</p>
<p>I have a suggestion.</p>
<p>I see the <code>.travis.yml</code> was removed [1], and I also saw another open source project remove their <code>.travis.yml</code> because they could not get the credits to continue to run Travis [2]. I feel Travis is not really a possible option for every open source project for now.</p>
<p>While we have RubyCI, I think we still have a motivation to run a CI on non-x86_64 architectures at a pull-request timing. So, I investigated alternative CI. When checking GitHub Actions, I do not feel it will happen soon on GitHub Actions [3]. Then I found an interesting CI called "Cirrus CI", that might enable us to run CI on non-x86_64 architectures such as Mac M1 (arm) ppc64le and s390x beyond the cloud.</p>
<p>Cirrus CI has 2 types of features: "cloud" and "persistent workers". I see the Cirrus CI "cloud" feature has been used in the QEMU and podman projects [4][5]. It has a unique freeBSD host. However the remarkable feature for the Ruby project is the "persistent workers" [6] announced a few months ago, that is beyond the cloud. Because this feature enables us to use our own machines as a CI running host. You can see the examples running the CI with the machines such as Mac M1, iPhone, ppc64le and s390x on the page [6]. Maybe the used machine does not even have the global static IP. You can see other articles [7][8] too.</p>
<p>I can see some benefits to start Cirrus CI for the Ruby project.</p>
<ul>
<li>Possibly we can check Mac M1 (arm), ppc64le, s390x cases using machines used in RubyCI [9] and someone's machine such as @ReiOdaira's ppc64le/s390x machines at the pull-request timing.</li>
<li>When we face the CI issue, we can login to the machine and use the interactive debugging tool such as gdb to fix it.</li>
<li>The config file is YAML format and it has the matrix feature [10]. We are familiar with the YAML and matrix.</li>
</ul>
<p>What do you think? Positive or negative?<br>
Thank you.</p>
<p>[1] ruby removed .travis.yml: <a href="https://github.com/ruby/ruby/commit/6b978d542704a5614af5e9375c4b31b8d2618652" class="external">https://github.com/ruby/ruby/commit/6b978d542704a5614af5e9375c4b31b8d2618652</a><br>
[2] simde removed .travis.yml: <a href="https://github.com/simd-everywhere/simde/commit/17a27e7f2c3114225899f2ace14010cbbb2139b5" class="external">https://github.com/simd-everywhere/simde/commit/17a27e7f2c3114225899f2ace14010cbbb2139b5</a><br>
[3] GitHub Actions and ppc64le: <a href="https://github.community/t/self-hosted-runner-on-ppc64el-architecture/155337" class="external">https://github.community/t/self-hosted-runner-on-ppc64el-architecture/155337</a><br>
[4] QEMU: <a href="https://gitlab.com/qemu-project/qemu/-/blob/master/.cirrus.yml" class="external">https://gitlab.com/qemu-project/qemu/-/blob/master/.cirrus.yml</a><br>
[5] Podman: <a href="https://github.com/containers/podman/blob/master/.cirrus.yml" class="external">https://github.com/containers/podman/blob/master/.cirrus.yml</a><br>
[6] The issue ticket of Persistent Workers: <a href="https://github.com/cirruslabs/cirrus-ci-docs/issues/263#issuecomment-746900845" class="external">https://github.com/cirruslabs/cirrus-ci-docs/issues/263#issuecomment-746900845</a><br>
[7] Persistent Workers blog: <a href="https://medium.com/cirruslabs/announcing-public-beta-of-cirrus-ci-persistent-workers-7327a38004be" class="external">https://medium.com/cirruslabs/announcing-public-beta-of-cirrus-ci-persistent-workers-7327a38004be</a><br>
[8] Persistent Workers guide: <a href="https://cirrus-ci.org/guide/persistent-workers/" class="external">https://cirrus-ci.org/guide/persistent-workers/</a><br>
[9] RubyCI: <a href="https://rubyci.org/" class="external">https://rubyci.org/</a><br>
[10] Cirrus CI matrix feature: <a href="https://cirrus-ci.org/guide/writing-tasks/#matrix-modification" class="external">https://cirrus-ci.org/guide/writing-tasks/#matrix-modification</a></p> Ruby master - Misc #17683 (Open): Current status of beginless range (experimental or not)https://bugs.ruby-lang.org/issues/176832021-03-10T09:40:02Zjnchito (Junichi Ito)
<p>A beginless range was experimentally introduced in Ruby 2.7.0:<br>
<a href="https://github.com/ruby/ruby/blob/v2_7_0/NEWS#label-Other+miscellaneous+changes" class="external">https://github.com/ruby/ruby/blob/v2_7_0/NEWS#label-Other+miscellaneous+changes</a></p>
<p>Is it still experimental feature or not?</p> Ruby master - Misc #17662 (Assigned): The heredoc pattern used in tests does not syntax highlight...https://bugs.ruby-lang.org/issues/176622021-02-27T16:22:19ZEregon (Benoit Daloze)
<p>This heredoc pattern</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="n">assert_ruby_status</span><span class="p">([],</span> <span class="s2">"</span><span class="si">#{</span><span class="o"><<-</span><span class="s2">"begin;"</span><span class="si">}</span><span class="se">\n</span><span class="si">#{</span><span class="o"><<-</span><span class="s1">'end;'</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="n">bug</span><span class="p">)</span>
<span class="k">begin</span><span class="p">;</span>
<span class="nb">exit</span><span class="p">(</span><span class="s2">"1"</span> <span class="o">==</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">start</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="o">&</span><span class="ss">:to_s</span><span class="p">).</span><span class="nf">value</span><span class="p">)</span>
<span class="k">end</span><span class="p">;</span>
</code></pre>
<p>completely breaks syntax highlighting in at least:</p>
<ul>
<li>GitHub: <a href="https://github.com/ruby/ruby/blob/36dde35e029c7a6607e6c674062ce6fc7a51c0bd/test/ruby/test_string.rb#L697" class="external">there</a> <a href="https://github.com/ruby/ruby/blob/36dde35e029c7a6607e6c674062ce6fc7a51c0bd/test/ruby/test_process.rb#L1545" class="external">are</a> <a href="https://github.com/ruby/ruby/blob/565aeb81e0886c835888a425e5d05ed99fb03238/test/ruby/test_thread.rb#L201" class="external">many</a> <a href="https://github.com/ruby/ruby/blob/36dde35e029c7a6607e6c674062ce6fc7a51c0bd/test/ruby/test_require.rb#L21" class="external">examples</a>
</li>
<li>Atom</li>
<li>RubyMine (and IntelliJ)</li>
<li>Likely many more editors based on TextMate grammars</li>
</ul>
<p>Could another pattern be used in tests inside the ruby/ruby repository (at least for <code>test/ruby</code>)?</p>
<p>Due to this issue, it is very annoying and inconvenient to look at/read/investigate many tests.</p>
<p>I think this pattern is also very complicated to understand (and using <code>;</code> is quite weird for this).<br>
I suggest to replace it with this obvious and simple pattern many people use:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="n">assert_ruby_status</span><span class="p">([],</span> <span class="o"><<~</span><span class="no">'RUBY'</span><span class="p">,</span> <span class="n">bug</span><span class="p">)</span><span class="sh">
exit("1" == Thread.start(1, &:to_s).value)
</span><span class="no"> RUBY</span>
</code></pre>
<p>This syntax highlights correctly in most (all?) editors, and as an added bonus the code inside the heredoc is also highlighted in some editors (due to the label being <code>RUBY</code>).</p> Ruby master - Misc #17637 (Open): Endless ranges with `nil` boundary weird behaviorhttps://bugs.ruby-lang.org/issues/176372021-02-17T10:49:17Zgud (gud gud)
<p>Basically it's about this <a href="https://andycroll.com/ruby/watch-out-for-nils-in-ranges/" class="external">https://andycroll.com/ruby/watch-out-for-nils-in-ranges/</a></p>
<p>Since Ruby 2.6 we have this weird syntax (0..nil) which is really really bug prone</p>
<p>e.g. we have dynamic upper boundary like</p>
<pre><code>lower = 0
upper = some_method(arg1, arg2)
(lower..upper).each do { |s| some_method2(s) }
</code></pre>
<p>We rarely do <code>nil</code> checks in Ruby so it's really easy to have Infinity loop in the end.<br>
Previous Argument error was more intuitive since it throws exception instead of silently looping forever.</p>
<ul>
<li>some additional strange behavior:</li>
</ul>
<pre><code>(0..nil).count
=> Infinity
(0..Float::INFINITY).count
=> hangs, I guess same infinity loop
</code></pre>
<p>Having explicit parameter <code>Float::INFINITY</code> (as in previous versions) looks more like a proper design instead of allowing <code>nil</code> as a valid parameter.</p>
<p>You may think of it as <strong>I would like to have a range from 0 to nothing, what is it actually ?</strong><br>
And I guess the answer is <strong>Nothing</strong>.<br>
Fixing <code>(0..Float::INFINITY).count</code> this case it also important I believe.</p>
<p>Tested on <code>ruby 2.7.1p83</code></p> Ruby master - Misc #17586 (Open): Please run Windows CI in all std-lib reposhttps://bugs.ruby-lang.org/issues/175862021-01-27T17:40:41ZMSP-Greg (Greg L)
<p>Please consider adding Windows CI to all std-lib repos.</p>
<p>Having ruby/ruby CI fail due to std-lib commits merged from their respective repos causes downstream issues, wasted time, etc.</p>
<p>See <a href="https://github.com/ruby/irb/pull/179" class="external">https://github.com/ruby/irb/pull/179</a> for an example adding Windows to Actions CI.</p>
<p>Re Actions matrix:</p>
<ol>
<li>The os change from <code>*-latest</code> to <code>*-<numeric></code> (eg, <code>macos-latest</code> to <code>macos-11.0</code>). The <code>-latest</code> designation changes, so some may prefer a <code>-<numeric></code> tag. See <a href="https://github.com/actions/virtual-environments#available-environments" class="external">https://github.com/actions/virtual-environments#available-environments</a> for available environments and their naming.</li>
<li>Using numeric versions for Ruby causes an issue with 3.0, as it’s interpreted as 3, which is only a sematic major requirement. Hence, quote all numeric versions.</li>
</ol> Ruby master - Misc #17569 (Open): `uri` lib maintainershiphttps://bugs.ruby-lang.org/issues/175692021-01-22T15:48:33Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<p>I’d like to merge <a href="https://github.com/ruby/uri/pull/15" class="external">https://github.com/ruby/uri/pull/15</a> but it is an API change. I would release v1.0.0. <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/271">@akr (Akira Tanaka)</a> is the official maintainer of <code>uri</code>… Is he still interested in this role? Otherwise we could put “Ruby core team” in the listing…</p> Ruby master - Misc #17565 (Open): Prefer use of access(2) in rb_file_load_ok() to check for exist...https://bugs.ruby-lang.org/issues/175652021-01-20T20:05:56Zleehambley (Lee Hambley)
<p>When using Ruby in Docker (2.5 in our case, but the code is unchanged in 15 years across all versions) with a large $LOAD_PATH some millions of calls are made to <code>open(2)</code> with a mean cost of 130µsec per call, where a call to <code>access(2)</code> has a cost around 5× lower (something around 28µsec).</p>
<p>With a Rails 5 app, without Zeitwerk, the load path is searched iteratively looking for a file to define a constant, this causes something like 2,000,000 calls to <code>open(2)</code> of which 97.5% are failing with <code>ENOENT</code>.</p>
<p>I believe that the cost of two syscalls (<code>open(2)</code> only after successful <code>access(2)</code>) would, in our case, at least because we would shave-off something like 1,900,000×90µsec (2.85 minutes) from the three minute boot time for our application.</p>
<p>I prepared a very naïve patch with a simple early-return in <code>rb_file_load_ok</code>:</p>
<pre><code>diff --git a/file.c b/file.c
index 3bf092c05c..c7a7635125 100644
--- a/file.c
+++ b/file.c
@@ -5986,6 +5986,16 @@ rb_file_load_ok(const char *path)
O_NDELAY |
#endif
0);
+ if (access(path, R_OK) == -1) return 0;
int fd = rb_cloexec_open(path, mode, 0);
if (fd == -1) return 0;
rb_update_max_fd(fd);
</code></pre>
<p>This hasn't been exhaustively tested as I simply haven't had time yet, but at least it compiled and passed <code>make check</code>.</p>
<p>I spoke with Aaron Patterson on Twitter, who suggested maybe a wiser approach would be a heuristic approach one level higher (<code>rb_find_file</code>?) which switches the strategy based on the length of the LOAD_PATH.</p>
<p>Alternatively, maybe the patch could be conditional, guarded somehow, and conditionally compiled only into the Rubies built for Docker, in a way that is portable to the common Ruby version managers.</p>
<p>I am opening this ticket to track my own work, as much as anything, with no expectation that someone implement this on my behalf. I am eager to contribute to Ruby for all the benefit I have seen from it in my career.</p>
<p>If someone knows hints why this may be an unsuccessful adventure, I gratefully receive any and all feedback.</p> Ruby master - Misc #17502 (Open): C vs Rubyhttps://bugs.ruby-lang.org/issues/175022021-01-02T20:25:28Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<p>Some features are coded in a mix of Ruby and C (e.g. ractor.rb).</p>
<p>External gems don't have access to this. The C-API to deal with keyword parameters is also very verbose the parsing and the engine does not know it.</p>
<p>Moreover, some optimization PRs are simply rewriting C-code into Ruby using pseudo C code.</p>
<p>I understand the intentions are great, but changes like <a href="https://github.com/ruby/ruby/pull/4018/files" class="external">https://github.com/ruby/ruby/pull/4018/files</a> seem a symptom that something needs to be improved with the C api.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gd">-static VALUE
- flo_zero_p(VALUE num)
- {
- return flo_iszero(num) ? Qtrue : Qfalse;
- }
</span># in different file:
<span class="gi">+ def zero?
+ Primitive.attr! 'inline'
+ Primitive.cexpr! 'flo_iszero(self) ? Qtrue : Qfalse'
+ end
</span></code></pre>
<p>It seems to me that this is a way to circumvent a deeper issue. Is this the right direction?</p>
<p>Is there a plan for an API that would:</p>
<ol>
<li>be accessible to C extensions</li>
<li>can't be re-written any faster in pseuso-C in Ruby</li>
<li>has an easy way to define keyword parameters?</li>
</ol>
<p>I realize that RBS may give perfect signatures, but ideally <code>parameters</code> would be more informative for C-functions too.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Ractor</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:yield</span><span class="p">).</span><span class="nf">parameters</span>
<span class="c1"># => [[:req, :obj], [:key, :move]] # good!</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:initialize</span><span class="p">).</span><span class="nf">parameters</span>
<span class="c1"># => [[:rest]] # not good, should be [[:key, :blocking]]</span>
</code></pre> Ruby master - Misc #17376 (Assigned): Reduce number of GitHub Actionshttps://bugs.ruby-lang.org/issues/173762020-12-08T09:45:17Znaruse (Yui NARUSE)naruse@airemix.jp
<p>At this time we have 127 checks for GitHub commits, but unfortunately GitHub UI only shows 100 checks.<br>
It sometimes makes we don't see a failed check.<br>
Could you reduce number of GitHub actions at least less than 100?</p> Ruby master - Misc #17337 (Open): Don't embed Ruby build-time configuration in Rubyhttps://bugs.ruby-lang.org/issues/173372020-11-20T09:49:55Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<p>When Ruby 3.0 is built without C++ compiler available, subsequent builds of Eventmachine extensions (or any other gems that require C++ compiler) fail:</p>
<pre><code>... snip ...
"make \"DESTDIR=\""
I. -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I. -DHAVE_OPENSSL_SSL_H -DHAVE_OPENSSL_ERR_H -DWITH_SSL -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_CALL_WITHOUT_GVL -DHAVE_RB_THREAD_FD_SELECT -DHAVE_TYPE_RB_FDSET_T -DHAVE_RB_WAIT_FOR_SINGLE_FD -DHAVE_RB_TIME_NEW -DHAVE_INOTIFY_INIT -DHAVE_INOTIFY -DHAVE_WRITEV -DHAVE_PIPE2 -DHAVE_ACCEPT4 -DHAVE_CONST_SOCK_CLOEXEC -DOS_UNIX -DHAVE_EPOLL_CREATE -DHAVE_EPOLL -DHAVE_CLOCK_GETTIME -DHAVE_CONST_CLOCK_MONOTONIC_RAW -DHAVE_CONST_CLOCK_MONOTONIC -fPIC -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -m64 -o binder.o -c binder.cpp
make: I.: No such file or directory
make: [Makefile:237: binder.o] Error 127 (ignored)
</code></pre>
<p>This happens since <a href="https://github.com/ruby/ruby/commit/50b18e81295ad2d45975e4d8ea1e0c7e85140b97" class="external">1</a>.</p>
<p>I would like to question not just the commit, but the whole principle. Availability of C++ compiler during build of Ruby does not have anything to do with availability of C++ compiler during build of whatever Ruby extension.</p>
<p>Moreover, the machine where Ruby is built might be different from the machine where Ruby is used and the extension is installed. Typical examples are binary Linux distributions. RVM also uses precompiled Ruby binaries if I am not mistaken.</p>
<p>Therefore, I would appreciate if we reconsider embedding Ruby build-time configuration in Ruby and reusing it for Ruby extension build. This would solve the issue of Eventmachine I started with, and also issues such as <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Ruby configuration options should not be reused for gem builds (Closed)" href="https://bugs.ruby-lang.org/issues/14422">#14422</a></p>
<p>Just FTR, to avoid this kind of issues, as a Ruby maintainer on Red Hat platforms, I am considering to ship rbconfig.rb in vendorlib, which would dynamically fix these issues. E.g. set the <code>CONFIG['CXX']</code> based on C++ compiler availability during extension build. But I'd like to avoid it, at least as a downstream modification.</p> Ruby master - Misc #17309 (Open): URI.escape being deprecated, yet there is no replacementhttps://bugs.ruby-lang.org/issues/173092020-11-07T01:26:10Zchucke (Tiago Cardoso)
<p>I'm on ruby 2.7.2 . The moment I do</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">uri</span> <span class="o">=</span> <span class="s2">"http://bücher.ch"</span>
<span class="no">URI</span><span class="p">.</span><span class="nf">escape</span> <span class="n">uri</span>
<span class="p">(</span><span class="n">irb</span><span class="p">):</span><span class="mi">5</span><span class="p">:</span> <span class="ss">warning: </span><span class="no">URI</span><span class="p">.</span><span class="nf">escape</span>
<span class="s2">"http://b%C3%BCcher.ch"</span>
</code></pre>
<p>I get that warning. Rubocop also tells me:</p>
<p>"""<br>
URI.escape method is obsolete and should not be used. Instead, use CGI.escape, URI.encode_www_form or URI.encode_www_form_component depending on your specific use case.<br>
"""</p>
<p>However, none of the suggestions does the same as <code>URI.escape</code>.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">CGI</span><span class="p">.</span><span class="nf">escape</span> <span class="n">uri</span>
<span class="o">=></span> <span class="s2">"http%3A%2F%2Fb%C3%BCcher.ch"</span>
<span class="no">URI</span><span class="p">.</span><span class="nf">encode_www_form_component</span> <span class="n">uri</span>
<span class="o">=></span> <span class="s2">"http%3A%2F%2Fb%C3%BCcher.ch"</span>
<span class="no">URI</span><span class="p">.</span><span class="nf">encode_www_form</span> <span class="n">uri</span>
<span class="no">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
<span class="no">NoMethodError</span> <span class="p">(</span><span class="n">undefined</span> <span class="nb">method</span> <span class="sb">`map' for "http://bücher.ch":String)
Did you mean? tap
</span></code></pre>
<p>So my question is: why is this being deprecated? And if there's still reason, what to exactly replace it for, so I can keep the exact same behaviour?</p> Ruby master - Misc #17199 (Open): id outputed by inspect and to_s output does not allow to find a...https://bugs.ruby-lang.org/issues/171992020-09-28T11:40:39ZAnnih (Baptiste Courtois)
<p>Hello, here is my first ruby issue sorry in advance if it is incorrectly filled.</p>
<a name="Issue"></a>
<h1 >Issue<a href="#Issue" class="wiki-anchor">¶</a></h1>
<p>The value returned by <code>#object_id</code> is not aligned anymore with displayed info in <code>#inspect</code> and <code>#to_s</code> methods.</p>
<a name="with-ruby-lt-27"></a>
<h2 >with ruby < 2.7<a href="#with-ruby-lt-27" class="wiki-anchor">¶</a></h2>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Object</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">tap</span> <span class="p">{</span> <span class="o">|</span><span class="n">o</span><span class="o">|</span> <span class="nb">p</span> <span class="s2">"#to_s=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">to_s</span><span class="si">}</span><span class="s2">, #inspect=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">, #__id__=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">__id__</span><span class="si">}</span><span class="s2">, shifted_id=</span><span class="si">#{</span><span class="p">(</span><span class="n">o</span><span class="p">.</span><span class="nf">__id__</span> <span class="o"><<</span> <span class="mi">1</span><span class="p">).</span><span class="nf">to_s</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span> <span class="p">}</span>
<span class="s2">"#to_s=#<Object:0x0000000000d202a8>, #inspect=#<Object:0x0000000000d202a8>, #__id__=6881620, shifted_id=d202a8"</span>
</code></pre>
<a name="with-ruby-gt-27"></a>
<h2 >with ruby >= 2.7<a href="#with-ruby-gt-27" class="wiki-anchor">¶</a></h2>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="no">Object</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">tap</span> <span class="p">{</span> <span class="o">|</span><span class="n">o</span><span class="o">|</span> <span class="nb">p</span> <span class="s2">"#to_s=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">to_s</span><span class="si">}</span><span class="s2">, #inspect=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">, #__id__=</span><span class="si">#{</span><span class="n">o</span><span class="p">.</span><span class="nf">__id__</span><span class="si">}</span><span class="s2">, shifted_id=</span><span class="si">#{</span><span class="p">(</span><span class="n">o</span><span class="p">.</span><span class="nf">__id__</span> <span class="o"><<</span> <span class="mi">1</span><span class="p">).</span><span class="nf">to</span>
<span class="n">s</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span> <span class="p">}</span>
<span class="s2">"#to_s=#<Object:0x0000555dc8640b88>, #inspect=#<Object:0x0000555dc8640b88>, #__id__=220, shifted_id=1b8"</span>
</code></pre>
<a name="Consequences"></a>
<h1 >Consequences<a href="#Consequences" class="wiki-anchor">¶</a></h1>
<p>It makes harder:</p>
<ul>
<li>to implement a clean override of the <code>#inspect</code> method. i.e. How to keep the same output without ability to compute to the same "object_id" value.</li>
<li>to debug the object using the inspect output. i.e. <code>ObjectSpace._id2ref(id_from_inspect >> 1)</code> used to work, now it doesn't (<code>RangeError: <xXx> is not id value</code>).</li>
</ul>
<a name="Suggestion"></a>
<h1 >Suggestion<a href="#Suggestion" class="wiki-anchor">¶</a></h1>
<p>IMHO either:</p>
<ul>
<li>the <code>#to_s</code> and <code>#inspect</code> documentation are obsolete <code>The default [...] [shows|prints] [...] an encoding of the object id</code> and the change could have been a bit more advertised</li>
<li>they should use the result of <code>#object_id</code> instead of displaying the object pointer address</li>
</ul>
<p>Another solution could be to provide a method to get access to the address, but I'm not sure you want that.</p>
<p>P.S. While debugging my problem I found <a href="https://www.ruby-forum.com/t/understanding-object-id-in-ruby-2-7/260268/4" class="external">this ruby-forum thread</a> where people dived a bit more than me into ruby's code.</p> Ruby master - Misc #17180 (Open): Ractor and constant referencinghttps://bugs.ruby-lang.org/issues/171802020-09-19T21:24:45Zkirs (Kir Shatrov)shatrov@me.com
<p>Hey there.</p>
<p>From a high level, this bug report is describing my experience building something simple with Ractor without any shared state. I hope this is helpful as a feedback for the stable Ractor implementation in Ruby 3.0.</p>
<p>I've been iterating on a simple web server that utilizes Ractor. One of the things that a web server does is parsing HTTP headers. Thankfully, there's code in Ruby's standard library that can do that, so I decided to leverage <code>WEBrick::HTTPRequest</code>.</p>
<p>Here's my example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'webrick'</span>
<span class="nb">require</span> <span class="s1">'stringio'</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Ractor</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">s</span> <span class="o">=</span> <span class="no">StringIO</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s2">"GET / HTTP/1.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Host: example.com</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"User-Agent: curl/7.64.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Accept: */*"</span><span class="p">)</span>
<span class="n">req</span> <span class="o">=</span> <span class="no">WEBrick</span><span class="o">::</span><span class="no">HTTPRequest</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">WEBrick</span><span class="o">::</span><span class="no">Config</span><span class="o">::</span><span class="no">HTTP</span><span class="p">)</span>
<span class="n">req</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Ractor</span><span class="p">.</span><span class="nf">select</span><span class="p">(</span><span class="n">parser</span><span class="p">)</span>
</code></pre>
<p>This fails with <code>can not access non-sharable objects in constant WEBrick::Config::HTTP by non-main Ractor. (NameError)</code>.</p>
<p>Let's try to copy that constant and pass it to the Ractor:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'webrick'</span>
<span class="nb">require</span> <span class="s1">'stringio'</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Ractor</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">WEBrick</span><span class="o">::</span><span class="no">Config</span><span class="o">::</span><span class="no">HTTP</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
<span class="n">s</span> <span class="o">=</span> <span class="no">StringIO</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s2">"GET / HTTP/1.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Host: example.com</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"User-Agent: curl/7.64.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Accept: */*"</span><span class="p">)</span>
<span class="n">req</span> <span class="o">=</span> <span class="no">WEBrick</span><span class="o">::</span><span class="no">HTTPRequest</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">config</span><span class="p">)</span>
<span class="n">req</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Ractor</span><span class="p">.</span><span class="nf">select</span><span class="p">(</span><span class="n">parser</span><span class="p">)</span>
</code></pre>
<p>This failed with <code>can't dump hash with default proc</code>. I've guessed that maybe <code>WEBrick::Config::HTTP</code> has a default proc? Let's try to work around:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'webrick'</span>
<span class="nb">require</span> <span class="s1">'stringio'</span>
<span class="n">http_c</span> <span class="o">=</span> <span class="no">WEBrick</span><span class="o">::</span><span class="no">Config</span><span class="o">::</span><span class="no">HTTP</span>
<span class="n">http_c</span><span class="p">.</span><span class="nf">default</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="n">parser</span> <span class="o">=</span> <span class="no">Ractor</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">http_c</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
<span class="n">s</span> <span class="o">=</span> <span class="no">StringIO</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s2">"GET / HTTP/1.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Host: example.com</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"User-Agent: curl/7.64.1</span><span class="se">\n</span><span class="s2">"</span><span class="p">\</span>
<span class="s2">"Accept: */*"</span><span class="p">)</span>
<span class="n">req</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Ractor</span><span class="p">.</span><span class="nf">select</span><span class="p">(</span><span class="n">parser</span><span class="p">)</span>
</code></pre>
<p>It failed with:</p>
<pre><code>/Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:570:in `read_line': can not access non-sharable objects in constant WEBrick::LF by non-main ractor. (NameError)
</code></pre>
<p>I went ahead and made WEBrick freeze strings:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/webrick/httputils.rb b/lib/webrick/httputils.rb
index 76d4bd0dc7..273c596368 100644
</span><span class="gd">--- a/lib/webrick/httputils.rb
</span><span class="gi">+++ b/lib/webrick/httputils.rb
</span><span class="p">@@ -13,9 +13,9 @@</span>
require 'tempfile'
<span class="err">
</span> module WEBrick
<span class="gd">- CR = "\x0d" # :nodoc:
- LF = "\x0a" # :nodoc:
- CRLF = "\x0d\x0a" # :nodoc:
</span><span class="gi">+ CR = "\x0d".freeze # :nodoc:
+ LF = "\x0a".freeze # :nodoc:
+ CRLF = "\x0d\x0a".freeze # :nodoc:
</span><span class="err">
</span></code></pre>
<p>Now it's failing with:</p>
<pre><code>/Users/kir/src/github.com/ruby/ruby/lib/singleton.rb:124:in `instance': can not access instance variables of classes/modules from non-main Ractors (RuntimeError)
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/utils.rb:132:in `register'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/utils.rb:256:in `timeout'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:559:in `_read_data'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:570:in `read_line'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:447:in `read_request_line'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:201:in `parse'
from ./test.rb:16:in `block in <main>'
</code></pre>
<p>because WEBrick is manipulating with a class variable.</p>
<p>I've commented out some of that code in WEBrick for the sake of experiment and it started to fail with:</p>
<pre><code>/Users/kir/src/github.com/ruby/ruby/lib/uri/common.rb:171:in `parse': can not access non-sharable objects in constant URI::RFC3986_PARSER by non-main ractor. (NameError)
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:484:in `parse_uri'
from /Users/kir/src/github.com/ruby/ruby/lib/webrick/httprequest.rb:217:in `parse'
</code></pre>
<p>This can be easily reproduced with a smaller piece of code:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">parser</span> <span class="o">=</span> <span class="no">Ractor</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="no">URI</span><span class="o">::</span><span class="n">parse</span><span class="p">(</span><span class="s2">"http://example.com"</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Ractor</span><span class="p">.</span><span class="nf">select</span><span class="p">(</span><span class="n">parser</span><span class="p">)</span>
</code></pre>
<hr>
<p>IMO, something as tiny and safe as <code>URI::parse</code> should be allowrd to be called from a Ractor. We're either being too strict to the developer in terms of what constants they can reference from inside a Ractor, or we're missing a bunch of <code>freeze</code> in the standard library. It's likely the latter, but in that case we'll need to push for <code>freeze</code> as much as we can to make Ractor friendly to developers.</p>
<p>I'm very excited to see this happening in Ruby 3.0 and I'd like to help as much as I can to make it a great feature of Ruby.</p> Ruby master - Misc #17154 (Open): Update Pathname Documentation to Clarify Expected Behaviorhttps://bugs.ruby-lang.org/issues/171542020-09-05T02:29:29Zresperat (Ralph Esperat)
<p>I would like to suggest adding a sentence to the documentation for <a href="https://ruby-doc.org/stdlib-2.7.1/libdoc/pathname/rdoc/Pathname.html" class="external">Pathname</a> to make clear the unusual behavior of <code>Pathname#+</code> when an absolute path is included in the arguments. In such a situation, <code>Pathname#+</code> drops the paths prior to the last absolute path which I understand to be the intended behavior, but it is not obviously intended, only showing up tangentially as an example in the documentation.</p>
<pre><code>p3 = p1 + "/etc/passwd" # Pathname:/etc/passwd
</code></pre>
<p>The Pathname documentation states that " <strong>All functionality</strong> from File, FileTest, and some from Dir and FileUtils is included, <strong>in an unsurprising way</strong> ..." and later when referring to core methods such as <code>Pathname#+</code> "These methods are <strong>effectively manipulating a String</strong> , because that's all a path is." However, similar uses of both <a href="https://ruby-doc.org/core-2.7.1/File.html" class="external">File</a> and <a href="https://ruby-doc.org/core-2.7.1/String.html" class="external">String</a> would produce the expected result of including all of the arguments:</p>
<pre><code>s1 = "/usr"
s2 = s1 + "/etc/passwd" # "/usr/etc/passwd"
f1 = File.new("/usr" + "/etc/passwd") # (No such file or directory @ rb_sysopen - /usr/etc/passwd)
</code></pre>
<p>A bug report was previously filed to "fix" this functionality (<a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Status: Rejected" href="https://bugs.ruby-lang.org/issues/15564">Bug #15564: Pathname#+(pathpart) returns pathpart when pathpart is absolute</a>), not understanding it to be intentional. Other common help websites such as <a href="https://stackoverflow.com/questions/12464361/concatenating-absolute-paths-with-the-pathname-class" class="external">Stack Overflow</a> also show users who do not expect this behavior from this method. Adding a statement to the documentation for the <a href="https://ruby-doc.org/stdlib-2.7.1/libdoc/pathname/rdoc/Pathname.html#method-i-2B" class="external">Pathname#+</a> method will make it clear to users exactly what to expect and that this is the intended behavior. I would suggest simply the following, the first sentence of which is already present:</p>
<p><code>Appends a pathname fragment to self to produce a new Pathname object. If an absolute path is provided as any of the arguments, discards all arguments prior to the last absolute path provided.</code></p>
<p>I appreciate your consideration of this request.</p>
<p>ruby -v: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]</p> Ruby master - Misc #17137 (Assigned): Cooperation on maintaining official docker ruby imageshttps://bugs.ruby-lang.org/issues/171372020-08-31T19:52:31Zdeivid (David Rodríguez)
<p>It was pointed out to me at <a href="https://github.com/docker-library/ruby/issues/323" class="external">https://github.com/docker-library/ruby/issues/323</a> that the ruby-core team has started maintaining their own docker images at <a href="https://github.com/ruby/ruby-docker-images" class="external">https://github.com/ruby/ruby-docker-images</a>, and that the base Dockerfiles were initially started from the official docker images.</p>
<p>The maintainers of the official images would be interesting in collaborating on maintaining these images. Maybe merging the projects would be a nice idea from an end user point of view. I'm guessing there's a reason why <a href="https://github.com/ruby/ruby-docker-images" class="external">https://github.com/ruby/ruby-docker-images</a> was started as a separate project, but maybe any improvements over the official project could be merged back. The obvious new feature that I see in the README is the ability to build development images of specific revisions.</p>
<p>Anyways, I mentioned the approach of the docker folks to hsbt and he told me to open a ticket here. So here it is!</p>
<p>Regards!</p> Ruby master - Misc #17053 (Open): RDoc for Hash Keyshttps://bugs.ruby-lang.org/issues/170532020-07-27T15:42:50Zburdettelamar@yahoo.com (Burdette Lamar)burdettelamar@example.com
<p>marcandre <a href="https://github.com/ruby/ruby/pull/3139#issuecomment-663281047" class="external">comments</a> on my pull request regarding <a href="https://docs.ruby-lang.org/en/master/Hash.html#class-Hash-label-Hash+Keys" class="external">documentation of Hash in Rdoc</a>:</p>
<blockquote>
<p>The only thing I would change is that I would shorten the doc on the "Invalid Hash Keys". As far as I know, this is simply not a[n] important concern as nearly all Ruby objects respond_to? :hash and :eql?</p>
</blockquote>
<blockquote>
<p>I personally would recommend adding a single example in the Hash.html#class-Hash-label-Hash+Keys section and I would remove the rest, or at least remove the examples. They burden the reader with something that is of no use to them.</p>
</blockquote>
<p>I have misgivings about it:</p>
<ul>
<li>Some of the material, for example, the text and example for user-defined hash keys, is very old.</li>
<li>I consolidated some material from earlier doc for individual methods, which now link to the relevant sections.</li>
<li>All are factual and not repeated elsewhere in the page.</li>
</ul>
<p>This is an API reference documentation. Ruby should have a <em>reference documentation</em>, and therefore should not omit anything.</p>
<p>If such material is to be included, I see three possibilities:</p>
<ul>
<li>Include inline, as is now.</li>
<li>Link to a footnote on the same page, with a back link.</li>
<li>Link to another rdoc page in <code>doc/</code> dir.</li>
</ul>
<p>I'd love to hear some opinions on this.</p> Ruby master - Misc #16895 (Open): Request for cooperation: Try your applications/libraries with m...https://bugs.ruby-lang.org/issues/168952020-05-15T09:13:23Zko1 (Koichi Sasada)
<a name="Summary"></a>
<h2 >Summary<a href="#Summary" class="wiki-anchor">¶</a></h2>
<p>If you maintain a Ruby application or library, please consider testing with the Ruby <code>master</code> branch, including the <em>debug</em> build. This should be in addition to testing with all supported stable releases of Ruby.</p>
<p>To make this easier, we are providing Docker images and GitHub actions, as outlined below.</p>
<a name="Details"></a>
<h2 >Details<a href="#Details" class="wiki-anchor">¶</a></h2>
<p>The rapid pace of Ruby development sometimes introduces bugs, such as incorrect behaviour or unexpected incompatibilities. Despite our best efforts and testing, without your feedback, we cannot catch every issue.</p>
<p>Understanding how our changes impact downstream code is important feedback for the Ruby core developers. We want to know how your applications work on master.</p>
<p>If you encounter an error when testing with the <code>master</code> branch (e.g. <code>[BUG] ...</code> in output log), <a href="https://bugs.ruby-lang.org" class="external">please report it</a>. It will be very helpful.</p>
<h3>Testing With <code>master</code>
</h3>
<p>Testing using the <code>master</code> branch (sometimes referred to as <code>ruby-head</code>) will make your Ruby scripts ready for the next Ruby version. It also helps us catch incompatibilities as we change and evolve Ruby's public interface.</p>
<a name="Testing-With-Debug-Build"></a>
<h3 >Testing With Debug Build<a href="#Testing-With-Debug-Build" class="wiki-anchor">¶</a></h3>
<p>Testing with the master branch debug build enables many assertions within the Ruby interpreter.</p>
<p>These assertions can detect incorrect usage of the C extensions, and also bugs in the interpreter when running your program.</p>
<p>These assertions have an impact on the performance of the interpreter.</p>
<p>To compile a debug build, refer the later section titled "Building With Debug Mode".</p>
<h2>Continuous Integration With <code>master</code>
</h2>
<p>Building Ruby for your own testing environment can be difficult, so we are providing two convenient ways to use the master branch in your existing testing pipeline:</p>
<ul>
<li>Docker Images</li>
<li>Github Action</li>
</ul>
<a name="Docker-Images"></a>
<h3 >Docker Images<a href="#Docker-Images" class="wiki-anchor">¶</a></h3>
<p>The <a href="https://hub.docker.com/r/rubylang/ruby" class="external">rubylang docker repository</a> provides images for various Ruby versions, including nightly builds of master with and without debug assertions"</p>
<ul>
<li>Nightly built master: <code>rubylang/ruby:master-nightly-bionic</code>
</li>
<li>Nightly debug built master: <code>rubylang/ruby:master-debug-nightly-bionic</code>
</li>
</ul>
<p>Here is an example <code>Dockerfile</code>:</p>
<pre><code class="dockerfile syntaxhl" data-language="dockerfile"><span class="k">FROM</span><span class="s"> rubylang/ruby:master-nightly-bionic</span>
</code></pre>
<p>Then to build:</p>
<pre><code class="shell syntaxhl" data-language="shell"><span class="nv">$ </span>docker build <span class="nb">.</span>
Sending build context to Docker daemon 2.048kB
Step 1/1 : FROM rubylang/ruby:master-nightly-bionic
master-nightly-bionic: Pulling from rubylang/ruby
...
Status: Downloaded newer image <span class="k">for </span>rubylang/ruby:master-nightly-bionic
<span class="nt">---</span><span class="o">></span> 059d367a8fbd
Successfully built 059d367a8fbd
</code></pre>
<a name="GitHub-Action"></a>
<h3 >GitHub Action<a href="#GitHub-Action" class="wiki-anchor">¶</a></h3>
<p>The <a href="https://github.com/ruby/setup-ruby" class="external">GitHub Action to setup Ruby</a> provides both <code>ruby-head</code> and <code>ruby-debug</code> builds.</p>
<p>Here is an example workflow to test on all Ruby stable releases, including <code>ruby-head</code> and <code>ruby-debug</code>:</p>
<pre><code class="yaml syntaxhl" data-language="yaml"><span class="na">name</span><span class="pi">:</span> <span class="s">Development</span>
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">push</span><span class="pi">]</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">test</span><span class="pi">:</span>
<span class="na">strategy</span><span class="pi">:</span>
<span class="na">fail-fast</span><span class="pi">:</span> <span class="kc">false</span>
<span class="na">matrix</span><span class="pi">:</span>
<span class="na">os</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">ubuntu</span><span class="pi">]</span>
<span class="na">ruby</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">2.5</span><span class="pi">,</span> <span class="nv">2.6</span><span class="pi">,</span> <span class="nv">2.7</span><span class="pi">,</span> <span class="nv">head</span><span class="pi">,</span> <span class="nv">debug</span><span class="pi">]</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">${{ matrix.os }}-latest</span>
<span class="na">continue-on-error</span><span class="pi">:</span> <span class="s">${{ matrix.ruby == 'head' || matrix.ruby == 'debug' }}</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">ruby/setup-ruby@v1</span>
<span class="na">with</span><span class="pi">:</span>
<span class="na">ruby-version</span><span class="pi">:</span> <span class="s">${{ matrix.ruby }}</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">bundle install</span>
<span class="pi">-</span> <span class="na">run</span><span class="pi">:</span> <span class="s">bundle exec rake</span>
</code></pre>
<p>See the <a href="https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby" class="external">documentation</a> for more details on how to use this action.</p>
<a name="Building-With-Debug-Mode"></a>
<h2 >Building With Debug Mode<a href="#Building-With-Debug-Mode" class="wiki-anchor">¶</a></h2>
<p>To create a debug build of Ruby, execute the following commands:</p>
<pre><code>$ git clone https://github.com/ruby/ruby.git
$ cd ruby
$ autoconf
$ cppflags=-DRUBY_DEBUG=1 ./configure --prefix=$HOME/.rubies/ruby-debug
$ make install
</code></pre>
<p>If you are using <code>chruby</code>, you can switch to the above build:</p>
<pre><code>$ chruby ruby-debug
</code></pre>
<p>You can find more details to build Ruby master in the <a href="https://github.com/ruby/ruby/blob/master/README.md#how-to-compile-and-install" class="external">README</a>.</p>
<a name="Acknowledgement"></a>
<h2 >Acknowledgement<a href="#Acknowledgement" class="wiki-anchor">¶</a></h2>
<p>We thank <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/482">@mrkn (Kenta Murata)</a> for the Docker image, <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> for the GitHub Action, and <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/3344">@ioquatix (Samuel Williams)</a> for reviewing this announcement.</p> Ruby master - Misc #16805 (Assigned): Coroutine's license is unclearhttps://bugs.ruby-lang.org/issues/168052020-04-21T10:23:49Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<p>Files under <code>coroutine/</code> start like this:</p>
<pre><code class="C syntaxhl" data-language="C"><span class="o">/*</span>
<span class="o">*</span> <span class="n">This</span> <span class="n">file</span> <span class="n">is</span> <span class="n">part</span> <span class="n">of</span> <span class="n">the</span> <span class="s">"Coroutine"</span> <span class="n">project</span> <span class="n">and</span> <span class="n">released</span> <span class="n">under</span> <span class="n">the</span> <span class="n">MIT</span> <span class="n">License</span><span class="p">.</span>
<span class="o">*</span>
</code></pre>
<p>The problem is, there is no definition of "the MIT License" throughout the entire project.</p>
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/3344">@ioquatix (Samuel Williams)</a> can you add your terms somewhere inside of LEGAL? There are several other materials also under the MIT license. You can follow how they are listed up.</p> Ruby master - Misc #16803 (Open): Discussion: those internal macros reside in public API headershttps://bugs.ruby-lang.org/issues/168032020-04-21T07:45:49Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<p>A while ago I merged <a href="https://github.com/ruby/ruby/pull/2991" class="external">https://github.com/ruby/ruby/pull/2991</a> ("Split ruby.h"). This seems working. But the changeset raised several questions.</p>
<p>The biggest one relates to those newly publicised macros and inline functions. For instance <code>RUBY3_STATIC_ASSERT</code> is a macro that expands to either <code>_Static_assert</code> (for C) or <code>static_assert</code> (for C++). A similar mechanism has been implemented inside of our repository for a while. The pull request moved the definition around to be visible outside.</p>
<a name="Discussion-1"></a>
<h4 >Discussion #1<a href="#Discussion-1" class="wiki-anchor">¶</a></h4>
<p>Is it a good idea or a bad idea, to make them visible worldwide?</p>
<a name="Discussion-2"></a>
<h4 >Discussion #2<a href="#Discussion-2" class="wiki-anchor">¶</a></h4>
<p>Why not publicise everything? For instance debuggers could benefit from ruby internal symbols.</p>
<a name="Discussion-3"></a>
<h4 >Discussion #3<a href="#Discussion-3" class="wiki-anchor">¶</a></h4>
<p>It is relatively hard for us to change public APIs (doing so could break 3rd party gems). We don't want that happen for internal APIs. How do we achieve future flexibility?</p> Ruby master - Misc #16750 (Open): Change typedef of VALUE for better type checkinghttps://bugs.ruby-lang.org/issues/167502020-04-01T18:22:10ZDan0042 (Daniel DeLorme)
<p>VALUE is currently defined as <code>typedef unsigned long VALUE</code>, but as we all know that only creates an <em>alias</em>, not an actual <em>type</em>. Since the compiler gives no warnings when comparing with other integer values, it's easy to have bugs such as <code>v == 42</code> which should have been <code>v == INT2FIX(42)</code>. Actually not so long ago I saw nobu fixing a bug of that kind where an ID had been mixed up with a VALUE.</p>
<p>So in order to prevent these kinds of bugs I propose changing VALUE to a non-scalar type such as:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="c1">//example for 64-bit system</span>
<span class="k">typedef</span> <span class="k">union</span> <span class="n">VALUE</span> <span class="p">{</span>
<span class="k">struct</span> <span class="n">RBasic</span><span class="o">*</span> <span class="n">as_basic</span><span class="p">;</span> <span class="c1">//easy access to obj.as_basic->klass and obj.as_basic->flags</span>
<span class="kt">void</span><span class="o">*</span> <span class="n">as_ptr</span><span class="p">;</span> <span class="c1">//can assign ptr = obj.as_ptr without a cast</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">i</span><span class="p">;</span> <span class="c1">//raw int value for bitwise operations</span>
<span class="kt">int</span> <span class="n">immediate</span> <span class="o">:</span> <span class="mi">3</span><span class="p">;</span> <span class="c1">//obj.immediate != 0 if obj is immediate type such as fixnum, flonum, static symbol</span>
<span class="kt">int</span> <span class="n">is_fixnum</span> <span class="o">:</span> <span class="mi">1</span><span class="p">;</span> <span class="c1">//obj.is_fixnum == 1 if obj is fixnum</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">flag</span> <span class="o">:</span> <span class="mi">2</span><span class="p">;</span> <span class="c1">//obj.flonum.flag == 2 if obj is flonum</span>
<span class="kt">int</span> <span class="n">bits</span> <span class="o">:</span> <span class="mi">62</span><span class="p">;</span>
<span class="p">}</span> <span class="n">flonum</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="kt">int</span> <span class="n">flag</span> <span class="o">:</span> <span class="mi">8</span><span class="p">;</span> <span class="c1">//obj.symbol.flag == 0x0c if obj is a static symbol</span>
<span class="kt">int</span> <span class="n">id</span> <span class="o">:</span> <span class="mi">56</span><span class="p">;</span> <span class="c1">//-> obj.symbol.id == STATIC_SYM2ID(obj)</span>
<span class="p">}</span> <span class="n">symbol</span><span class="p">;</span>
<span class="p">}</span> <span class="n">VALUE</span><span class="p">;</span>
</code></pre>
<p>This will allow proper type-checking via the compiler.</p>
<p>A little-known fact, structs and unions can be passed (and returned) by value to functions; this 64-bit union has the same performance as a 64-bit int. This approach also allows to simplify code like <code>((struct RBasic*)obj)->flags</code> into the much more readable <code>obj.as_basic->flags</code>, and <code>FIXNUM_P(obj)</code> can be expressed directly as <code>obj.is_fixnum</code>. The only downside is that direct comparison of union variables is not possible, so you would need to use for example <code>obj.i == Qfalse.i</code></p>
<p>To summarize, stricter type-checking would eliminate an entire class of bugs, and I estimate this change would require modifications to <strong>no more than 14%</strong> of the codebase (62,213 lines). Very much worth it I believe.</p> Ruby master - Misc #16747 (Assigned): Repository reorganization requesthttps://bugs.ruby-lang.org/issues/167472020-04-01T06:58:09Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<p>Back in 0.49, there were only 60 files and 3 directories at the root of this project. This was already at some level, but OK-ish. Now, as we are reaching 3.0, we currently have 167 files and 26 directories. The project has grown up. I believe we need some housekeeping.</p>
<p>I would like to introduce directories and move things around, like <a href="https://github.com/jemalloc/jemalloc/" class="external">what they do for jemalloc</a>.</p>
<ul>
<li>Create directory named <code>src</code> and move sources there.</li>
</ul>
<p>There is no need to cargo-cult them so suggestions are welcome.</p> Ruby master - Misc #16678 (Open): Array#values_at has unintuitive behavior when supplied a range ...https://bugs.ruby-lang.org/issues/166782020-03-07T06:21:48Zprajjwal (Prajjwal Singh)
<p>Consider the following:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># frozen_string_literal: true</span>
<span class="n">a</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">to_a</span>
<span class="nb">p</span> <span class="n">a</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="mi">3</span><span class="o">..</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># => [4, 5, nil]</span>
<span class="nb">p</span> <span class="n">a</span><span class="p">.</span><span class="nf">values_at</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="o">..</span><span class="mi">3</span><span class="p">)</span> <span class="c1"># => []</span>
</code></pre>
<p>When the range begins with a negative <code>(-1, 0, 1, 2, 3)</code>, it returns an empty array, which surprised me because I was expecting <code>[1, 2, 3, 4]</code>.</p>
<p>The argument for this is that it cold be confusing to allow this because the index <code>-1</code> could refer to the last argument and it would be unintuitive to return an array <code>[5, 1, 2, 3, 4]</code> with jumbled values.</p>
<p>The argument against it is that it makes perfect sense to account for this case and return <code>[nil, 1, 2, 3, 4]</code>.</p>
<p>Opening a dialog to see what others think of this.</p> Ruby master - Misc #16659 (Open): Documentation on Regexp missing for absence pattern (?~pat)https://bugs.ruby-lang.org/issues/166592020-02-27T16:16:20Zsvoop (Sven Schwyn)svoop_he38hj327c@delirium.ch
<p>The absence pattern <code>(?~pat)</code> available since Ruby 2.4.1 is <a href="https://git.ruby-lang.org/ruby.git/tree/doc/regexp.rdoc" class="external">not yet documented on <code>Regexp</code></a> as of today.</p>
<p>(Found it by coincidence reading <a href="https://medium.com/rubyinside/the-new-absent-operator-in-ruby-s-regular-expressions-7c3ef6cd0b99" class="external">this article by Peter Cooper</a>.</p> Ruby master - Misc #16630 (Assigned): Deprecate pub/ruby/*snapshot* and use pub/ruby/snapshot/* i...https://bugs.ruby-lang.org/issues/166302020-02-13T08:51:43Zznz (Kazuhiro NISHIYAMA)
<p>In <a href="https://www.ruby-lang.org/en/downloads/" class="external">https://www.ruby-lang.org/en/downloads/</a>, snapshots links to <code>pub/ruby/snapshot.*</code> and <code>pub/ruby/stable-snapshot.*</code> as official snapshot tarballs now.</p>
<p>I want to change links to snapshot tarballs in <a href="https://cache.ruby-lang.org/pub/ruby/snapshot/" class="external">https://cache.ruby-lang.org/pub/ruby/snapshot/</a>.</p>
<p>They created by <a href="https://github.com/ruby/actions/" class="external">https://github.com/ruby/actions/</a> now.</p>
<p>Tarballs under <code>pub/ruby/snapshot/</code> have branch name (e.g. <code>ruby_2_7</code>) in filenames.<br>
And they are tested. (but they remain even if tests failed.)</p>
<p><code>pub/ruby/*snapshot.*</code> are not tested.<br>
And there are fewer files under <code>pub/ruby/</code> without sub-directory recently. (e.g. <code>ruby- 2.7.*</code> are in <code>pub/ruby/2.7/</code> only.)<br>
So I want to remove them.</p>
<p>Because of the plan, <code>stable-snapshot.*</code> are still snapshot of <code>ruby_2_6</code> instead of <code>ruby_2_7</code>.</p> Ruby master - Misc #16512 (Assigned): Improving `www.ruby-lang.org` reference by merging with `ru...https://bugs.ruby-lang.org/issues/165122020-01-16T08:38:13Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/710">@zverok (Victor Shepelev)</a> prepared better-looking reference pages at <code>rubyreferences.github.io</code>. I think there's room for improvement of reference pages on <code>www.ruby-lang.org</code>. <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/572">@hsbt (Hiroshi SHIBATA)</a> Is there a chance to work with?</p>
<p>Matz.</p> Ruby master - Misc #16507 (Open): =~ vs include? or match?https://bugs.ruby-lang.org/issues/165072020-01-12T23:27:07ZMSP-Greg (Greg L)
<p>While working on getting the mswin build working on Actions, I thought adding mswin? and ci? methods to Minitest::Unit::Guard in tool/lib/minitest/unit.rb would be helpful.</p>
<p>Currently some tests are failing that are guarded/skipped based on ENV['APPVEYOR']. For ci?, I'd combine Travis, AppVeyor, & Actions.</p>
<p>There are quite a few instances where =~ is used for a boolean return. Would it be considered appropriate to replace those calls with include? or match?</p> Ruby master - Misc #16487 (Open): Potential for SIMD usage in ruby-corehttps://bugs.ruby-lang.org/issues/164872020-01-08T09:48:29Zbyroot (Jean Boussier)byroot@ruby-lang.org
<a name="Context"></a>
<h3 >Context<a href="#Context" class="wiki-anchor">¶</a></h3>
<p>There are several ruby core methods that could be optimized with the use of SIMD instructions.</p>
<p>I experimented a bit on <code>coderange_scan</code> <a href="https://github.com/Shopify/ruby/pull/2" class="external">https://github.com/Shopify/ruby/pull/2</a>, and Pavel Rosický experimented on <code>String#strip</code> <a href="https://github.com/ruby/ruby/pull/2815" class="external">https://github.com/ruby/ruby/pull/2815</a>.</p>
<a name="Problem"></a>
<h3 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h3>
<p>The downside of SIMD instructions is that they are not universally available.<br>
So it means maintaining several versions of the same code, and switching them either statically or dynamically.</p>
<p>And since most Ruby users use precompiled binaries from repositories and such, it would need to be dynamic if we want most users to benefit from it.</p>
<p>So it's not exactly "free speed", as it means a complexified codebase.</p>
<a name="Question"></a>
<h3 >Question<a href="#Question" class="wiki-anchor">¶</a></h3>
<p>So the question is to know wether ruby-core is open to patches using SIMD instructions ? And if so under which conditions.</p>
<p>cc <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/10">@shyouhei (Shyouhei Urabe)</a></p> Ruby master - Misc #16436 (Open): hash missing #last method, make it not so consistent (it has #f...https://bugs.ruby-lang.org/issues/164362019-12-19T16:21:35Zzw963 (Wei Zheng)
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span> <span class="o">=</span> <span class="p">{</span><span class="n">x</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span> <span class="n">z</span><span class="p">:</span><span class="mi">3</span><span class="p">}</span>
<span class="p">{</span>
<span class="ss">:x</span> <span class="o">=></span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">:y</span> <span class="o">=></span> <span class="mi">2</span><span class="p">,</span>
<span class="ss">:z</span> <span class="o">=></span> <span class="mi">3</span>
<span class="p">}</span>
<span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">first</span>
<span class="p">[</span>
<span class="ss">:x</span><span class="p">,</span>
<span class="mi">1</span>
<span class="p">]</span>
<span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">first</span>
<span class="ss">:x</span>
<span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">last</span>
<span class="mi">1</span>
</code></pre>
<p>last method not exist.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">last</span>
<span class="no">Exception</span><span class="p">:</span> <span class="no">NoMethodError</span><span class="p">:</span> <span class="n">undefined</span> <span class="nb">method</span> <span class="sb">`last' for {:x=>1, :y=>2, :z=>3}:Hash
</span></code></pre>
<p>We have to use #to_a to make last working.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">to_a</span><span class="p">.</span><span class="nf">last</span>
<span class="p">[</span>
<span class="ss">:z</span><span class="p">,</span>
<span class="mi">3</span>
<span class="p">]</span>
<span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">to_a</span><span class="p">.</span><span class="nf">last</span><span class="p">.</span><span class="nf">first</span>
<span class="ss">:z</span>
<span class="p">(</span><span class="n">pry</span><span class="p">)</span><span class="ss">:main</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">></span> <span class="n">h</span><span class="p">.</span><span class="nf">to_a</span><span class="p">.</span><span class="nf">last</span><span class="p">.</span><span class="nf">last</span>
<span class="mi">3</span>
</code></pre> Ruby master - Misc #16408 (Open): Ruby docs list incorrect method signatures for PTY::getpty/PTY:...https://bugs.ruby-lang.org/issues/164082019-12-08T18:30:18Zyarmiganosca (Chris Hoffman)
<p>Ruby documentation says the method signature for <code>PTY::getpty</code>/<code>PTY::spawn</code> is</p>
<pre><code> * PTY.spawn(command_line) { |r, w, pid| ... }
* PTY.spawn(command_line) => [r, w, pid]
* PTY.spawn(command, arguments, ...) { |r, w, pid| ... }
* PTY.spawn(command, arguments, ...) => [r, w, pid]
</code></pre>
<p>However, running the following command with any Ruby since at least 2.1.9 (and based on git history, I believe since 2.0.0) will demonstrate that these method signatures are incorrect:</p>
<pre><code>ruby -rpty -e 'r, w, pid = PTY.spawn({"GREETING" => "hello"}, "echo $GREETING world"); puts r.gets(11); r.close; w.close'
</code></pre>
<p>(it will output <code>hello world</code>)</p>
<p>Further testing confirms that, aside from the block, <code>PTY::spawn</code> has the same method signature as <code>Kernel#exec</code>, <code>Kernel#system</code>, <code>Kernel#spawn</code>, and <code>Process::spawn</code>. This makes sense given that their C implementations all call the C method <code>rb_execarg_new</code> to handle the complex argument possibilities they all have to deal with.</p>
<p>I have a branch with tests that verify this: <a href="https://github.com/yarmiganosca/ruby/tree/fix-PTY-getpty-and-spawn-docs-and-tests" class="external">https://github.com/yarmiganosca/ruby/tree/fix-PTY-getpty-and-spawn-docs-and-tests</a></p>
<p>Travis isn't running CI on this branch, but I'm assuming that's account related and not an issue with my code, especially since the UI just informs me that I'm not allowed to.</p>
<p>I'm happy to also change the docs for <code>PTY::spawn</code> in this branch, but I wasn't sure which docs to emulate. The docs for this "family" of methods are inconsistent in how they explain the complicated options available to users:</p>
<ul>
<li>
<code>Kernel#spawn</code> explains in full the argument possibilities.</li>
<li>
<code>Kernel#exec</code> and <code>Kernel#system</code> both link to the docs for <code>Kernel#spawn</code>.</li>
<li>
<code>Process::spawn</code> explains in full (I think it might be a copy-paste job from the <code>Kernel#spawn</code> docs).</li>
<li>The docs for <code>IO::popen</code> (another method that uses the "execargs signature" with some modifications) list, but don't explain all the argument possibilities. They also don't do all that great a job at explaining where the other arguments from <code>IO::popen</code>'s signature fit alongside the "execargs signature".</li>
</ul>
<p>So it seems that there isn't consensus on the best way to explain this complex method signature (and modifications of it) to readers of the docs. Like I said, I'm happy to fix the <code>PTY::spawn</code> docs in this branch, but I was hoping for some guidance on which of the above documentation approaches to emulate.</p> Ruby master - Misc #16396 (Open): What is the reason for this behaviour of Find.find?https://bugs.ruby-lang.org/issues/163962019-12-03T13:03:26Zstiuna (Juan Gregorio)cart4for1@mail.com
<p>When I have a script at <code>D:\Downloads\Ruby 2.5.3\rbL\comp\codeShort.rb</code> with few other files in the same folder, the following searches only the folder where the <code>.rb</code> script is located:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Find</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="s1">'D:'</span><span class="p">)</span>
</code></pre>
<p>When I have a script at <code>D:\Downloads\Ruby 2.5.3\rbL\codeShort.rb</code> with many other files in the same folder, the same code as above searches the entire disk D.</p>
<p>To search the entire disk D in the first case, I did this:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Find</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="s1">'D:/'</span><span class="p">)</span>
</code></pre>
<p>But I don't understand why the two cases behave differently with the same instruction just because they script are in different directories.</p> Ruby master - Misc #16346 (Open): Confusing macro name: RUBY_MARK_NO_PIN_UNLESS_NULLhttps://bugs.ruby-lang.org/issues/163462019-11-13T07:39:44Zy_kojima (Yohei Kojima)
<p>function <code>rb_gc_mark_no_pin</code> is renamed to <code>rb_gc_mark_movable</code> in the commit <code>aac4d9d6c7e6b6b0742f3941b574f6006ccb5672</code>.<br>
But macro <code>RUBY_MARK_NO_PIN_UNLESS_NULL</code> is not renamed then.</p>
<p>Now, <code>RUBY_MARK_NO_PIN_UNLESS_NULL</code> calls <code>rb_gc_mark_movable</code>; this is a bit confusing.<br>
So I suggest renaming <code>RUBY_MARK_NO_PIN_UNLESS_NULL</code> to <code>RUBY_MARK_MOVABLE_UNLESS_NULL</code> like this patch.</p> Ruby master - Misc #16267 (Open): MinGW CI - add to Actions ?https://bugs.ruby-lang.org/issues/162672019-10-21T01:10:18ZMSP-Greg (Greg L)
<p>Actions has three embedded MSYS2 installs, as they currently install the three Windows Rubies with 'DevKits'. So, MinGW could be built/tested on Actions.</p>
<p>I have a GH action at <a href="https://github.com/MSP-Greg/msys2-action" class="external">https://github.com/MSP-Greg/msys2-action</a> that's used in Puma's CI, it allows updating the 'toolchain' and additional packages (typically things like ragel, openssl, etc).</p>
<p>JFYI...</p> Ruby master - Misc #16235 (Open): ENV.assoc spec test does not test invalid namehttps://bugs.ruby-lang.org/issues/162352019-10-03T21:05:12Zburdettelamar@yahoo.com (Burdette Lamar)burdettelamar@example.com
<p>The most important thing here is an added test for an invalid name argument to ENV.assoc, which should raise TypeError.</p>
<p>I've also added to spec_helper.rb:</p>
<ul>
<li>Methods :reserve_names and :release_names, to reserve and release names used in the test.</li>
<li>Method :mock_to_str, to return a mock object that responds to :to_str.</li>
</ul>
<p>The updated assoc_spec.rb uses all of these, as will many other spec tests for ENV, as I get to them.</p>
<p>It's known that some of the ENV spec tests do not test a method's return value, and that some do not test error conditions (as above), so more to come. All in good time.</p> Ruby master - Misc #16188 (Open): What are the performance implications of the new keyword argume...https://bugs.ruby-lang.org/issues/161882019-09-29T18:27:43ZEregon (Benoit Daloze)
<p>In <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: "Real" keyword argument (Closed)" href="https://bugs.ruby-lang.org/issues/14183">#14183</a>, keyword arguments became further separated from positional arguments.</p>
<p>Contrary to the original design though, keyword and positional arguments are not fully separated for methods not accepting keyword arguments.<br>
Example: <code>foo(key: :value)</code> will <code>def foo(hash)</code> will pass a positional argument.<br>
This is of course better for compatibility, but I wonder what are the performance implications.</p>
<p>The block argument is completely separate in all versions, so no need to concern ourselves about that.</p>
<p>In Ruby <= 2.6:</p>
<ul>
<li>The caller never needs to know about the callee's arguments, it can just take all arguments and pass them as an array.<br>
The last argument might be used to extract keyword, but this is all done at the callee side.</li>
<li>Splitting kwargs composed of Symbol and non-Symbol keys can be fairly expensive, but it is a rare occurrence.<br>
If inlining the callee and kwargs are all passed as a literal Hash at the call site, there shouldn't be any overhead compared to positional arguments once JIT'ed.</li>
</ul>
<p>In Ruby 2.7:</p>
<ul>
<li>The caller needs to pass positional and keyword arguments separately, at least when calling a method accepting kwargs.<br>
But, if it calls a methods not accepting kwargs, then the "kwargs" (e.g. <code>foo(key: :value)</code>) should be treated just like a final Hash positional argument.</li>
<li>(If we had complete separation, then we could always pass positional and keyword arguments separately, so the caller could once again ignore the callee)</li>
</ul>
<p>How is the logic implemented in MRI for 2.7?</p>
<p>Specializing the caller for a given callee is a well-known technique.<br>
However, it becomes more difficult if different methods are called from the same callsite (polymorphic call), especially if one accepts kwargs and another does not.<br>
In that case, I think we will see a performance cost to this approach, by having to pass arguments differently based on the method to be called.</p>
<p>What about delegation using <code>ruby2_keywords</code>?<br>
Which checks does that add (compared to 2.6) in the merged approach with the Hash flag?</p> Ruby master - Misc #16160 (Open): Lazy init thread local storagehttps://bugs.ruby-lang.org/issues/161602019-09-09T21:36:28Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>References PR <a href="https://github.com/ruby/ruby/pull/2295" class="external">https://github.com/ruby/ruby/pull/2295</a></p>
<a name="Why"></a>
<h3 >Why?<a href="#Why" class="wiki-anchor">¶</a></h3>
<p>The <code>local_storage</code> member of execution context is lazy initialized and drives the <code>Thread#[]</code> and <code>Thread#[]=</code> APIs, which are Fiber local and not Thread local storage. I think the same lazy init pattern should be applied to the APIs below as well - reduces one <code>Hash</code> alloc per thread created that does not use thread locals.</p>
<a name="Lazy-allocates-thread-local-storage-for-the-following-APIs"></a>
<h3 >Lazy allocates thread local storage for the following APIs<a href="#Lazy-allocates-thread-local-storage-for-the-following-APIs" class="wiki-anchor">¶</a></h3>
<ul>
<li>
<code>Thread#thread_variable_get</code> - early returns <code>nil</code> on locals Hash not initialised</li>
<li>
<code>Thread#thread_variable_set</code> - forces allocation of the locals Hash if not initilalised</li>
<li>
<code>Thread#thread_variables</code> - early returns the empty array AND saves on Hash iteration if locals Hash not initialised</li>
<li>
<code>Thread#thread_variable?</code> - early returns <code>false</code> on locals Hash not initialised</li>
</ul>
<a name="Other-notes"></a>
<h3 >Other notes<a href="#Other-notes" class="wiki-anchor">¶</a></h3>
<ul>
<li>Moved initial implementation from <code>internal.h</code> to <code>thread.c</code> local to call sites.</li>
<li>Preferred <code>defs/id.def</code> for the <code>locals</code> ID (seeing this pattern used more often, but not sure if that is preferred to inline <code>rb_intern</code> yet. Either way there's quite a few different conventions around IDs in the codebase at the moment and happy to help converging to a standard instead.</li>
<li>Maybe a flag is overkill and <code>NIL_P</code> on <code>locals</code> ivar could also work ...</li>
</ul>
<p>Thoughts?</p> Ruby master - Misc #16157 (Open): What is the correct and *portable* way to do generic delegation?https://bugs.ruby-lang.org/issues/161572019-09-09T14:10:42ZDan0042 (Daniel DeLorme)
<p>With the keyword argument changes in 2.7 we must now specify keyword arguments explicitly when doing generic delegation. But this change is not compatible with 2.6, where it adds an empty hash to the argument list of methods that do not need/accept keyword arguments.</p>
<p>To illustrate the problem:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">ProxyWithoutKW</span> <span class="o"><</span> <span class="no">BasicObject</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
<span class="vi">@target</span> <span class="o">=</span> <span class="n">target</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">method_missing</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">&</span><span class="n">b</span><span class="p">)</span>
<span class="vi">@target</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">&</span><span class="n">b</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">ProxyWithKW</span> <span class="o"><</span> <span class="no">BasicObject</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
<span class="vi">@target</span> <span class="o">=</span> <span class="n">target</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">method_missing</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">o</span><span class="p">,</span> <span class="o">&</span><span class="n">b</span><span class="p">)</span>
<span class="vi">@target</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="o">**</span><span class="n">o</span><span class="p">,</span> <span class="o">&</span><span class="n">b</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Test</span>
<span class="k">def</span> <span class="nf">args</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">)</span> <span class="n">a</span> <span class="k">end</span>
<span class="k">def</span> <span class="nf">arg</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="n">a</span> <span class="k">end</span>
<span class="k">def</span> <span class="nf">opts</span><span class="p">(</span><span class="o">**</span><span class="n">o</span><span class="p">)</span> <span class="n">o</span> <span class="k">end</span>
<span class="k">end</span>
<span class="c1"># 2.6 2.7 3.0</span>
<span class="no">ProxyWithoutKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">args</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="c1"># [42] [42] [42] ok</span>
<span class="no">ProxyWithoutKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">arg</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="c1"># 42 42 42 ok</span>
<span class="no">ProxyWithoutKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">opts</span><span class="p">(</span><span class="ss">k: </span><span class="mi">42</span><span class="p">)</span> <span class="c1"># {:k=>42} {:k=>42} +warn [{:k=>42}] incompatible with >= 2.7</span>
<span class="no">ProxyWithKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">args</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="c1"># [42, {}] [42] [42] incompatible with <= 2.6</span>
<span class="no">ProxyWithKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">arg</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="c1"># error 42 42 incompatible with <= 2.6</span>
<span class="no">ProxyWithKW</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Test</span><span class="p">.</span><span class="nf">new</span><span class="p">).</span><span class="nf">opts</span><span class="p">(</span><span class="ss">k: </span><span class="mi">42</span><span class="p">)</span> <span class="c1"># {:k=>42} {:k=>42} +warn {:k=>42} must ignore warning? cannot use pass_positional_hash in 2.6</span>
</code></pre>
<p>I don't know how to solve this, so I'm asking for the <strong>official</strong> correct way to write portable delegation code. And by <strong>portable</strong> I mean code that can be used in gems that target ruby 2.6 and above.</p> Ruby master - Misc #16130 (Open): [Discussion / Ideas] Finding a good name for the concept of/beh...https://bugs.ruby-lang.org/issues/161302019-08-27T11:59:08Zshevegen (Robert A. Heiler)shevegen@gmail.com
<p>In recent presentions this year, and perhaps prior to that as well if I<br>
remember correctly, matz mentioned that the core team (and matz) may be<br>
looking for a good name to the concept of/behind guilds.</p>
<p>The thread here, this issue, is PRIMARILY confined with this, the name -<br>
if you have any good suggestion for names in this context, or discussions<br>
that may relate to this secondarily, please feel free to chime in and<br>
comment. This is primarily meant to give some ideas, possibly.</p>
<p>Since koichi is also heavily involved here (matz pointed out that koichi<br>
likes the name guilds, and came up with the idea/proposal/implementation),<br>
I think this should be considered too. And of course how ruby users may<br>
want to use/view the concept behind guilds, and actually use them in<br>
their own code - the best idea is not great if nobody is using the<br>
concept. Like with refinements ... great idea but I found the API<br>
somewhat strange. :D (May also be because subclassing is so easy with<br>
"Foo < Bar"; ideally we could have something like this with refinements<br>
too, but this is for another proposal or discussion - this here is<br>
about the name for the concept behind guilds.)</p>
<p>Anyway.</p>
<p>I'll give my opinion too, on the names, but I will decouple this from<br>
the initial suggestion here, and reply to my own issue.</p>
<p>Note that this here really is primarily concerned with finding a good<br>
NAME, which is not trivial, since names may have different meanings<br>
in different contexts.</p>
<p>Furthermore, some links for those who may be curious - some of which<br>
are old, and I really just randomly linked these in:</p>
<p><a href="http://www.atdot.net/~ko1/activities/2016_rubykaigi.pdf" class="external">http://www.atdot.net/~ko1/activities/2016_rubykaigi.pdf</a><br>
<a href="https://mensfeld.pl/2016/11/getting-ready-for-new-concurrency-in-ruby-3-with-guilds/" class="external">https://mensfeld.pl/2016/11/getting-ready-for-new-concurrency-in-ruby-3-with-guilds/</a><br>
<a href="https://olivierlacan.com/posts/concurrency-in-ruby-3-with-guilds/" class="external">https://olivierlacan.com/posts/concurrency-in-ruby-3-with-guilds/</a></p> Ruby master - Misc #16124 (Assigned): Let the transient heap belong to objspacehttps://bugs.ruby-lang.org/issues/161242019-08-24T12:41:53Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>As per comment from Nobu in <a href="https://github.com/ruby/ruby/pull/2303#issuecomment-523248875" class="external">https://github.com/ruby/ruby/pull/2303#issuecomment-523248875</a> , I took an initial stab @ a tighter integration between objspace and the transient heap in <a href="https://github.com/ruby/ruby/pull/2400" class="external">https://github.com/ruby/ruby/pull/2400</a></p>
<a name="Benefits"></a>
<h3 >Benefits<a href="#Benefits" class="wiki-anchor">¶</a></h3>
<ul>
<li>Multi-VM (MVM) friendly - ( vm -> objspace -> theap )</li>
<li>The 32MB (current size) arena lazy allocated on ruby init is now properly freed on shutdown as well</li>
<li>It feels strange that the evacuation from the current global theap is to objspace, whereas the space evacuated from is a global arena.</li>
</ul>
<a name="Not-so-great"></a>
<h3 >Not so great<a href="#Not-so-great" class="wiki-anchor">¶</a></h3>
<ul>
<li>A fast reference to a global variable <code>global_transient_heap</code> becomes a function call to <code>rb_objspace_get_theap()</code> and related pointer chasing from vm -> objspace -> theap</li>
<li>Some internal transient heap structs moved to the header file now leaks into all other reference sites where this source file (<code>transient_heap.c</code>) as previously just used for API</li>
<li>I'm not sure exactly of the boundary Koichi had in mind for the GC compile module and how tightly it should (or shouldn't) be coupled to the transient heap. <code>struct rb_objspace*</code> declarations elsewhere for example reveals nothing about the structure members for example, whereas with this PR a lot of transient heap internals are exposed via the header file now</li>
<li>Also possible to move <code>transient_heap.c</code> into <code>gc.c</code> - I feel theap is not an experimental feature anymore and has been stable for quite some time with plausible performance benefits. The downside of that is <code>gc.c</code> is quite dense already, but then all ruby heap management concerns belong to one compile unit.</li>
</ul>
<p>In a similar vein the global method cache could perhaps belong to the VM instance as well, effectively better alignment with MVM and also easier to have a balanced VM setup and teardown sequence without anything left dangling on ruby shutdown.</p>
<p>Thoughts?</p> Ruby master - Misc #16114 (Open): Naming of "beginless range"https://bugs.ruby-lang.org/issues/161142019-08-21T23:34:44Zpwim (Paul McMahon)paul@doorkeeper.jp
<p><a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Startless range (Closed)" href="https://bugs.ruby-lang.org/issues/14799">#14799</a> introduces a "beginless range" to complement the already existing "endless range". However, "beginless" isn't an existing word in English. Since the term for something without a beginning is "beginingless", I'd propose renaming "beginless range" to "beginningless range".</p> Ruby master - Misc #16025 (Assigned): 'st_check_for_sizeof_st_index_t' declared as array with a n...https://bugs.ruby-lang.org/issues/160252019-07-27T05:32:44Zvadimp (Vadim Peretokin)
<p>Compilation of st.h with Emscripten 1.38.30 fails:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="n">st</span><span class="p">.</span><span class="n">h</span><span class="o">:</span><span class="mi">65</span><span class="o">:</span><span class="mi">45</span><span class="o">:</span> <span class="n">error</span><span class="o">:</span> <span class="err">'</span><span class="n">st_check_for_sizeof_st_index_t</span><span class="err">'</span> <span class="n">declared</span> <span class="n">as</span> <span class="n">an</span>
<span class="n">array</span> <span class="n">with</span> <span class="n">a</span> <span class="n">negative</span> <span class="n">size</span>
<span class="k">typedef</span> <span class="kt">char</span> <span class="n">st_check_for_sizeof_st_index_t</span><span class="p">[</span><span class="n">SIZEOF_VOIDP</span> <span class="o">==</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="k">sizeof</span><span class="p">(</span><span class="n">st_index_t</span><span class="p">)</span> <span class="o">?</span> <span class="mi">1</span> <span class="o">:</span> <span class="o">-</span><span class="mi">1</span><span class="p">];</span>
<span class="o">^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span class="mi">3</span><span class="n">rdparty</span><span class="o">/</span><span class="n">edbee</span><span class="o">-</span><span class="n">lib</span><span class="o">/</span><span class="n">vendor</span><span class="o">/</span><span class="n">onig</span><span class="o">/</span><span class="n">config</span><span class="p">.</span><span class="n">h</span><span class="o">:</span><span class="mi">109</span><span class="o">:</span><span class="mi">22</span><span class="o">:</span> <span class="n">note</span><span class="o">:</span> <span class="n">expanded</span> <span class="n">from</span> <span class="n">macro</span> <span class="err">'</span><span class="n">SIZEOF_VOIDP</span><span class="err">'</span>
<span class="cp">#define SIZEOF_VOIDP 8
</span> <span class="o">^</span>
<span class="mi">1</span> <span class="n">error</span> <span class="n">generated</span><span class="p">.</span>
<span class="n">shared</span><span class="o">:</span><span class="n">ERROR</span><span class="o">:</span> <span class="n">compiler</span> <span class="n">frontend</span> <span class="n">failed</span> <span class="n">to</span> <span class="n">generate</span> <span class="n">LLVM</span> <span class="n">bitcode</span><span class="p">,</span> <span class="n">halting</span>
<span class="n">Makefile</span><span class="o">:</span><span class="mi">36871</span><span class="o">:</span> <span class="n">recipe</span> <span class="k">for</span> <span class="n">target</span> <span class="err">'</span><span class="n">regcomp</span><span class="p">.</span><span class="n">o</span><span class="err">'</span> <span class="n">failed</span>
</code></pre>
<p>Both sizeof are set to 8:</p>
<pre><code class="c syntaxhl" data-language="c"><span class="n">onig</span><span class="err">$</span> <span class="n">cat</span> <span class="n">config</span><span class="p">.</span><span class="n">h</span> <span class="o">|</span> <span class="n">grep</span> <span class="n">SIZEOF_LONG</span>
<span class="cp">#define SIZEOF_LONG 8
#define SIZEOF_LONG_LONG 8
</span></code></pre>
<p>Is there a way to fix this issue or add a workaround for emscripten (<code>__EMSCRIPTEN__</code>)?</p> Ruby master - Misc #15806 (Assigned): Explicitly initialise encodings on init to remove branches ...https://bugs.ruby-lang.org/issues/158062019-04-27T23:41:34Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>References Github PR <a href="https://github.com/ruby/ruby/pull/2128" class="external">https://github.com/ruby/ruby/pull/2128</a></p>
<p>I noticed that the encoding table is loaded on startup of even just <code>miniruby</code> (minimal viable interpreter use case) through this backtrace during ruby setup:</p>
<pre><code>/home/lourens/src/ruby/ruby/miniruby(rb_enc_init+0x12) [0x56197b0c0c72] encoding.c:587
/home/lourens/src/ruby/ruby/miniruby(rb_usascii_encoding+0x1a) [0x56197b0c948a] encoding.c:1357
/home/lourens/src/ruby/ruby/miniruby(Init_sym+0x7a) [0x56197b24810a] symbol.c:42
/home/lourens/src/ruby/ruby/miniruby(rb_call_inits+0x1d) [0x56197b11afed] inits.c:25
/home/lourens/src/ruby/ruby/miniruby(ruby_setup+0xf6) [0x56197b0ec9d6] eval.c:74
/home/lourens/src/ruby/ruby/miniruby(ruby_init+0x9) [0x56197b0eca39] eval.c:91
/home/lourens/src/ruby/ruby/miniruby(main+0x5a) [0x56197b051a2a] ./main.c:41
</code></pre>
<p>Therefore I think it makes sense to instead initialize encodings explicitly just prior to symbol init, which is the first entry point into the interpreter loading that currently triggers <code>rb_enc_init</code> and remove the initialization check branches from the various lookup methods.</p>
<p>Some of the branches collapsed, <code>cachegrind</code> output, columns are <code>Ir Bc Bcm Bi Bim</code> with <code>Ir</code> (instructions retired), <code>Bc</code> (branches taken) and <code>Bcm</code> (branches missed) relevant here as there are no indirect branches (function pointers etc.):</p>
<p>(hot function, many instructions retired and branches taken and missed)</p>
<pre><code> . . . . . rb_encoding *
. . . . . rb_enc_from_index(int index)
835,669 0 0 0 0 {
13,133,536 6,337,652 50,267 0 0 if (!enc_table.list) {
3 0 0 0 0 rb_enc_init();
. . . . . }
23,499,349 8,006,202 293,161 0 0 if (index < 0 || enc_table.count <= (index &= ENC_INDEX_MASK)) {
. . . . . return 0;
. . . . . }
30,024,494 0 0 0 0 return enc_table.list[index].enc;
1,671,338 0 0 0 0 }
</code></pre>
<p>(cold function, representative of the utf8 variant more or less too)</p>
<pre><code> . . . . . rb_encoding *
. . . . . rb_ascii8bit_encoding(void)
. . . . . {
27,702 9,235 955 0 0 if (!enc_table.list) {
. . . . . rb_enc_init();
. . . . . }
9,238 0 0 0 0 return enc_table.list[ENCINDEX_ASCII].enc;
9,232 0 0 0 0 }
</code></pre>
<p>I think lazy loading encodings and populating the table is fine, but initializing it can be done more explicitly in the boot process.</p> Ruby master - Misc #15802 (Open): Reduce the minimum string buffer size from 127 to 63 byteshttps://bugs.ruby-lang.org/issues/158022019-04-27T16:40:14Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>References Github PR <a href="https://github.com/ruby/ruby/pull/2151" class="external">https://github.com/ruby/ruby/pull/2151</a> - another small change, but posting here for further discussion.</p>
<p>While having a look through <code>String</code> specific allocation paths with <a href="http://valgrind.org/docs/manual/dh-manual.html#dh-manual.overview" class="external">dhat</a> on redmine I noticed many string buffer use cases actually need way less than the current <code>128</code> byte minimum size (127 current minimum size + sentinel).</p>
<p>These auxiliary buffers are malloc heap specific as the <code>String</code> type is not supported by the transient heap due to complexity.</p>
<a name="How-to-interpret-the-DHAT-output"></a>
<h4 >How to interpret the DHAT output<a href="#How-to-interpret-the-DHAT-output" class="wiki-anchor">¶</a></h4>
<p>From the example output below, we can draw the following conclusions for the specific allocation site leading up to <code>rb_str_buf_new</code>:</p>
<ul>
<li>Total allocated size of <code>434944</code> bytes with a very low read and write access ratio under 25%</li>
<li>The buffer is thus 75% larger than it should be for this particular site (more examples further down below)</li>
<li>Short lived as expected from a buffer use case, but did occupy non-insignificant heap space still for a fair amount of time</li>
<li>The <code>0</code>s at the tail end of the output represent memory never accessed (rows are byte offsets)</li>
<li>As an aside, this particular string's first few characters are <code>hot</code> (accessed more frequently than the tail end)</li>
</ul>
<pre><code>==26579== -------------------- 95 of 300 --------------------
==26579== max-live: 330,368 in 2,581 blocks
==26579== tot-alloc: 434,944 in 3,398 blocks (avg size 128.00)
==26579== deaths: 3,347, at avg age 368,102,626 (2.64% of prog lifetime)
==26579== acc-ratios: 0.22 rd, 0.25 wr (97,275 b-read, 110,341 b-written)
==26579== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==26579== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==26579== by 0x284A9B: rb_str_buf_new (string.c:1331)
==26579== by 0x31B9F7: rb_ary_join (array.c:2331)
==26579== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==26579== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==26579== by 0x2F7C7C: vm_sendish (vm_insnhelper.c:3623)
==26579== by 0x2F7C7C: vm_exec_core (insns.def:789)
==26579== by 0x2EE34D: rb_vm_exec (vm.c:1892)
==26579== by 0x2EEEED: invoke_iseq_block_from_c (vm.c:1104)
==26579== by 0x2EEEED: invoke_block_from_c_bh (vm.c:1122)
==26579== by 0x2EEEED: vm_yield (vm.c:1167)
==26579== by 0x2EEEED: rb_yield_0 (vm_eval.c:980)
==26579== by 0x2EEEED: rb_yield_1 (vm_eval.c:986)
==26579== by 0x2EEEED: rb_yield (vm_eval.c:996)
==26579== by 0x31153B: rb_ary_each (array.c:2087)
==26579== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==26579== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==26579== by 0x2F7D2B: vm_sendish (vm_insnhelper.c:3623)
==26579== by 0x2F7D2B: vm_exec_core (insns.def:771)
==26579== by 0x2EE34D: rb_vm_exec (vm.c:1892)
==26579==
==26579== Aggregated access counts by offset:
==26579==
==26579== [ 0] 11407 8731 9660 11112 11171 11724 13165 13609 10826 10998 11436 11420 11405 10295 9710 9316
==26579== [ 16] 5664 4959 3874 3607 2965 2395 1908 1509 1285 967 597 261 149 141 140 124
==26579== [ 32] 73 57 56 55 55 55 54 52 51 51 51 51 51 51 51 51
==26579== [ 48] 34 34 34 34 34 34 17 0 0 0 0 0 0 0 0 0
==26579== [ 64] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<a name="Why-lower-is-better"></a>
<h4 >Why lower is better<a href="#Why-lower-is-better" class="wiki-anchor">¶</a></h4>
<p>The easiest reproducible case from the benchmark suite is the <code>require</code> benchmark (the allocation site below is from Rails though - DHAT is very slow and doesn't make sense in context of a benchmark):</p>
<pre><code>==28383== -------------------- 572 of 600 --------------------
==28383== max-live: 36,992 in 289 blocks
==28383== tot-alloc: 91,520 in 715 blocks (avg size 128.00)
==28383== deaths: 553, at avg age 908,212,488 (8.68% of prog lifetime)
==28383== acc-ratios: 6.15 rd, 0.41 wr (563,074 b-read, 37,525 b-written)
==28383== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==28383== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==28383== by 0x284A9B: rb_str_buf_new (string.c:1331)
==28383== by 0x290D56: str_gsub (string.c:5163)
==28383== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==28383== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==28383== by 0x2F7C7C: vm_sendish (vm_insnhelper.c:3623)
==28383== by 0x2F7C7C: vm_exec_core (insns.def:789)
==28383== by 0x2EE34D: rb_vm_exec (vm.c:1892)
==28383== by 0x18B336: rb_load_internal0 (load.c:612)
==28383== by 0x18E1B0: rb_require_internal (load.c:1028)
==28383== by 0x18E3B2: rb_require_safe (load.c:1074)
==28383== by 0x18E3B2: rb_f_require (load.c:821)
==28383== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==28383== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==28383== by 0x2F1F42: vm_call_method (vm_insnhelper.c:2712)
==28383==
==28383== Aggregated access counts by offset:
==28383==
==28383== [ 0] 17936 15574 14945 15051 14538 15487 15599 15077 14658 14483 14826 14849 15232 15238 15522 15310
==28383== [ 16] 15055 15020 15143 15156 15271 14807 14656 14271 14051 13761 13365 13156 12756 12530 12302 12002
==28383== [ 32] 7652 7427 7106 6807 6594 6315 6044 5932 5750 5606 5520 5439 5347 5237 5174 5095
==28383== [ 48] 2252 2211 2158 2128 2117 2088 2075 2045 2034 2018 2006 1992 1974 1973 1964 1964
==28383== [ 64] 183 183 183 183 183 183 183 183 183 183 183 183 183 183 183 183
==28383== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28383== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28383== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ /usr/local/bin/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver --executables="compare-ruby::~/src/ruby/trunk/ruby --disable=gems -I.ext/common --disable-gem" --executables="built-ruby::./miniruby -I./lib -I. -I.ext/common -r./prelude --disable-gem" -v --repeat-count=24 -r ips $(ls ./benchmark/*require.{yml,rb} 2>/dev/null)
compare-ruby: ruby 2.7.0dev (2019-04-25 trunk 9bfc185a0d) [x86_64-linux]
built-ruby: ruby 2.7.0dev (2019-04-25 lower-str-buf-.. 9bfc185a0d) [x86_64-linux]
Calculating -------------------------------------
compare-ruby built-ruby
require 1.870 2.408 i/s - 1.000 times in 0.534865s 0.415268s
Comparison:
require
built-ruby: 2.4 i/s
compare-ruby: 1.9 i/s - 1.29x slower
lourens@CarbonX1:~/src/ruby/ruby$ /usr/local/bin/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver --executables="compare-ruby::~/src/ruby/trunk/ruby --disable=gems -I.ext/common --disable-gem" --executables="built-ruby::./miniruby -I./lib -I. -I.ext/common -r./prelude --disable-gem" -v --repeat-count=24 -r memory $(ls ./benchmark/*require.{yml,rb} 2>/dev/null)
compare-ruby: ruby 2.7.0dev (2019-04-25 trunk 9bfc185a0d) [x86_64-linux]
built-ruby: ruby 2.7.0dev (2019-04-25 lower-str-buf-.. 9bfc185a0d) [x86_64-linux]
Calculating -------------------------------------
compare-ruby built-ruby
require 28.128M 26.932M bytes - 1.000 times
Comparison:
require
built-ruby: 26932000.0 bytes
compare-ruby: 28128000.0 bytes - 1.04x larger
</code></pre>
<p>Also the <code>hash_aref_dsym_long</code> benchmark has significant memory reduction:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ /usr/local/bin/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver --executables="compare-ruby::~/src/ruby/trunk/ruby --disable=gems -I.ext/common --disable-gem" --executables="built-ruby::./miniruby -I./lib -I. -I.ext/common -r./prelude --disable-gem" -v --repeat-count=6 -r memory $(ls ./benchmark/hash_aref_dsym_long.{yml,rb} 2>/dev/null)
compare-ruby: ruby 2.7.0dev (2019-04-26 trunk 5689c46457) [x86_64-linux]
built-ruby: ruby 2.7.0dev (2019-04-26 lower-str-buf-.. 9abd605533) [x86_64-linux]
last_commit=Reduce the minimum string buffer size from 127 to 63 bytes
Calculating -------------------------------------
compare-ruby built-ruby
hash_aref_dsym_long 117.580M 104.984M bytes - 1.000 times
Comparison:
hash_aref_dsym_long
built-ruby: 104984000.0 bytes
compare-ruby: 117580000.0 bytes - 1.12x larger
</code></pre>
<a name="Other-allocation-sites-of-note"></a>
<h4 >Other allocation sites of note<a href="#Other-allocation-sites-of-note" class="wiki-anchor">¶</a></h4>
<pre><code>==26579== -------------------- 98 of 300 --------------------
==26579== max-live: 323,456 in 2,527 blocks
==26579== tot-alloc: 402,816 in 3,147 blocks (avg size 128.00)
==26579== deaths: 3,147, at avg age 377,614,197 (2.71% of prog lifetime)
==26579== acc-ratios: 0.21 rd, 0.16 wr (87,620 b-read, 64,622 b-written)
==26579== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==26579== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==26579== by 0x284A9B: rb_str_buf_new (string.c:1331)
==26579== by 0x294584: rb_str_inspect (string.c:5911)
==26579== by 0x2F33F8: vm_call0_cfunc_with_frame (vm_eval.c:86)
==26579== by 0x2F33F8: vm_call0_cfunc (vm_eval.c:100)
==26579== by 0x2F33F8: vm_call0_body.constprop.410 (vm_eval.c:132)
==26579== by 0x2FE4CE: rb_vm_call0 (vm_eval.c:60)
==26579== by 0x2FE4CE: rb_call0 (vm_eval.c:309)
==26579== by 0x2FE4CE: rb_call (vm_eval.c:603)
==26579== by 0x2FE4CE: rb_funcall_with_block (vm_eval.c:857)
==26579== by 0x2EEFBE: vm_yield_with_symbol (vm_insnhelper.c:2869)
==26579== by 0x2EEFBE: invoke_block_from_c_bh (vm.c:1131)
==26579== by 0x2EEFBE: vm_yield (vm.c:1167)
==26579== by 0x2EEFBE: rb_yield_0 (vm_eval.c:980)
==26579== by 0x2EEFBE: rb_yield_1 (vm_eval.c:986)
==26579== by 0x2EEFBE: rb_yield (vm_eval.c:996)
==26579== by 0x317627: rb_ary_collect_bang (array.c:3052)
==26579== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==26579== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==26579== by 0x2F7D2B: vm_sendish (vm_insnhelper.c:3623)
==26579== by 0x2F7D2B: vm_exec_core (insns.def:771)
==26579== by 0x2EE34D: rb_vm_exec (vm.c:1892)
==26579== by 0x2EEEED: invoke_iseq_block_from_c (vm.c:1104)
==26579== by 0x2EEEED: invoke_block_from_c_bh (vm.c:1122)
==26579== by 0x2EEEED: vm_yield (vm.c:1167)
==26579== by 0x2EEEED: rb_yield_0 (vm_eval.c:980)
==26579== by 0x2EEEED: rb_yield_1 (vm_eval.c:986)
==26579== by 0x2EEEED: rb_yield (vm_eval.c:996)
==26579==
==26579== Aggregated access counts by offset:
==26579==
==26579== [ 0] 13063 14338 12631 14007 13323 12720 11984 11344 9232 7280 6288 5776 4256 3856 3584 2976
==26579== [ 16] 1968 1216 1296 720 160 48 48 48 64 16 0 0 0 0 0 0
==26579== [ 32] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 48] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 64] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<pre><code>==26579== -------------------- 186 of 300 --------------------
==26579== max-live: 155,136 in 1,212 blocks
==26579== tot-alloc: 155,136 in 1,212 blocks (avg size 128.00)
==26579== deaths: 651, at avg age 245,150,551 (1.75% of prog lifetime)
==26579== acc-ratios: 1.63 rd, 0.33 wr (253,660 b-read, 51,700 b-written)
==26579== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==26579== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==26579== by 0x284A9B: rb_str_buf_new (string.c:1331)
==26579== by 0x13AA2E: rb_file_join (file.c:4732)
==26579== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==26579== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==26579== by 0x2F7C7C: vm_sendish (vm_insnhelper.c:3623)
==26579== by 0x2F7C7C: vm_exec_core (insns.def:789)
==26579== by 0x2EE34D: rb_vm_exec (vm.c:1892)
==26579== by 0x2EEEED: invoke_iseq_block_from_c (vm.c:1104)
==26579== by 0x2EEEED: invoke_block_from_c_bh (vm.c:1122)
==26579== by 0x2EEEED: vm_yield (vm.c:1167)
==26579== by 0x2EEEED: rb_yield_0 (vm_eval.c:980)
==26579== by 0x2EEEED: rb_yield_1 (vm_eval.c:986)
==26579== by 0x2EEEED: rb_yield (vm_eval.c:996)
==26579== by 0x374412: dir_yield (dir.c:803)
==26579== by 0x374412: dir_each_entry (dir.c:860)
==26579== by 0x374412: dir_each (dir.c:830)
==26579== by 0x1355D2: rb_ensure (eval.c:1076)
==26579== by 0x373447: dir_foreach (dir.c:2955)
==26579== by 0x2E5B4E: vm_call_cfunc_with_frame (vm_insnhelper.c:2207)
==26579== by 0x2E5B4E: vm_call_cfunc (vm_insnhelper.c:2225)
==26579==
==26579== Aggregated access counts by offset:
==26579==
==26579== [ 0] 17075 17890 16143 17188 15855 18234 17616 19067 13136 11474 10379 11350 10094 9420 8556 7614
==26579== [ 16] 5325 5125 5030 4865 3582 2982 3010 3018 2997 3011 3056 3104 2695 2748 2696 2680
==26579== [ 32] 1741 1726 1664 1612 1285 1191 1138 1067 1043 1008 984 983 971 985 972 970
==26579== [ 48] 565 562 568 559 560 557 557 557 557 557 557 557 557 557 557 557
==26579== [ 64] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
==26579== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==26579== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<pre><code>==28716== -------------------- 64 of 300 --------------------
==28716== max-live: 273,280 in 2,135 blocks
==28716== tot-alloc: 446,464 in 3,488 blocks (avg size 128.00)
==28716== deaths: 2,277, at avg age 365,758,007 (3.44% of prog lifetime)
==28716== acc-ratios: 0.21 rd, 0.15 wr (96,380 b-read, 71,208 b-written)
==28716== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==28716== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==28716== by 0x284A9B: rb_str_buf_new (string.c:1331)
==28716== by 0x294584: rb_str_inspect (string.c:5911)
==28716==
==28716== Aggregated access counts by offset:
==28716==
==28716== [ 0] 14382 15824 13909 15391 14767 14067 13241 12472 10180 7932 6842 6270 4714 4213 3909 3280
==28716== [ 16] 2173 1349 1386 810 216 63 54 55 68 20 1 0 0 0 0 0
==28716== [ 32] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 48] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 64] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<pre><code>==28716== -------------------- 276 of 300 --------------------
==28716== max-live: 19,712 in 154 blocks
==28716== tot-alloc: 146,432 in 1,144 blocks (avg size 128.00)
==28716== deaths: 1,115, at avg age 150,688,929 (1.41% of prog lifetime)
==28716== acc-ratios: 0.06 rd, 0.08 wr (9,024 b-read, 12,049 b-written)
==28716== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==28716== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==28716== by 0x284A9B: rb_str_buf_new (string.c:1331)
==28716== by 0x23B70C: rb_reg_regsub (re.c:3820)
==28716==
==28716== Aggregated access counts by offset:
==28716==
==28716== [ 0] 3569 4271 3304 1590 670 640 673 692 537 526 525 518 502 482 465 443
==28716== [ 16] 223 203 182 153 127 113 101 85 71 63 57 52 49 48 44 38
==28716== [ 32] 14 9 7 6 5 4 4 4 3 1 0 0 0 0 0 0
==28716== [ 48] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 64] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==28716== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<pre><code>==29686== -------------------- 391 of 600 --------------------
==29686== max-live: 8,192 in 64 blocks
==29686== tot-alloc: 9,472 in 74 blocks (avg size 128.00)
==29686== deaths: 74, at avg age 207,459,973 (1.96% of prog lifetime)
==29686== acc-ratios: 0.21 rd, 0.17 wr (1,998 b-read, 1,628 b-written)
==29686== at 0x4C2DECF: malloc (in /usr/lib/valgrind/vgpreload_exp-dhat-amd64-linux.so)
==29686== by 0x1521D3: objspace_xmalloc0 (gc.c:9407)
==29686== by 0x284A9B: rb_str_buf_new (string.c:1331)
==29686== by 0x30BB8A: inspect_ary (array.c:2379)
==29686==
==29686== Aggregated access counts by offset:
==29686==
==29686== [ 0] 296 296 296 370 370 370 370 370 296 222 296 74 0 0 0 0
==29686== [ 16] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 32] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 48] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 64] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 80] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 96] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
==29686== [ 112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
</code></pre>
<a name="Rails-specific-redmine-boot"></a>
<h4 >Rails specific - redmine boot<a href="#Rails-specific-redmine-boot" class="wiki-anchor">¶</a></h4>
<p>Booting redmine only, no real work done otherwise - about <code>101720</code> bytes different in current malloc sizes. I understand <code>GC.malloc_allocated_size</code> to reflect the current delta between <code>xmalloc</code> and <code>xfree</code>, but may be wrong. And that value is not representative of total malloc heap churn, judging by the much higher total allocated values coming back from <a href="https://github.com/ruby/ruby/pull/2151#issuecomment-487300456" class="external">valgrind</a>.</p>
<pre><code>lourens@CarbonX1:~/src/redmine$ bundle exec rails c -e production
/home/lourens/src/redmine/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.1.1/lib/active_record/associations/builder/collection_association.rb:26: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
Loading production environment (Rails 5.2.1.1)
irb(main):001:0> RUBY_DESCRIPTION
=> "ruby 2.7.0dev (2019-04-26 trunk 5689c46457) [x86_64-linux]"
irb(main):002:0> GC.start
=> nil
irb(main):003:0> GC.malloc_allocated_size
=> 74907128
</code></pre>
<pre><code>lourens@CarbonX1:~/src/redmine$ bundle exec rails c -e production
/home/lourens/src/redmine/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.1.1/lib/active_record/associations/builder/collection_association.rb:26: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
Loading production environment (Rails 5.2.1.1)
irb(main):001:0> RUBY_DESCRIPTION
=> "ruby 2.7.0dev (2019-04-26 lower-str-buf-.. 9abd605533) [x86_64-linux]"
irb(main):002:0> GC.start
=> nil
irb(main):003:0> GC.malloc_allocated_size
=> 74805408
</code></pre>
<a name="Rails-specific-some-requests-workload"></a>
<h4 >Rails specific - some requests / workload<a href="#Rails-specific-some-requests-workload" class="wiki-anchor">¶</a></h4>
<p>Same sequence of redmine requests - 6 for this branch and trunk respectively using this as a simple initializer for reporting on exit, a <code>228600</code> bytes difference.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">if</span> <span class="no">GC</span><span class="p">.</span><span class="nf">respond_to?</span><span class="p">(</span><span class="ss">:malloc_allocated_size</span><span class="p">)</span>
<span class="nb">at_exit</span> <span class="k">do</span>
<span class="nb">p</span> <span class="s2">"</span><span class="si">#{</span><span class="no">RUBY_DESCRIPTION</span><span class="si">}</span><span class="s2"> GC.malloc_allocated_size: </span><span class="si">#{</span><span class="no">GC</span><span class="p">.</span><span class="nf">malloc_allocated_size</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<pre><code>[2019-04-25 23:42:43] INFO WEBrick 1.4.2
[2019-04-25 23:42:43] INFO ruby 2.7.0 (2019-04-26) [x86_64-linux]
[2019-04-25 23:42:43] INFO WEBrick::HTTPServer#start: pid=31335 port=3000
127.0.0.1 - - [25/Apr/2019:23:42:48 WEST] "GET / HTTP/1.1" 200 4373
http://localhost:3000/news -> /
127.0.0.1 - - [25/Apr/2019:23:42:49 WEST] "GET /projects HTTP/1.1" 200 5470
http://localhost:3000/ -> /projects
127.0.0.1 - - [25/Apr/2019:23:42:51 WEST] "GET /activity HTTP/1.1" 200 7373
http://localhost:3000/projects -> /activity
127.0.0.1 - - [25/Apr/2019:23:42:51 WEST] "GET /issues HTTP/1.1" 200 19195
http://localhost:3000/activity -> /issues
127.0.0.1 - - [25/Apr/2019:23:42:52 WEST] "GET /time_entries HTTP/1.1" 200 12508
http://localhost:3000/issues -> /time_entries
127.0.0.1 - - [25/Apr/2019:23:42:53 WEST] "GET /issues/gantt HTTP/1.1" 200 22597
http://localhost:3000/time_entries -> /issues/gantt
127.0.0.1 - - [25/Apr/2019:23:42:54 WEST] "GET /issues/calendar HTTP/1.1" 200 16712
http://localhost:3000/issues/gantt -> /issues/calendar
127.0.0.1 - - [25/Apr/2019:23:42:54 WEST] "GET /news HTTP/1.1" 200 5472
http://localhost:3000/issues/calendar -> /news
^C[2019-04-25 23:42:58] INFO going to shutdown ...
[2019-04-25 23:42:59] INFO WEBrick::HTTPServer#start done.
Exiting
"ruby 2.7.0dev (2019-04-26 trunk 5689c46457) [x86_64-linux] GC.malloc_allocated_size: 84901440"
</code></pre>
<pre><code>[2019-04-25 23:41:51] INFO WEBrick 1.4.2
[2019-04-25 23:41:51] INFO ruby 2.7.0 (2019-04-26) [x86_64-linux]
[2019-04-25 23:41:51] INFO WEBrick::HTTPServer#start: pid=31029 port=3000
127.0.0.1 - - [25/Apr/2019:23:41:59 WEST] "GET / HTTP/1.1" 200 4373
http://localhost:3000/activity -> /
127.0.0.1 - - [25/Apr/2019:23:42:02 WEST] "GET /projects HTTP/1.1" 200 5470
http://localhost:3000/ -> /projects
127.0.0.1 - - [25/Apr/2019:23:42:04 WEST] "GET /activity HTTP/1.1" 200 7373
http://localhost:3000/projects -> /activity
127.0.0.1 - - [25/Apr/2019:23:42:05 WEST] "GET /issues HTTP/1.1" 200 19195
http://localhost:3000/activity -> /issues
127.0.0.1 - - [25/Apr/2019:23:42:06 WEST] "GET /time_entries HTTP/1.1" 200 12508
http://localhost:3000/issues -> /time_entries
127.0.0.1 - - [25/Apr/2019:23:42:07 WEST] "GET /issues/gantt HTTP/1.1" 200 22597
http://localhost:3000/time_entries -> /issues/gantt
127.0.0.1 - - [25/Apr/2019:23:42:07 WEST] "GET /javascripts/raphael.js?1543965852 HTTP/1.1" 200 90648
http://localhost:3000/issues/gantt -> /javascripts/raphael.js?1543965852
127.0.0.1 - - [25/Apr/2019:23:42:08 WEST] "GET /issues/calendar HTTP/1.1" 200 16712
http://localhost:3000/issues/gantt -> /issues/calendar
127.0.0.1 - - [25/Apr/2019:23:42:09 WEST] "GET /news HTTP/1.1" 200 5472
http://localhost:3000/issues/calendar -> /news
^C[2019-04-25 23:42:14] INFO going to shutdown ...
[2019-04-25 23:42:15] INFO WEBrick::HTTPServer#start done.
Exiting
"ruby 2.7.0dev (2019-04-26 lower-str-buf-.. 9abd605533) [x86_64-linux] GC.malloc_allocated_size: 84672840"
</code></pre> Ruby master - Misc #15744 (Open): Improvement needed to documentation of 'Literals'https://bugs.ruby-lang.org/issues/157442019-04-02T17:34:46ZCaryInVictoria (Cary Swoveland)cary@swoveland.com
<p>Documentation of "Literals" for v2.6.0 is given here: <a href="https://docs.ruby-lang.org/en/2.6.0/syntax/literals_rdoc.html" class="external">https://docs.ruby-lang.org/en/2.6.0/syntax/literals_rdoc.html</a>. (I don't think it has been changed for some time.) It gives examples of literals but does not provide a definition. It is comparable to defining an array by giving a few examples. I believe a definition is needed.</p>
<p>I would like to suggest a definition, but I confess I don't know what a Ruby literal is. A definition is attempted at this Wiki for computer programming generally: <a href="https://en.wikipedia.org/wiki/Literal_(computer_programming)" class="external">https://en.wikipedia.org/wiki/Literal_(computer_programming)</a>.</p>
<p>I suspect a Ruby literal is an object whose value is in some sense "known" at compile-time. For example, I would think <code>1</code>, <code>1.0</code> and <code>{ a: [1, 'cat', ['dog', ['pig', ..10]]] }</code> are literals but <code>{ v=>1, 2=>3 }</code> in <code>h = { v=>1, 2=>3 }</code>, <code>v</code> being a variable, is not. Or is it? If the previous line of code had been <code>v=3</code>, Ruby could, at compile-time, infer that the line could be replaced with <code>h = {3=>1, 2=>3}</code>, in which case it would be "known". This example is meant to illustrate why I earlier said "in some sense".</p> Ruby master - Misc #15654 (Open): Documentation for Complex is wrong or misleadinghttps://bugs.ruby-lang.org/issues/156542019-03-11T08:43:36Zsawa (Tsuyoshi Sawada)
<p>The documentation for <code>Complex</code> <a href="https://ruby-doc.org/core-2.6/Complex.html" class="external">https://ruby-doc.org/core-2.6/Complex.html</a> says or implies that a complex can be created by literal like <code>2+1i</code>, but that is actually calling the method <code>+</code> on receiver <code>2</code> with argument <code>1i</code>. The description should be changed to make it clear that <code>2+ 1i</code> is not a literal but is applying a method.</p> Ruby master - Misc #15568 (Open): TracePoint(:raise)#parameters raises RuntimeErrorhttps://bugs.ruby-lang.org/issues/155682019-01-26T21:10:29Zbaweaver (Brandon Weaver)keystonelemur@gmail.com
<p>Currently trying to get the <code>trace.parameters</code> of a method in a <code>raise</code> event will lead to a RuntimeError. I would contend that it should not, and that it would be perfectly valid to ask for the parameters in the case of an exception.</p>
<p>The reason I do this is to see the arguments at the time of exception:</p>
<pre><code>def extract_args(trace)
trace.parameters.map(&:last).to_h do |name|
[name, trace.binding.eval(name.to_s)]
end
end
</code></pre>
<p>I've noticed that I can technically "cheat" and get these same values like this:</p>
<pre><code>def extract_args(trace)
trace.binding.eval('local_variables').to_h do |name|
[name, trace.binding.eval(name.to_s)]
end
end
</code></pre>
<p>Having the ability to get the parameters in a <code>raise</code> context would be very useful for debugging.</p>
<p>I'm tempted to also suggest <code>TracePoint#local_variables</code> as it would provide additional context in a more exposed way than <code>TracePoint#binding.eval('local_variables')</code></p> Ruby master - Misc #15514 (Open): Add documentation for implicit array decompositionhttps://bugs.ruby-lang.org/issues/155142019-01-07T10:26:51Zsos4nt (Stefan Schüßler)mail@stefanschuessler.de
<p>The documentation for <a href="http://ruby-doc.org/core/doc/syntax/assignment_rdoc.html#label-Array+Decomposition" class="external">Array Decomposition</a> says: <em>"[...] you can decompose an Array during assignment using parenthesis [sic]"</em> and gives an example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="nb">p</span> <span class="ss">a: </span><span class="n">a</span><span class="p">,</span> <span class="ss">b: </span><span class="n">b</span> <span class="c1"># prints {:a=>1, :b=>2}</span>
</code></pre>
<p>But – as we all know – it's also possible <em>without</em> parentheses, i.e.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="nb">p</span> <span class="ss">a: </span><span class="n">a</span> <span class="p">,</span> <span class="ss">b: </span><span class="n">b</span> <span class="c1">#=> {:a=>1, :b=>2}</span>
</code></pre>
<p>This also applies to block arguments when yielding multiple values vs. yielding a single array:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">foo</span>
<span class="k">yield</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bar</span>
<span class="k">yield</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">end</span>
<span class="n">foo</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">|</span> <span class="nb">p</span> <span class="ss">a: </span><span class="n">a</span><span class="p">,</span> <span class="ss">b: </span><span class="n">b</span> <span class="p">}</span>
<span class="c1">#=> {:a=>1, :b=>2}</span>
<span class="n">bar</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">|</span> <span class="nb">p</span> <span class="ss">a: </span><span class="n">a</span><span class="p">,</span> <span class="ss">b: </span><span class="n">b</span> <span class="p">}</span>
<span class="c1">#=> {:a=>1, :b=>2}</span>
</code></pre>
<p>In both cases, parentheses are optional.</p>
<p>This implicit array decomposition could be quite surprising for newcomers. The documentation should cover it.</p> Ruby master - Misc #15510 (Open): Easter egg in Thread.handle_interrupthttps://bugs.ruby-lang.org/issues/155102019-01-05T20:42:12Zlarskanis (Lars Kanis)
<p>The docs of <code>Thread.handle_interrupt</code> are quite clear about the argument to be passed. It must be a hash of <code>ExceptionClass => :TimingSymbol</code> pairs. I never thought that anything other than a <code>Exception</code> derivation would be accepted.</p>
<p>But then I read the tests to this method and wondered: <a href="https://github.com/ruby/ruby/blob/trunk/test/ruby/test_thread.rb#L783" class="external">Some of the tests</a> set <code>Object</code> as <code>ExceptionClass</code>. Moreover the method is not covered by ruby-spec and JRuby <a href="https://github.com/jruby/jruby/blob/66d2905ae233a39e36fcbe3ade6382c2892ade8e/test/mri/excludes/TestThread.rb" class="external">excludes several failing tests</a>.</p>
<p>So I inspected the code and found some obscure behavior: There are actually <a href="https://github.com/ruby/ruby/blob/trunk/thread.c#L115-L116" class="external">two non-exceptions</a> which can be masked by <code>Thread.handle_interrupt(Integer => :TimingSymbol)</code>. It is main thread termination and <code>Thread#kill</code>. Now this <a href="https://github.com/ruby/ruby/blob/trunk/thread.c#L1891-L1894" class="external">blur sentence in the docs</a> makes some more sense:</p>
<blockquote>
<p>interrupt means asynchronous event and corresponding procedure by Thread#raise, Thread#kill, signal trap (not supported yet) and main thread termination (if main thread terminates, then all other thread will be killed).</p>
</blockquote>
<p>So they are implemented as integers internally. However IMHO <code>Thread.handle_interrupt(Integer => :TimingSymbol)</code> is ... ugly.</p>
<p>Some proposals are:</p>
<ol>
<li>Make non-exceptions an ruby implementation specific feature, adjusts current tests and optionally add tests which are tagged as implementation specific.</li>
<li>Document it officially, but choose some more meaningful class names instead of <code>Integer</code>
</li>
</ol> Ruby master - Misc #15487 (Assigned): Clarify default gems maintanance policyhttps://bugs.ruby-lang.org/issues/154872018-12-29T12:30:23Zzverok (Victor Shepelev)zverok.offline@gmail.com
<p>In addition to <a class="issue tracker-5 status-7 priority-4 priority-default closed" title="Misc: Default gems README.md (Feedback)" href="https://bugs.ruby-lang.org/issues/15486">#15486</a>, I'd like to raise the question of the general <em>maintanance policy</em> for "default" Ruby gems, in particular:</p>
<ul>
<li>who is responsible for each gem and how they should be contacted?</li>
<li>what are goals and policies for gems code quality and documentation?</li>
<li>where do default gems are discussed?</li>
<li>what are some promises/guarantees default gems maintainers try to fulfill?</li>
</ul>
<p>The most demonstrative example I'd like to point is <code>json</code> gem:</p>
<ul>
<li>The source at <a href="https://github.com/ruby/json" class="external">ruby/json</a> is NOT authoritative as far as I can tell, the authoritative one is <a href="https://github.com/flori/json" class="external">flori/json</a>
</li>
<li>The gem still holds signs of the times it was independent (<code>Pure</code> and <code>Ext</code> JSON implementations, but <code>Pure</code> is not copied into the <code>ruby/lib</code> on releases, rendering standard docs pretty weird), and has NO mention it is THE json gem of Ruby</li>
<li>The gem seems unmaintained, considering the amount of <a href="https://github.com/flori/json/pulls" class="external">PRs</a> and <a href="https://github.com/flori/json/issues" class="external">issues</a>, lot of them without any reaction for months</li>
<li>When I tried to update JSON docs, in <a href="https://bugs.ruby-lang.org/issues/14581" class="external">core tracker issue</a> I was asked to make a PR to "upstream repository", but there, the PRs (<a href="https://github.com/flori/json/pull/347" class="external">#347</a>, <a href="https://github.com/flori/json/pull/349" class="external">#349</a>) was simply ignored; Ruby 2.6 was released without new docs, despite the fact PRs were made at <strong>March</strong> and require almost no code review (unlike even some promising optimization PRs, that were also hanging there since Feb/Mar)</li>
</ul>
<p>It is just one unfortunate case (TBH, my experience with contributing to other libraries, like <code>csv</code> and <code>psych</code> was much smoother), but it demonstrates some common lack of transparency in maintaining of Ruby's standard library</p> Ruby master - Misc #15431 (Open): Hashes and arrays should not require commas to seperate values ...https://bugs.ruby-lang.org/issues/154312018-12-18T04:14:25Znsuchy (Nathaniel Suchy)me@lunorian.is
<p>Ruby should not require commas for hash and array values if using new lines. I think the syntax would look cleaner without.</p>
<p><strong>Example</strong></p>
<pre><code>myHash = {
:key => “value”,
:anotherKey => “another value”
}
</code></pre>
<p><strong>Could be</strong></p>
<pre><code>myHash = {
:key => “value”
:anotherKey => “another value”
}
</code></pre>
<p><strong>And</strong></p>
<pre><code>myArray = [
1,
2,
3
]
</code></pre>
<p>Could be:</p>
<pre><code>myArray = [
1
2
3
]
</code></pre>
<p>The syntax looks a bit cleaner, with the new lines is there a need to require a comma? I look forward to hearing the community’s thoughts on this idea :)</p> Ruby master - Misc #15418 (Open): Date.parse('2018')https://bugs.ruby-lang.org/issues/154182018-12-15T18:43:56Zfoonlyboy (Eike Dierks)
<p>Date.parse('2018')<br>
ArgumentError: invalid date</p>
<p>I did expect that to return the same as:<br>
Date.parse('2018-1-1')<br>
=> Mon, 01 Jan 2018</p>
<p>working with dates and times is really weird and complicated,<br>
so it makes sense to be strict with parsing.</p>
<p>I watched this one:<br>
<a href="https://www.youtube.com/watch?v=-5wpm-gesOY" class="external">https://www.youtube.com/watch?v=-5wpm-gesOY</a><br>
He's really coming up with some some crazy test cases.</p>
<p>In ruby this is split between Time and DateTime,<br>
some in the core, some in in the standard lib.</p>
<hr>
<p>Im looking forward for the new version,<br>
let's freeze the strings!</p>
<p>~eike</p> Ruby master - Misc #15402 (Open): Shrinking excess retained memory of container types on promotio...https://bugs.ruby-lang.org/issues/154022018-12-11T20:43:38Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>I've been toying with the idea of the viability of attempting to reclaim over provisioned memory from buffer capacity of container objects like <code>Array</code> and <code>String</code>, effectively reducing the footprint of retained memory of such objects.</p>
<p>GC at the moment covers these dominant paths:</p>
<ul>
<li>Collection of shallow memory: unreferenced object slot with values encoded on the object</li>
<li>Collection of retained memory: unreferenced object slot with off ruby object heap pointer to String buffer, Array buffer etc. (<code>heap.aux</code>)</li>
<li>Finalization hooks like reclaiming resources for <code>Tempfile</code> for example</li>
</ul>
<p>I explored in <a href="https://github.com/ruby/ruby/pull/2037" class="external">https://github.com/ruby/ruby/pull/2037</a> (more details and data points on the PR) a forth one:</p>
<ul>
<li>Shrinking over provisioned buffer capacity of <code>Array</code> (also applies to <code>String</code> and likely others) on promotion to uncollectible (<code>Also garbage collect excess retained space from types with a first class capacity and buffer on promotion to uncollectible</code>)</li>
</ul>
<p>Sharing here for feedback in case anyone has ideas for a more appropriate hook, or additional precondition for such a hook. Or if excess buffer capacity can even be considered first class garbage in a GC context.</p>
<p>I chose <code>Array</code> as a proof of concept because the type already have this optimization through <code>ary_shrink_capa</code> through <code>ary_make_shared</code> and the threshold for not encoding members on the object is quite low at 3 elements. Plausible that many framework / boot specific long lived arrays are larger than that and because the growth factor on expansion is <code>2x</code>, also likely a fair amount of over provisioned capacity.</p>
<p>Results of the changeset:</p>
<ul>
<li>Benchmark <code>so_binary_trees</code> - <code>26%</code> reduction in total memory usage</li>
<li>Also a very noticeable <code>24%</code> difference with <code>app_lc_fizzbuzz</code>
</li>
<li>General few bytes reduction for almost all core benchmarks</li>
<li>Mainline <code>redmine</code> after boot - <code>1.5%</code> or <code>45kb</code> reduction in <code>Array</code> retained memory size</li>
</ul>
<p>Implementation caveats:</p>
<ul>
<li>Promotion to uncollectible may be a bad heuristic for shrinking buffer capacity.</li>
<li>Needed to create a new <code>rb_ary_shrink_capa</code> function as <code>ary_shrink_capa</code> is private API and has several assertions (frozen and shared check) that hard fails during GC. That way shrinking responsibility and accounting remains the responsibility of <code>array.c</code> - the GC just calls it (same as with the <code>memsize</code> APIs)</li>
<li>I tried running it during GC which worked fine for benchmarks like <code>so_binary_trees</code> but failed under GC stress and larger heaps because of <code>TRY_WITH_GC</code> via <code>objspace_xrealloc</code>, which can invoke GC</li>
<li>A reasonable workaround for this was to use the postponed job API which is used by GC for object finalization, but that's one job per object space, not 1 per Array being shrinked, which may hit the 1000 item postponed job buffer for some heaps. It degrades gracefully though with fallback being the optimization simply not being applied to the excess objects in the set.</li>
<li>Have no idea about the future of the postponed job API and if this is an appropriate use case</li>
<li>
<code>RVALUE_PAGE_OLD_UNCOLLECTIBLE_SET</code> only special cases <code>Array</code> at the moment - it's easy to support other types</li>
</ul>
<p>Outliers to still evaluate:</p>
<ul>
<li>Fragmentation does not get significantly worse through <code>reallocs</code> for specific rare cases post GC (no data)</li>
<li>The effect of <code>objspace_malloc_increase</code> called by <code>objspace_xrealloc</code> on GC frequency (I think not much given the small reduction on retained usage, but have no data to prove yet)</li>
<li>How well the postponed job pattern scales to large heaps and how much of the job slots are consumed (no data)</li>
</ul>
<p>Thoughts on exploring more types or is the pattern tainted / broken to begin with?</p> Ruby master - Misc #15249 (Open): Documentation for attr_accessor and attr_reader should be corre...https://bugs.ruby-lang.org/issues/152492018-10-23T20:09:02ZCaryInVictoria (Cary Swoveland)cary@swoveland.com
<p>The documentation for <a href="http://ruby-doc.org/core-2.5.1/Module.html#method-i-attr_accessor" class="external">Module#attr_accessor</a> (v2.5.1) begins, "Defines a named attribute for this module, where the name is symbol.id2name, creating an instance variable (@name) and...". Similarly, the documentation for <a href="http://ruby-doc.org/core-2.5.1/Module.html#method-i-attr_reader" class="external">Module#attr_reader</a> states, "Creates instance variables and...". These statements do not appear to be correct:</p>
<pre><code>class C
attr_accessor :dog
attr_reader :cat
end
C.new.instance_variables #=> []
</code></pre> Ruby master - Misc #15202 (Open): Adding Coverity Scan to CI to see the result casuallyhttps://bugs.ruby-lang.org/issues/152022018-10-04T14:06:33Zjaruga (Jun Aruga)
<p>Recently I reported issues detected by code analysis tool mainly using Coverity Scan.</p>
<p>The 9 issues categorized as "important" was fixed by <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Fixing issues detected by an Analysis tool. (Closed)" href="https://bugs.ruby-lang.org/issues/15116">#15116</a>. (Thank you!)</p>
<blockquote>
<p><a href="https://bugs.ruby-lang.org/issues/15116" class="external">https://bugs.ruby-lang.org/issues/15116</a></p>
<p>However as a "not important" issues, around 1000 issues were detected by the tool for the ruby 2.5.1.<br>
I am considering how to deal with this or report those.<br>
I might open an another ticket for that.</p>
</blockquote>
<p>However there are around 1000 "not important" issues.</p>
<p>Right now I do not share the report file (840KByte) for that, because it makes people tired.<br>
If someone want to see it, I am happy to share it here as an attachment.</p>
<p>Instead of that, It looks good to me that someone could see the result of coverity scan casually anytime to fix those in long term.</p>
<p>What I want to propose is to add coverity scan test on rubyci or Travis CI.</p>
<p>I do not know how coverity scan is used on current Ruby project as a regular workflow.<br>
But I could see it is actually used from the setting [2] and some tickets. [3]</p>
<p>I found how to use Coverity Scan on Travis CI [4], and the used cases [5][6].</p>
<p>How do you think?</p>
<ul>
<li>[1] rubyci: <a href="https://www.rubyci.org/" class="external">https://www.rubyci.org/</a>
</li>
<li>[2] coverity scan ruby project: <a href="https://scan.coverity.com/projects/ruby" class="external">https://scan.coverity.com/projects/ruby</a>
</li>
<li>[3] coverity scan used tickets:
<ul>
<li><a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61862" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/61862</a></li>
<li><a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/55763" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/55763</a></li>
<li><a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/50734" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/50734</a></li>
</ul>
</li>
<li>[4] How to use Coverity Scan on Travis CI: <a href="https://scan.coverity.com/travis_ci" class="external">https://scan.coverity.com/travis_ci</a>
</li>
<li>[5] The cases for coverity scan on Travis CI:
<ul>
<li><a href="https://github.com/nanoporetech/scrappie/blob/master/.travis.yml" class="external">https://github.com/nanoporetech/scrappie/blob/master/.travis.yml</a></li>
<li><a href="https://github.com/JanusGraph/janusgraph/blob/master/.travis.yml" class="external">https://github.com/JanusGraph/janusgraph/blob/master/.travis.yml</a></li>
</ul>
</li>
</ul> Ruby master - Misc #15007 (Open): Let all Init_xxx and extension APIs frequently called from init...https://bugs.ruby-lang.org/issues/150072018-08-18T23:17:31Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>References Github PR <a href="https://github.com/ruby/ruby/pull/1934" class="external">https://github.com/ruby/ruby/pull/1934</a></p>
<a name="Why"></a>
<h3 >Why?<a href="#Why" class="wiki-anchor">¶</a></h3>
<p>An incremental extraction from PR <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>, specifically addressing the feedback from Yui Naruse in <a href="https://github.com/ruby/ruby/pull/1922#issuecomment-413796710" class="external">https://github.com/ruby/ruby/pull/1922#issuecomment-413796710</a></p>
<p>The <a href="https://github.com/torvalds/linux/blob/ca04b3cca11acbaf904f707f2d9ca9654d7cc226/include/linux/compiler-gcc.h#L191-L206" class="external">Linux kernel</a>, <a href="https://github.com/php/php-src/blob/2d71a28954a4f20709718ee7cb2b850d334c561c/Zend/zend_portability.h#L220" class="external">PHP 7</a> and other projects use the <code>hot</code> and <code>cold</code> function attributes to help with better code layout.</p>
<p>I noticed Ruby is very much CPU frontend bound (not feeding instructions into the CPU pipelines as fast as it maybe could) and therefore even most micro benchmarks have a high CPI (cycles per instruction) rate. This PR is part of a larger chunk of work I'd like to do around improving CPU frontend throughput and can take a stab at formally writing up those ideas if there's any interest from the community. I don't know.</p>
<a name="Implementation"></a>
<h3 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h3>
<p>This PR has an exclusive focus on having the <code>Init_xxx</code> functions for the core classes and those bundled in <code>ext</code> being flagged to be optimized for size as they're called only once at runtime.</p>
<p>The GCC specific <a href="https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Function-Attributes.html" class="external">cold</a> function attribute works in the following way (from GCC docs):</p>
<pre><code>The cold attribute is used to inform the compiler that a function is unlikely executed. The function is optimized for size rather than speed and on many targets it is placed into special subsection of the text section so all cold functions appears close together improving code locality of non-cold parts of program. The paths leading to call of cold functions within code are marked as unlikely by the branch prediction mechanism. It is thus useful to mark functions used to handle unlikely conditions, such as perror, as cold to improve optimization of hot functions that do call marked functions in rare occasions.
When profile feedback is available, via -fprofile-use, hot functions are automatically detected and this attribute is ignored.
</code></pre>
<p>By declaring a function as <code>cold</code> when defined we get the following benefits:</p>
<ul>
<li>No-op on platforms that does not support the attribute</li>
<li>Size optimization of cold functions with a smaller footprint in the instruction cache</li>
<li>Therefore CPU frontend throughput increases due to a lower ratio of instruction cache misses and a lower ITLB overhead - see <a href="https://user-images.githubusercontent.com/379/44204858-4c085100-a14c-11e8-86b8-d87fcb5e4985.png" class="external">original chunky PR</a> VS <a href="https://user-images.githubusercontent.com/379/44204870-4f9bd800-a14c-11e8-9bee-14c8ad8d3a7d.png" class="external">then trunk</a>
</li>
<li>This effect can further be amplified in future work with the <code>hot</code> attribute</li>
</ul>
<a name="Extension-APIs-flagged-as-cold"></a>
<h4 >Extension APIs flagged as cold<a href="#Extension-APIs-flagged-as-cold" class="wiki-anchor">¶</a></h4>
<p>These are and should typically only be called on extension init, and thus safe to optimize for size as well.</p>
<ul>
<li><code>void rb_define_method_id(VALUE, ID, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_undef(VALUE, ID));</code></li>
<li><code>void rb_define_protected_method(VALUE, const char*, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_define_private_method(VALUE, const char*, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_define_singleton_method(VALUE, const char*, VALUE(*)(ANYARGS), int));</code></li>
<li><code>void rb_define_alloc_func(VALUE, rb_alloc_func_t));</code></li>
<li><code>void rb_undef_alloc_func(VALUE));</code></li>
<li><code>VALUE rb_define_class(const char*,VALUE));</code></li>
<li><code>VALUE rb_define_module(const char*));</code></li>
<li><code>VALUE rb_define_class_under(VALUE, const char*, VALUE));</code></li>
<li><code>VALUE rb_define_module_under(VALUE, const char*));</code></li>
<li><code>void rb_define_variable(const char*,VALUE*));</code></li>
<li><code>void rb_define_virtual_variable(const char*,VALUE(*)(ANYARGS),void(*)(ANYARGS)));</code></li>
<li><code>void rb_define_hooked_variable(const char*,VALUE*,VALUE(*)(ANYARGS),void(*)(ANYARGS)));</code></li>
<li><code>void rb_define_readonly_variable(const char*,const VALUE*));</code></li>
<li><code>void rb_define_const(VALUE,const char*,VALUE));</code></li>
<li><code>void rb_define_global_const(const char*,VALUE));</code></li>
<li><code>void rb_define_method(VALUE,const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>(void rb_define_module_function(VALUE,const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>void rb_define_global_function(const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>void rb_undef_method(VALUE,const char*));</code></li>
<li><code>void rb_define_alias(VALUE,const char*,const char*));</code></li>
<li><code>void rb_define_attr(VALUE,const char*,int,int));</code></li>
<li><code>void rb_global_variable(VALUE*));</code></li>
<li><code>void rb_gc_register_mark_object(VALUE));</code></li>
<li><code>void rb_gc_register_address(VALUE*));</code></li>
<li><code>void rb_gc_unregister_address(VALUE*));</code></li>
</ul>
<a name="Text-segment-reductions"></a>
<h4 >Text segment reductions<a href="#Text-segment-reductions" class="wiki-anchor">¶</a></h4>
<p>Small changes (<code>3144</code> bytes reduction of the text segment) because this is incremental groundwork and and initial low risk PR.</p>
<p>this branch:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ size ruby
text data bss dec hex filename
3462153 21056 71344 3554553 363cf9 ruby
</code></pre>
<p>trunk:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/trunk$ size ruby
text data bss dec hex filename
3465297 21056 71344 3557697 364941 ruby
</code></pre>
<p>Diffs for individual object files: <a href="https://www.diffchecker.com/T0GVzX1q" class="external">https://www.diffchecker.com/T0GVzX1q</a></p>
<p>Default <code>text.unlikely</code> section where init functions are moved to:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ readelf -S vm.o
There are 34 section headers, starting at offset 0x2a04f8:
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .text PROGBITS 0000000000000000 00000040
000000000001c37f 0000000000000000 AX 0 0 16
[ 2] .rela.text RELA 0000000000000000 00114100
000000000000a7d0 0000000000000018 I 31 1 8
[ 3] .data PROGBITS 0000000000000000 0001c3c0
0000000000000030 0000000000000000 WA 0 0 16
[ 4] .bss NOBITS 0000000000000000 0001c400
00000000000002b0 0000000000000000 WA 0 0 32
[ 5] .rodata.str1.8 PROGBITS 0000000000000000 0001c400
0000000000000d6f 0000000000000001 AMS 0 0 8
[ 6] .text.unlikely PROGBITS 0000000000000000 0001d16f <<<<<<<<<<<<<<<
0000000000001aa9 0000000000000000 AX 0 0 1
</code></pre>
<p>The relocations for <code>vm.o</code>:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ ld -M vm.o
--- truncated ---
.text 0x0000000000400120 0x1de2f
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
.text.unlikely
0x0000000000400120 0x1aa9 vm.o
0x000000000040038f rb_define_alloc_func
0x00000000004003bf rb_undef_alloc_func
0x00000000004003c5 Init_Method
0x0000000000400512 Init_vm_eval
0x00000000004007a1 Init_eval_method
0x0000000000400a54 rb_undef
0x0000000000400c1d Init_VM
0x000000000040185f Init_BareVM
0x0000000000401b16 Init_vm_objects
0x0000000000401b61 Init_top_self
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
*fill* 0x0000000000401bc9 0x7
.text 0x0000000000401bd0 0x1c37f vm.o
0x00000000004022f0 rb_f_notimplement
0x0000000000404780 rb_vm_ep_local_ep
0x00000000004047b0 rb_vm_frame_block_handler
0x00000000004047e0 rb_vm_cref_new_toplevel
0x0000000000404870 rb_vm_block_ep_update
0x0000000000404890 ruby_vm_special_exception_copy
0x0000000000406960 rb_ec_stack_overflow
0x00000000004069c0 rb_vm_push_frame
0x0000000000406b20 rb_vm_pop_frame
0x0000000000406b30 rb_error_arity
0x0000000000407180 rb_vm_frame_method_entry
0x00000000004075e0 rb_vm_rewrite_cref
0x00000000004076f0 rb_simple_iseq_p
0x0000000000407700 rb_vm_opt_struct_aref
0x0000000000407730 rb_vm_opt_struct_aset
0x0000000000407750 rb_clear_constant_cache
--- truncated ---
</code></pre>
<p>I also dabbled with the idea of an <code>INITFUNC</code> macro that also places the <code>Init_xxx</code> functions into a <code>text.init</code> section as the <a href="https://linuxgazette.net/157/amurray.html" class="external">kernel does</a> for a possible future optimization of stripping out ELF sections for setup / init specific functions. I don't think that makes sense for now and possibly only interesting for mruby or embedded.</p>
<a name="Possible-next-units-of-work"></a>
<h3 >Possible next units of work<a href="#Possible-next-units-of-work" class="wiki-anchor">¶</a></h3>
<a name="Cold-code-specific"></a>
<h4 >Cold code specific<a href="#Cold-code-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>Incrementally PR corner case error handling functions such as <code>rb_bug</code> from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>Ditto for generic error handling functions (<code>rb_raise</code> and friends) from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>Class specific error handling functions (load errors, encoding errors in the IO module, sys errors etc.) from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>GCC 5+ also supports <code>cold</code> <a href="https://gcc.gnu.org/onlinedocs/gcc/Label-Attributes.html" class="external">labels</a> , which I took a stab with in the bloated <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
</ul>
<a name="TLB-translation-lookaside-buffer-specific"></a>
<h4 >TLB (translation lookaside buffer) specific<a href="#TLB-translation-lookaside-buffer-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>Further ITLB overhead investigation</li>
<li>Ruby binaries built with O3 and debug symbols come in at just short of 18MB, or roughly 9 hugepages on linux. PHP core developers were able to squeeze a few % by remapping code to hugepages on supported systems - <a href="http://developers-club.com/posts/270685/" class="external">http://developers-club.com/posts/270685/</a> . Implementation <a href="https://github.com/php/php-src/blob/fb0389b1010de5a6459bcf286409423f69e74aaf/ext/opcache/ZendAccelerator.c#L2645-L2750" class="external">here</a>
</li>
</ul>
<a name="Bytecode-specific"></a>
<h4 >Bytecode specific<a href="#Bytecode-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>The <a href="https://software.intel.com/en-us/vtune-amplifier-help-task-api" class="external">Intel Tracing Task API</a> is very well suited for the instruction sequences YARV generates and to infer better per instruction CPU utilization and identify any stalls (frontend, backend, branches etc.) to drive further work.</li>
</ul> Ruby master - Misc #12911 (Open): Translate docshttps://bugs.ruby-lang.org/issues/129112016-11-08T16:04:00Zsho-h (Sho Hashimoto)sho-h@netlab.jp
<p>translate doc/*.ja.rdoc</p>
<p>c.f. <a href="https://blade.ruby-lang.org/ruby-dev/47319">[ruby-dev:47319]</a></p> Ruby master - Misc #11783 (Open): Do you have any idea if you have a budgets?https://bugs.ruby-lang.org/issues/117832015-12-07T10:29:55Zko1 (Koichi Sasada)
<p>Do you have any idea about Ruby interpreter implementation to do with budgets?</p>
<a name="Background"></a>
<h1 >Background<a href="#Background" class="wiki-anchor">¶</a></h1>
<p>Now, we are summarizing many contributions from many people, organizations and companies.</p>
<p><a href="https://docs.google.com/document/d/1y1sQc40qeuWjF84rVrTmH-ogZ_iNGqU-RUSE8GDlRuk/edit?usp=sharing" class="external">https://docs.google.com/document/d/1y1sQc40qeuWjF84rVrTmH-ogZ_iNGqU-RUSE8GDlRuk/edit?usp=sharing</a></p>
<p>(please let me know if you know any other contributes)<br>
(sorry we wrote contributions especially for MRI, because we don't know)</p>
<p>The great recent news is we get new mac mini machine to run CI on El Capitan. We already have a mac mini machine running CI, but on Yosemite. So we can run CI on both Yosemite and El Capitan.</p>
<p>This new mac mini machine was sponsored by YassLab, Japanese small company.</p>
<p>At first, we ask Nihon-Ruby-no-Kai to prepare this machine, and Takahashi-san (chair man of this organization) tweet about it ("anyone can support it?"). Yasukawa-san, the president of YassLab answers "ok, we'll support it".</p>
<p>We learned that if we show requirements explicitly, anyone may help us.<br>
Listing is important.</p>
<a name="Any-idea"></a>
<h1 >Any idea?<a href="#Any-idea" class="wiki-anchor">¶</a></h1>
<p>Today's developers meeting, we had discussed about that and itemize some dreams.</p>
<blockquote>
<p>nurse: VPS severs for CI are welcome. Especially for Azure.<br>
ko1: travel fee (1,000,000 JPY?) for hackathon to gather MRI developers in one place<br>
ko1: physical machines for development and benchmarks (300,000 JPY)<br>
nobu: development machine (400,000 JPY) because he has several trouble on current machine.<br>
nurse: icc (and other softwares) to try.<br>
martin: grant project for MRI development topics<br>
ko1: education to grow other MRI developer (no estimation)</p>
</blockquote>
<p>Do you have any other idea?<br>
I'll show these list at RubyKaigi, and someone may consider to support us.</p>
<p>(IMO, maybe sponsoring nobu's machine is great contribution for Ruby worlds.<br>
Nobu will put companies logo stickers on his laptop)</p>
<p>Thanks,<br>
Koichi</p> Ruby master - Misc #10791 (Open): [PATCH 1/1] Remove unnecessary passing value from doc for Obser...https://bugs.ruby-lang.org/issues/107912015-01-27T19:43:48Zgogotanaka (Kazuki Tanaka)mail@tanakakazuki.com
<p>Hi, when reading doc for Observable, I notice little unnecessary code which may cause little confusion in example.</p>
<p>take your time.</p>
<p>gogo.</p>