Ruby Issue Tracking System: Issueshttps://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112022-10-02T14:18:03ZRuby Issue Tracking System
Redmine Ruby master - Feature #19036 (Closed): Provide a way to set path for File instances created with ...https://bugs.ruby-lang.org/issues/190362022-10-02T14:18:03Zheadius (Charles Nutter)headius@headius.com
<p>Ruby provides <code>IO.for_fd</code> to instantiate an IO object from an existing file descriptor value. The logic for this simply calls the base <code>IO.new</code> logic, which for all IO and subtypes simply wraps the given file descriptor.</p>
<p>When called against File, or other subtypes of IO, this has the side effect of creating an IO instance with that type, e.g. <code>File.for_fd</code> will behave identically to <code>IO.for_fd</code> except that the class of the resulting object will be File.</p>
<p>Unfortunately, this results in a File object that does not have any <code>path</code> associated with it:</p>
<pre><code>3.1.2 :001 > f = File.open('README.md')
=> #<File:README.md>
3.1.2 :002 > f.path
=> "README.md"
3.1.2 :003 > f2 = File.for_fd(f.fileno)
=> #<File:fd 5>
3.1.2 :004 > f2.path
(irb):4:in `path': File is unnamed (TMPFILE?) (IOError)
from (irb):4:in `<main>'
from /home/headius/.rvm/rubies/ruby-3.1.2/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `load'
from /home/headius/.rvm/rubies/ruby-3.1.2/bin/irb:25:in `<main>'
</code></pre>
<p>I propose that there should be a way, via an extra parameter or a keyword argument, to provide a path when constructing a new File via <code>for_fd</code>.</p>
<p>Possible forms:</p>
<ul>
<li><code>File.for_fd(fileno, "my/path")</code></li>
<li><code>File.for_fd(fileno, path: "my/path")</code></li>
</ul>
<p>This would necessitate a separate implementation for <code>File.for_fd</code> unless we want to make it possible to set a path for all <code>for_fd</code> calls (which may not make sense for many of them).</p>
<p>This came up while trying to implement a pure-Ruby (plus FFI) version of the "pty" library. Without overriding the <code>path</code> function, it is not possible for the File object returned by <code>PTY.open</code> to gain the "masterpty:" filename, and therefore it does not clearly indicate it is from a PTY.</p>
<p>See <a href="https://github.com/jruby/jruby/pull/7391" class="external">https://github.com/jruby/jruby/pull/7391</a>, an attempt to match inspect output for these return values using <code>define_singleton_method</code>. Providing a way to set the path would make this automatic without the singleton definition.</p> Ruby master - Feature #18568 (Feedback): Explore lazy RubyGems boot to reduce need for --disable-...https://bugs.ruby-lang.org/issues/185682022-02-02T19:13:52Zheadius (Charles Nutter)headius@headius.com
<p>In <a href="https://bugs.ruby-lang.org/issues/17684" class="external">https://bugs.ruby-lang.org/issues/17684</a> there was debate about whether the <code>--disable-gems</code> flag should be removed. Several folks were in favor, since Ruby without RubyGems is fairly limited, but others wanted to keep the flag for small, fast command line scripts that do not depend on RubyGems.</p>
<p>Lazily loading RubyGems might be a middle ground, and it has been explored in some depth by TruffleRuby:</p>
<p><a href="https://github.com/oracle/truffleruby/blob/master/src/main/ruby/truffleruby/core/lazy_rubygems.rb" class="external">https://github.com/oracle/truffleruby/blob/master/src/main/ruby/truffleruby/core/lazy_rubygems.rb</a></p>
<p><a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> shows how this improves their startup time in this article from a couple years ago:</p>
<p><a href="https://eregon.me/blog/2019/04/24/how-truffleruby-startup-became-faster-than-mri.html" class="external">https://eregon.me/blog/2019/04/24/how-truffleruby-startup-became-faster-than-mri.html</a></p>
<p>I believe this approach has merit and could be beneficial to both CRuby and JRuby if we can collaborate on how the lazy loading should happen and figuring out where the edges are. <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> may know some of those edges if they have run into them in TruffleRuby.</p>
<p>A simple test of <code>--disable-gems</code> on CRuby 3.1 shows what an impact it has, which we might be able to duplicate in a lazy boot WITHOUT losing RubyGems functionality and default gem upgrading:</p>
<pre><code>$ time ruby -e 1
real 0m0.107s
user 0m0.068s
sys 0m0.030s
$ time ruby --disable-gems -e 1
real 0m0.019s
user 0m0.007s
sys 0m0.008s
</code></pre>
<p>Over 80% of CRuby's base startup is due to eagerly booting RubyGems. We can do better!</p> Ruby master - Bug #18561 (Closed): Make singleton def operation and define_singleton_method expli...https://bugs.ruby-lang.org/issues/185612022-01-31T19:09:51Zheadius (Charles Nutter)headius@headius.com
<p>Currently nearly all uses of <code>define_singleton_method</code> or <code>def obj.foo</code> will ignore the caller's frame/cref visibility and use the default visibility of public. I propose this be made explicit in the code, documentation, and ruby specs.</p>
<pre><code>$ ruby -e 'class Foo; private; define_singleton_method(:foo) {p :ok}; end; Foo.foo'
:ok
$ ruby -e 'class Foo; private; def self.foo; p :ok; end; end; Foo.foo'
:ok
</code></pre>
<p>This works because the class in which the method is defined is nearly always different from the calling scope, since we are usually calling <code>define_singleton_method</code> against some other object. It "accidentally" ends up being public all the time, like <code>def self.foo</code>.</p>
<p>However, there is at least one (weird) edge case where the frame/cref visibility is honored:</p>
<pre><code>$ ruby -e '$o = Object.new; class << $o; private; $o.define_singleton_method(:foo){}; end; $o.foo'
-e:1:in `<main>': private method `foo' called for #<Object:0x00007fcf0e00dc98> (NoMethodError)
</code></pre>
<p>This also works for <code>def $o.foo</code> but I would argue this is unexpected behavior in both cases. It is difficult to trigger, since you have to already be within the target singleton class body, and the "normal" behavior everywhere else is to ignore the frame/cref visibility.</p>
<p>It would not be difficult to make both forms always use public visibility:</p>
<ul>
<li>Split off the actual method-binding logic from <code>rb_mod_define_method</code> into a separate function <code>mod_define_method_internal</code> that takes a visibility parameter.</li>
<li>Call that new method from <code>rb_mod_define_method</code> (with cref-based visibility calculation) and <code>rb_obj_define_method</code> (with explicit public visibility).</li>
</ul> Ruby master - Feature #18554 (Open): Move unicode_normalize to a default gemhttps://bugs.ruby-lang.org/issues/185542022-01-27T16:49:48Zheadius (Charles Nutter)headius@headius.com
<p>Could we move the rest of unicode_normalize to a default gem?</p>
<p>The recent updates were mostly updating the Unicode tables, which a user might want to be able to update in an existing Ruby installation. Additionally, this is one of the few stdlib we have to copy into JRuby from the CRuby repository; it would be easier for both if we just pulled in a default gem.</p> Ruby master - Bug #18169 (Closed): Local copies of gemified libraries are being released out of s...https://bugs.ruby-lang.org/issues/181692021-09-15T19:41:03Zheadius (Charles Nutter)headius@headius.com
<p>The CRuby codebase includes a number of libraries that have been gemified, more and more with each release. Unfortunately, these libraries are continually out of sync with both their home repositories and their released gems.</p>
<p>The problem lies in the fact that CRuby keeps a local copy of these libraries within the CRuby git repository, and allows committers to make changes either in the CRuby repository or in the gem's home repository. This has led to many releases of Ruby shipping code that <strong>does not correspond to any released version of the related gem</strong>.</p>
<p>I have filed several issues about this but the root cause has not been addressed:</p>
<ul>
<li><a href="https://github.com/ruby/ostruct/issues/11" class="external">https://github.com/ruby/ostruct/issues/11</a></li>
<li><a href="https://github.com/ruby/matrix/issues/12" class="external">https://github.com/ruby/matrix/issues/12</a></li>
<li><a href="https://github.com/ruby/prime/issues/11" class="external">https://github.com/ruby/prime/issues/11</a></li>
<li><a href="https://github.com/ruby/webrick/issues/48" class="external">https://github.com/ruby/webrick/issues/48</a></li>
<li><a href="https://github.com/ruby/rdoc/issues/835" class="external">https://github.com/ruby/rdoc/issues/835</a></li>
<li><a href="https://github.com/ruby/rexml/issues/79" class="external">https://github.com/ruby/rexml/issues/79</a></li>
<li><a href="https://github.com/ruby/fileutils/issues/59" class="external">https://github.com/ruby/fileutils/issues/59</a></li>
<li><a href="https://github.com/ruby/ostruct/issues/31" class="external">https://github.com/ruby/ostruct/issues/31</a></li>
</ul>
<p>If these gems are to live on their own as standalone libraries/gems then one of two things must happen:</p>
<ul>
<li>All changes to them must go into their repositories. This would be the cleanest option. CRuby would, like JRuby, source these libraries directly from released gems, and no copies of their sources would be versioned in the CRuby git repository.</li>
</ul>
<p>OR</p>
<ul>
<li>CRuby-local changes to these libraries must be prohibited from being released unless there is a corresponding gem release. This would require automated or manual auditing at release time, to ensure that the claimed gem version actually corresponds to the sources being shipped.</li>
</ul>
<p>In addition to making it impossible for external users of these gems to match CRuby releases, there are more serious implications:</p>
<ul>
<li>These hybrid intra-version changes to these libraries cannot be audited to a specific gem release. This could affect stability and security when users attempt to sync their local gem sets to the ones that shipped in a given version of Ruby.</li>
<li>Security fixes have gone out in CRuby releases but no corresponding x.x.y or x.x.x.y security release of the gem was released. This prevents users from fixing the security issue locally without either upgrading CRuby or also including new functionality changes (which may or may not work on the current version of Ruby).</li>
</ul>
<p>See the rexml issue above for one example of the security problem. Until the gem was released, it was not possible to install any gem version with the security fix without upgrading functionality elsewhere in rexml.</p>
<p>I believe it is time for CRuby to stop making changes to gemified libraries directly in the CRuby repository. These libraries have their own gems, repositories, and issue trackers, and that is where they should be maintained.</p> Ruby master - Feature #17771 (Open): String#start_with? should not construct MatchData or set $~https://bugs.ruby-lang.org/issues/177712021-04-01T18:08:02Zheadius (Charles Nutter)headius@headius.com
<p>I am working on making $~ more thread-safe in JRuby and came across this unexpected behavior:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="err">$</span> <span class="n">rvm</span> <span class="n">ruby</span><span class="o">-</span><span class="mf">3.0</span> <span class="k">do</span> <span class="n">ruby</span> <span class="o">-</span><span class="n">e</span> <span class="s1">'"foo".start_with?(/foo/); p $~'</span>
<span class="c1">#<MatchData "foo"></span>
</code></pre>
<p>The <code>start_with?</code> method was added 11 years ago in <a href="https://bugs.ruby-lang.org/issues/3388" class="external">https://bugs.ruby-lang.org/issues/3388</a> but I do not think the set of $~ was an intended feature. The <code>start_with?</code> method could be much faster and more thread-safe if it did not use the frame-local backref slot and did not allocate a MatchData.</p>
<p>Compare with <code>match?</code> which was added specifically (without MatchData or backref setting) to provide a fast way to check if a Regexp matches.</p>
<p>I propose that <code>start_with?</code> stop constructing MatchData, stop setting backref, and provide only its boolean result in the same way as <code>match?</code>.</p> Ruby master - Bug #16959 (Rejected): Weakmap has specs and third-party usage despite being a priv...https://bugs.ruby-lang.org/issues/169592020-06-12T22:09:15Zheadius (Charles Nutter)headius@headius.com
<p>Weakmap is still described as an internal API, and the documentation points users at WeakRef as the official public API:</p>
<p><a href="https://github.com/ruby/ruby/blob/1fb16dbb6e28b9f32f92554d29e646e088b21a98/gc.c#L11928-L11936" class="external">https://github.com/ruby/ruby/blob/1fb16dbb6e28b9f32f92554d29e646e088b21a98/gc.c#L11928-L11936</a></p>
<p>However there are now specs for its current set of features, even though those features have never been discussed or approved as a public API:</p>
<p><a href="https://github.com/ruby/spec/tree/dd8437628a6f2de5b74b338d4960682bb1590a60/core/objectspace/weakmap" class="external">https://github.com/ruby/spec/tree/dd8437628a6f2de5b74b338d4960682bb1590a60/core/objectspace/weakmap</a></p>
<p>And we are starting to see it being used by the community:</p>
<ul>
<li><a href="https://github.com/jruby/jruby/issues/6267" class="external">https://github.com/jruby/jruby/issues/6267</a></li>
<li><a href="https://github.com/rsim/oracle-enhanced/issues/2027" class="external">https://github.com/rsim/oracle-enhanced/issues/2027</a></li>
<li><a href="https://github.com/rails/rails/pull/39121" class="external">https://github.com/rails/rails/pull/39121</a></li>
</ul>
<p>One of two things needs to happen:</p>
<ul>
<li>Weakmap is made a public API after some discussion. It would be an official public feature only in 2.8/3.0 or higher.</li>
<li><del>The specs are be removed and Weakmap remains a private API not to be used by the community. I suspect the addition of the specs led to folks starting to use this private API.</del></li>
</ul>
<p>(edit: The Rails PR was merged after the specs, but the change is actually a year old, as mentioned below. In any case there's plenty of in-the-wild uses of WeakMap that go back even further.)</p>
<p>Personally, I'm in much more in favor of making WeakRef support all the features necessary to implement Weakmap in pure Ruby, rather than the other way around:</p>
<p><a href="https://bugs.ruby-lang.org/issues/6309" class="external">https://bugs.ruby-lang.org/issues/6309</a></p>
<p>But whatever happens it needs to happen soon, since this use case is now a merged feature in Rails master.</p> Ruby master - Bug #16918 (Closed): Dir.mktmpdir should yield a copy of the dir to protect cleanuphttps://bugs.ruby-lang.org/issues/169182020-05-29T01:14:26Zheadius (Charles Nutter)headius@headius.com
<p>If you modify the dir string passed into the block from <code>Dir.mktmpdir</code>, the logic to clean up the temporary directory may fail:</p>
<pre><code>$ rvm ruby-2.6.5 do ruby -rtmpdir -e "Dir.mktmpdir('foo') {|dir| dir << 'bar'}"
Traceback (most recent call last):
9: from -e:1:in `<main>'
8: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/tmpdir.rb:101:in `mktmpdir'
7: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:758:in `remove_entry'
6: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1480:in `postorder_traverse'
5: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:760:in `block in remove_entry'
4: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1425:in `remove'
3: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1436:in `remove_file'
2: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1442:in `platform_support'
1: from /Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1437:in `block in remove_file'
/Users/headius/.rvm/rubies/ruby-2.6.5/lib/ruby/2.6.0/fileutils.rb:1437:in `unlink': No such file or directory @ apply2files - /var/folders/cq/ylcgmnn556x33f5hsqd0h54h0000gn/T/foo20200528-99594-tuq6pubar (Errno::ENOENT)
</code></pre>
<p>I believe <code>Dir.mktmpdir</code> should protect its cleanup logic by yielding a copy of the dir string, rather than the exact string object it intends to use for cleanup.</p> Ruby master - Feature #16816 (Open): Prematurely terminated Enumerator should stay terminatedhttps://bugs.ruby-lang.org/issues/168162020-04-24T20:34:07Zheadius (Charles Nutter)headius@headius.com
<p>When iterating over an Enumerator, there are three different possible results of calling <code>next</code>:</p>
<ol>
<li>The next item is returned and a cursor is advanced</li>
<li>There's no next item and the Enumerator will forever raise <code>StopIteration</code>
</li>
<li>There's an error getting the next item which is raised out of <code>next</code>
</li>
</ol>
<p>This third case has some unexpected behavior that I discovered while working on <a href="https://github.com/jruby/jruby/issues/6157" class="external">https://github.com/jruby/jruby/issues/6157</a></p>
<p>It seems that when an Enumerator fails prematurely with an exception, any subsequent call to #next will cause it to restart.</p>
<p>This can be seen in a simple script I used to write a ruby/spec in <a href="https://github.com/jruby/jruby/pull/6190" class="external">https://github.com/jruby/jruby/pull/6190</a></p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Enumerator</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="mi">2</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span><span class="o">|</span><span class="n">i</span><span class="o">|</span> <span class="k">raise</span> <span class="n">i</span><span class="p">.</span><span class="nf">to_s</span> <span class="p">}</span>
<span class="p">}.</span><span class="nf">tap</span> <span class="p">{</span><span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="nb">p</span> <span class="mi">2</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="n">f</span><span class="p">.</span><span class="nf">next</span> <span class="k">rescue</span> <span class="vg">$!</span><span class="p">.</span><span class="nf">message</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre>
<p>The output from this is <code>[0, 0]</code>. After the iteration fails, the second <code>next</code> call causes it to restart and it fails again.</p>
<p>Contrast this to the behavior of item 3 above; when an Enumerator finishes iterating without error, it remains "finished" forever and can't be restarted.</p>
<p>I believe the restarting behavior is at best undocumented behavior and at worst incorrect and unspected. Take this example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">e</span> <span class="o">=</span> <span class="no">Enumerator</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">y</span><span class="o">|</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">new_database_cursor</span>
<span class="mi">5</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">y</span><span class="p">.</span><span class="nf">yield</span> <span class="n">c</span><span class="p">.</span><span class="nf">next_result</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre>
<p>If <code>next_result</code> here raises an error, a subsequent call to <code>next</code> on this enumerator will cause it to restart, re-acquire the cursor, and begin again.</p>
<p>As another example I ask a question: how do you indicate that an Enumerator failed due to an error, and <em>keep it failed</em> so it doesn't restart again?</p> Ruby master - Feature #16744 (Third Party's Issue): Flag to load current bundle without using bun...https://bugs.ruby-lang.org/issues/167442020-03-30T18:45:03Zheadius (Charles Nutter)headius@headius.com
<p>The <code>bundle exec</code> command is used by Ruby users primarily to start up a Ruby command or application with only its specific locked dependencies wired up. Unfortunately to do this it currently double-launches, executing a second Ruby command and doubling the base startup time for these use cases. With Bundler becoming part of the standard library, it seems a good time to add support for doing what <code>bundle exec</code> does without requiring a relaunch.</p>
<p>For many years, JRuby has implemented the <code>-G</code>/<code>--gemfile</code> flag which requires in <code>bundler/setup</code> before user code starts, eliminating the need to bundle exec in these situations.</p>
<p><a href="https://github.com/jruby/jruby/commit/ea0eed02b4eb57c2198afa9fd47f94bc46cfc69f" class="external">https://github.com/jruby/jruby/commit/ea0eed02b4eb57c2198afa9fd47f94bc46cfc69f</a></p>
<p>This is based on the same flags in Rubinius:</p>
<p><a href="https://github.com/rubinius/rubinius/commit/edc94f2e3a61d8c94031a942b2024c6c5aa3ea94" class="external">https://github.com/rubinius/rubinius/commit/edc94f2e3a61d8c94031a942b2024c6c5aa3ea94</a></p>
<p>There's at least one more commit later on in Rubinius that fixes some interations between <code>-G</code> and <code>-S</code>.</p>
<p>Obviously this does not eliminate <code>bundle exec</code> use cases where the target command is not a Ruby command, but that is a very specific (and rather strange to me) feature. The majority of use cases are booting up a Ruby command, for which this implementation of <code>-G</code> would be sufficient (I think).</p>
<p>There may be different or additional ways that <code>bundle exec</code> attempts to isolate the subcommand's environment and dependencies, but I believe requiring <code>bundler/setup</code> at boot comes pretty close.</p>
<p>See also this bug report where I suggest using JRuby's embedding APIs to do <code>bundle exec</code>. We very much want to help eliminate this double-launching, one way or another.</p>
<p><a href="https://github.com/rubygems/rubygems/issues/3246" class="external">https://github.com/rubygems/rubygems/issues/3246</a></p> Ruby master - Feature #16663 (Closed): Add block or filtered forms of Kernel#caller to allow earl...https://bugs.ruby-lang.org/issues/166632020-02-28T22:27:17Zheadius (Charles Nutter)headius@headius.com
<p>There are many libraries that use <code>caller</code> or <code>caller_locations</code> to gather stack information for logging or instrumentation. These methods generate an array of informational stack frames based on the current call stack.</p>
<p>Both methods accept parameters for <code>level</code> (skip some number of Ruby frames) and <code>length</code> (only return this many frames). However many use cases are unable to provide one or both of these.</p>
<p>Instrumentation uses, for example, may need to skip an unknown number of frames at the top of the trace, such as to dig out of rspec plumbing or active_record internals and report the first line of user code. In such cases, the typical pattern is to simply request <em>all</em> frames and then filter out the one that is desired.</p>
<p>This leads to a great deal of wasted work gathering those frames and constructing objects to carry them to the user. On optimizing runtimes like JRuby and TruffleRuby, it can have a tremendous impact on performance, since each frame has a much higher cost than on CRuby.</p>
<p>I propose that we need a new form of <code>caller</code> that takes a block for processing each element.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">find_matching_frame</span><span class="p">(</span><span class="n">regex</span><span class="p">)</span>
<span class="nb">caller</span> <span class="k">do</span> <span class="o">|</span><span class="n">frame</span><span class="o">|</span>
<span class="k">return</span> <span class="n">frame</span> <span class="k">if</span> <span class="n">frame</span><span class="p">.</span><span class="nf">file</span> <span class="o">=~</span> <span class="n">regex</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>An alternative API would be to allow passing a query object as a keyword argument, avoiding the block dispatch by performing the match internally:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">find_matching_frame</span><span class="p">(</span><span class="n">regex</span><span class="p">)</span>
<span class="nb">caller</span><span class="p">(</span><span class="ss">file: </span><span class="n">regex</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>This API would provide a middle ground between explicitly specifying a maximum number of stack frames and asking for all frames. Most common, hot-path uses of <code>caller</code> could be replaced by these forms, reducing overhead on all Ruby implementations and drastically reducing it where stack traces are expensive.</p> Ruby master - Feature #16150 (Open): Add a way to request a frozen string from to_shttps://bugs.ruby-lang.org/issues/161502019-09-07T05:10:37Zheadius (Charles Nutter)headius@headius.com
<p>Much of the time when a user calls to_s, they are just looking for a simple string representation to display or to interpolate into another string. In my brief exploration, the result of to_s is rarely mutated directly.</p>
<p>It seems that we could save a lot of objects by providing a way to explicitly request a <em>frozen</em> string.</p>
<p>For purposes of discussion I will call this to_frozen_string, which is a terrible name.</p>
<p>This would reduce string allocations dramatically when applied to many common to_s calls:</p>
<ul>
<li>Symbol#to_frozen_string could always return the same cached String representation. This method is <em>heavily</em> used by almost all Ruby code that intermingles Symbols and Strings.</li>
<li>nil, true, false, and any other singleton values in the system could similarly cache and return the same String object.</li>
<li>The strings coming from core types could also be in the fstring cache and deduplicated as a result.</li>
<li>User-provided to_s implementations could opt-in to caching and returning the same frozen String object when the author knows that the result will always be the same.</li>
</ul>
<p>A few ideas for what to call this:</p>
<ul>
<li>
<code>to_fstring</code> or <code>fstring</code> reflects internal the "fstring" cache but is perhaps not obvious for most users.</li>
<li>
<code>to_s(frozen: true)</code> is clean but there will be many cases when the kwargs hash doesn't get eliminated, making matters worse.</li>
<li>
<code>def to_s(frozen = false)</code> would be mostly free but may not be compatible with existing to_s params (like <code>Integer#to_s(radix)</code>
</li>
</ul>
<p>This idea was inspired by @schneems's talk at RubyConf Thailand, where he showed significant overhead in ActiveRecord from Symbol#to_s allocation.</p> Ruby master - Bug #15791 (Closed): Clarify reason for RbConfig's ruby_version not reflecting "tee...https://bugs.ruby-lang.org/issues/157912019-04-25T15:44:14Zheadius (Charles Nutter)headius@headius.com
<p>I did not realize that MRI always reports <code>RbConfig::CONFIG['ruby_version']</code> without the "teeny" value. Instead, it makes it always 0:</p>
<pre><code>$ rvm ruby-2.6.2 do ruby -v -e 'p RbConfig::CONFIG["ruby_version"]'
ruby 2.6.2p47 (2019-03-13 revision 67232) [x86_64-darwin18]
"2.6.0"
</code></pre>
<p>This seems like a bug to me. It is a visible behavior because several package-management systems (RubyGems, Bundler, stuff in Ruby switchers like RVM) use this value, rather than RUBY_VERSION, to set up directory paths.</p>
<p>I believe it should reflect the full, accurate version, but I have not been able to find any discussion about why it does not do so.</p> Ruby master - Bug #15711 (Closed): Remove use of _id2ref from DRbhttps://bugs.ruby-lang.org/issues/157112019-03-19T15:56:31Zheadius (Charles Nutter)headius@headius.com
<p>This issue relates to <a href="https://bugs.ruby-lang.org/issues/15408" class="external">https://bugs.ruby-lang.org/issues/15408</a></p>
<p>DRb uses <code>_idref</code> internally to implement a weak map, and this issue seeks to replace that code with an implementation that does not use <code>_id2ref</code>.</p>
<p>We will be deprecating <code>ObjectSpace._id2ref</code> in the near future since it fails to work like people expect (when implemented as a pointer address) or adds memory and invocation overhead to <code>object_id</code>.</p>
<p>An initial patch for this is provided by JRuby, which implements <code>object_id</code> using a monotonically-increasing value, and only allows <code>_id2ref</code> use with a command line flag.</p>
<p><a href="https://github.com/ruby/ruby/compare/trunk...jruby:jruby-ruby_2_6_0#diff-e979bf2f831d9826629559b8628809e9" class="external">https://github.com/ruby/ruby/compare/trunk...jruby:jruby-ruby_2_6_0#diff-e979bf2f831d9826629559b8628809e9</a></p>
<p>This implementation uses the stdlib <code>weakref</code> to implement a simple weak map, and it would be suitable as an implementation for now. However there's some inefficiency here because it has to periodically "clean" the hash of vacated references by scanning all entries.</p>
<p>There are two more efficient implementations that require additional work:</p>
<p>Alternate 1: Use <code>ObjectSpace::WeakMap</code>, which is an opaque VM-supported implementation of a weak Hash. Unfortunately I don't think <code>WeakMap</code> has ever been blessed as a public API, and since we're rapidly moving standard libraries to gems, it would not be appropriate to use an internal API. So, we can either make WeakMap an official part of the public standard API, or do alternate 2.</p>
<p>Alternate 2: Add weak reference queues to the weakref API, so users can implement their own efficient weak maps. Some of this has been discussed (at great length) in <a href="https://bugs.ruby-lang.org/issues/4168" class="external">https://bugs.ruby-lang.org/issues/4168</a>, and the JRuby team has supported the <a href="https://github.com/headius/weakling" class="external">weaklink</a> gem for many years (which provides a WeakRef+RefQueue implementation for JRuby).</p>
<p>The original patch works well for small numbers of remoted objects.</p> Ruby master - Bug #15661 (Closed): Disallow concurrent Dir.chdir with blockhttps://bugs.ruby-lang.org/issues/156612019-03-13T12:19:53Zheadius (Charles Nutter)headius@headius.com
<p><code>Dir.chdir</code> with a block should disallow concurrent use, since it will almost never produce the results a user expects.</p>
<p>In <a href="https://bugs.ruby-lang.org/issues/9785" class="external">https://bugs.ruby-lang.org/issues/9785</a> calls for <code>Dir.chdir</code> to be made thread-safe were rejected because the underlying native call is process-global. This is reasonable because there's no way to easily make the native chdir be thread-local (at least not without larger changes to CRuby itself).</p>
<p>However the block form of <code>chdir</code> is explicitly bounded, and clearly indicates that the dir should be changed only for the given block. I believe <code>Dir.chdir</code> should prevent multiple threads from attempting to do this bounded <code>chdir</code> at the same time.</p>
<p>Currently, if two threads attempt to do a <code>Dir.chdir</code> with a block, one of them will see a warning: "conflicting chdir during another chdir block". This warning is presumably intended to inform the user that they may see unpredictable results, but I can think of no cases where you would ever see predictable results.</p>
<p>In other words, there's no reason to allow a user to do concurrent <code>Dir.chdir</code> with a block because they will always be at risk of unpredictable results, and I believe this only leads to hard-to-diagnose bugs.</p>
<p>The warning should be a hard error.</p>
<p>The warning should also be turned into an error if a non-block <code>Dir.chdir</code> call happens while a block Dir.chdir is in operation. The important thing is to protect the integrity of the current directory during the lifetime of that block invocation.</p>
<p>In CRuby terms, the patch would be to check for <code>chdir_blocking > 0</code> and then simply error out, unless it's happening on the same thread.</p>
<p>Concurrent non-block <code>Dir.chdir</code> would be unaffected. This only protects the block form from having the current dir change while it is executing.</p>
<p>See <a href="https://github.com/jruby/jruby/issues/5649" class="external">https://github.com/jruby/jruby/issues/5649</a></p> Ruby master - Feature #15605 (Closed): json library needs more frequent releaseshttps://bugs.ruby-lang.org/issues/156052019-02-15T01:40:56Zheadius (Charles Nutter)headius@headius.com
<p>There has not been a release of the json gem since April 2017.</p>
<p>Unfortunately there's a long backlog of bugs and pull requests that have been sitting there, including some that prevent users from upgrading JRuby.</p>
<p>I think most of us who have been working on Ruby for any length of time knows that it has been difficult to get json changes released, and it's now starting to really hold back other development.</p>
<p>I propose that we start looking for an alternative maintainer of the library, find a way to get them push rights, and end this bottleneck.</p>
<p>FWIW only flori and I currently have push rights for the gem, but I do not have push rights for the repository.</p> Ruby master - Feature #15408 (Open): Deprecate object_id and _id2refhttps://bugs.ruby-lang.org/issues/154082018-12-13T00:53:26Zheadius (Charles Nutter)headius@headius.com
<p>Ruby currently provides the object_id method to get a "identifier" for a given object. According to the documentation, this ID is the same for every object_id call against a given object, and guaranteed not to be the same as any other active (i.e. alive) object. However, no guarantee is made about the ID being reused for a future object after the original has been garbage collected.</p>
<p>As a result, object_id can't be used to uniquely identify any object that might be garbage collected, since that ID may be associated with a completely different object in the future.</p>
<p>Ruby also provides a method to go from an object_id to the object reference itself: ObjectSpace._id2ref. This method has been in Ruby for decades and is often used to implement a weak hashmap from ID to reference, since holding the ID will not keep the object alive. However due to the problems with object_id not actually being unique, it's possible for _id2ref to return a different object than originally had that ID as object slots are reused in the heap.</p>
<p>The only way to implement object_id safely (with idempotency guarantees) would be to assign to all objects a monotonically-increasing ID. Alternatively, this ID could be assigned lazily only for those objects on which the code calls object_id. JRuby implements object_id in this way currently.</p>
<p>The only way to implement _id2ref safely would be to have a mapping in memory from those monotonically-increasing IDs to the actual objects. This would have to be a weak mapping to prevent the objects from being garbage collected. JRuby currently only supports _id2ref via a flag, since the additional overhead of weakly tracking every requested object_id is extremely high. An alternative for MRI would be to implement _id2ref as a heap scan, as it is implemented in Rubinius. This would make it entirely unpractical due to the cost of scanning the heap for every ID lookup.</p>
<p>I propose that both methods should immediately be deprecated for removal in Ruby 3.0.</p>
<ul>
<li>They do not do what people expect.</li>
<li>They cannot reliably do what they claim to do.</li>
<li>They eventually lead to difficult-to-diagnose bugs in every possible use case.</li>
</ul>
<p>Put simply, both methods have always been broken in MRI and making them unbroken would render them useless.</p> Ruby master - Bug #14712 (Closed): test_step does not work for implementations with full 64-bit f...https://bugs.ruby-lang.org/issues/147122018-04-25T17:35:29Zheadius (Charles Nutter)headius@headius.com
<p>In test/ruby/test_numeric.rb, in test_step, there's the following logic:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">assert_operator</span><span class="p">((</span><span class="mf">0.0</span><span class="p">).</span><span class="nf">step</span><span class="p">(</span><span class="n">bignum</span><span class="p">.</span><span class="nf">to_f</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">).</span><span class="nf">size</span><span class="p">,</span> <span class="p">:</span><span class="o">>=</span><span class="p">,</span> <span class="n">bignum</span><span class="p">)</span> <span class="c1"># may loose precision</span>
</code></pre>
<p>where bignum is</p>
<pre><code>bignum = RbConfig::LIMITS['FIXNUM_MAX'] + 1
</code></pre>
<p>This test passes in both CRuby and JRuby when using CRuby's "smallest bignum" value of 4611686018427387904.</p>
<p>This test fails in both CRuby and JRuby when using JRuby's "smallest bignum" value of 9223372036854775809.</p>
<p>The failure in JRuby is the same as in CRuby:</p>
<pre><code>$ rvm ruby-2.5 do ruby test/mri/runner.rb test/mri/ruby/test_numeric.rb -n test_step
Run options: -n test_step
# Running tests:
[1/1] TestNumeric#test_step = 0.00 s
1) Failure:
TestNumeric#test_step [/Users/headius/projects/jruby/test/mri/ruby/test_numeric.rb:292]:
Expected 9223372036854775808 to be >= 9223372036854775809.
</code></pre>
<p>Since we (JRuby) use CRuby's tests largely unmodified, I'd like this one to run successfully. However, hardcoding to the CRuby value would mean we aren't testing the same things (since that value would use fixnum logic), so I'm not sure the best way to improve this test.</p> Ruby master - Feature #14138 (Closed): Define English.rb aliases by default and eliminate the lib...https://bugs.ruby-lang.org/issues/141382017-11-28T19:10:32Zheadius (Charles Nutter)headius@headius.com
<p>There are many good reasons to simply define all the English.rb global aliases at boot:</p>
<ul>
<li>JRuby has done it for years with zero reported bugs.</li>
<li>Rubinius has done it for years with zero reported bugs.</li>
<li>Globals are in general discouraged, so the likelihood of collision with a user library is very low.</li>
<li>The globals that remain are better indicated by using the long name rather than the Perl-esque short names.</li>
<li>One less library to maintain.</li>
</ul>
<p>The only argument to not do this seems to be the collision concern, but that has not affected hundreds of users of JRuby and Rubinius over the past decade. I think we're safe.</p>
<p>This would be ideal for 2.5, if it's not too late, since this is the big "gemifying" release. Eliminating English.rb would fit into that well.</p> Ruby master - Bug #14096 (Closed): Psych allows invalid single quote escape characterhttps://bugs.ruby-lang.org/issues/140962017-11-10T04:13:42Zheadius (Charles Nutter)headius@headius.com
<p>In <a href="https://github.com/jruby/jruby/issues/4847" class="external">https://github.com/jruby/jruby/issues/4847</a> (and previously in <a href="https://github.com/jruby/jruby/issues/2199" class="external">https://github.com/jruby/jruby/issues/2199</a>) we have had users report that escaped single quotes within a YAML string do not parse in JRuby, although they parse in MRI.</p>
<p>According to the YAML specs (both 1.1 and 1.2), <code>\\'</code> is <em>not</em> a valid escape character. The YAML library we use, SnakeYAML, has pushed back on adding it since it is not part of the spec.</p>
<p>Why does MRI parse this invalid YAML?</p>
<pre><code class="yaml syntaxhl" data-language="yaml"><span class="nn">---</span>
<span class="kt">!!seq</span> <span class="pi">[</span>
<span class="kt">!!str</span> <span class="s2">"</span><span class="s">https://www.youtube.com/watch?v=DzpKasJJtRs"</span><span class="pi">,</span>
<span class="kt">!!str</span> <span class="s2">"</span><span class="s">2Pac</span><span class="nv"> </span><span class="s">-</span><span class="nv"> </span><span class="s">Dont</span><span class="nv"> </span><span class="s">Care</span><span class="nv"> </span><span class="s">What</span><span class="nv"> </span><span class="s">Ya</span><span class="err">\</span><span class="s">'ll</span><span class="nv"> </span><span class="s">Think</span><span class="nv"> </span><span class="s">Remix</span><span class="nv"> </span><span class="s">Music</span><span class="nv"> </span><span class="s">Video</span><span class="nv"> </span><span class="s">2017"</span><span class="pi">,</span>
<span class="pi">]</span>
</code></pre> Ruby master - Bug #14094 (Closed): IRB does not obey frame-level visibility modifiershttps://bugs.ruby-lang.org/issues/140942017-11-09T16:46:49Zheadius (Charles Nutter)headius@headius.com
<p>I noticed the following today; IRB does not appear to obey the frame-local visibility modifiers like public and private:</p>
<pre><code>$ irb
:here
irb(main):001:0> private
=> Object
irb(main):002:0> def foo; end; self.class.public_instance_methods.grep :foo
=> [:foo]
irb(main):003:0> private def foo; end; self.class.public_instance_methods.grep :foo
=> []
</code></pre>
<p>Note that the binding these commands are executed in is a method scope binding that <em>also</em> has its frame-local visibility set to private, so it seems like IRB is not working properly here.</p> Ruby master - Bug #14028 (Closed): RubyVM logic backported into Ruby 2.3 testshttps://bugs.ruby-lang.org/issues/140282017-10-18T19:22:34Zheadius (Charles Nutter)headius@headius.com
<p>For <a href="https://bugs.ruby-lang.org/issues/12517" class="external">https://bugs.ruby-lang.org/issues/12517</a> a test was added to test/coverage/test_coverage.rb that uses RubyVM.</p>
<p>This test is used by JRuby (at least) which does not support the MRI-specific RubyVM module. As a result, we cannot run this test to verify that we have appropriately fixed the related issue in JRuby.</p>
<p>Tests and stdlib that use RubyVM should provide a fallback for implementations that do not support RubyVM.</p> Ruby master - Bug #14010 (Closed): RubyVM logic in forwardable backported to 2.3, not removedhttps://bugs.ruby-lang.org/issues/140102017-10-12T18:49:09Zheadius (Charles Nutter)headius@headius.com
<p>Logic was added to forwardable.rb in at least one commit, and revised in others:</p>
<p><a href="https://github.com/ruby/ruby/commit/6fd18ca51bbce302865d23632b15af53d3e8f11b" class="external">https://github.com/ruby/ruby/commit/6fd18ca51bbce302865d23632b15af53d3e8f11b</a></p>
<p>On trunk, this logic was eventually replaced with code that works on Ruby impls other than MRI:</p>
<p><a href="https://github.com/ruby/ruby/commit/2283d14cc9fefa278dfde02bdf8d84ce50cfe16f" class="external">https://github.com/ruby/ruby/commit/2283d14cc9fefa278dfde02bdf8d84ce50cfe16f</a></p>
<p>I request that RubyVM not ever be used in stdlib unless guarded with a RUBY_ENGINE check plus fallback code. It is not possible for any implementation other than MRI to support RubyVM::InstructionSequence and whenever it is used in tests or stdlib we have to patch around it.</p>
<p>Please backport the remaining changes to forwardable to the ruby-2_3 branch for release, so we can return to using the stock forwardable.rb in JRuby.</p>
<p>Thank you!</p> Ruby master - Bug #13844 (Closed): Toplevel returns should fire ensureshttps://bugs.ruby-lang.org/issues/138442017-08-28T20:20:11Zheadius (Charles Nutter)headius@headius.com
<p>In the following contexts, a return always fires the ensure that wraps it:</p>
<pre><code>[] ~/projects/ruby $ ruby -e 'def foo; return; ensure; p :x; end; foo'
:x
[] ~/projects/ruby $ ruby -e 'def foo; 1.times { begin; return; ensure; p :x; end }; end; foo'
:x
</code></pre>
<p>However the new 2.4 support for toplevel returns does <em>not</em> fire ensures:</p>
<pre><code>$ ruby -e 'begin; return; ensure; p :x; end'
<no output>
</code></pre>
<p>I believe this is inconsistent with how returns work everywhere else (both valid and invalid returns always fire ensure) and it should be changed to match.</p> Ruby master - Bug #13723 (Closed): Change to use RubyVM for syntax check in test suite breaks sui...https://bugs.ruby-lang.org/issues/137232017-07-06T18:46:21Zheadius (Charles Nutter)headius@headius.com
<p>In revision 57158 (6b5f9277 on github) nobu modified the syntax checks in test/lib/test/unit/assertions.rb to use MRI-specific features.</p>
<p><a href="https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/57158" class="external">https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/57158</a></p>
<p>Specifically, instead of using eval to check syntax, it now uses <code>RubyVM::InstructionSequence</code>, which only exists on MRI.</p>
<p>Because of the way the MRI tests are structured, we need to use test/lib contents on JRuby to run the tests. This change means a number of tests that passed before now fail, because we don't support <code>RubyVM::InstructionSequence</code>.</p> Ruby master - Feature #13000 (Feedback): Implement Set#include? with Hash#include?https://bugs.ruby-lang.org/issues/130002016-12-02T16:25:39Zheadius (Charles Nutter)headius@headius.com
<p>Why does <code>Set#include?</code> not call <code>Hash#include?</code>? Currently it calls <code>Hash#[]</code>.</p>
<p>The protocol of Set already use <code>Hash#include?</code> for <code>==</code>.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/set.rb b/lib/set.rb
index 43c388c..f3dbe2d 100644
</span><span class="gd">--- a/lib/set.rb
</span><span class="gi">+++ b/lib/set.rb
</span><span class="p">@@ -230,7 +230,7 @@</span> def flatten!
#
# See also Enumerable#include?
def include?(o)
<span class="gd">- @hash[o]
</span><span class="gi">+ @hash.include?(o)
</span> end
alias member? include?
</code></pre> Ruby master - Bug #12938 (Closed): forwardable.rb is no longer compatible with any alternative im...https://bugs.ruby-lang.org/issues/129382016-11-14T21:31:04Zheadius (Charles Nutter)headius@headius.com
<p>The following commits have made forwardable.rb unusable on any implementation except MRI, because they use YARV-specific RubyVM module and features. I believe all of these changes were done by nobu.</p>
<p>These introduce uses of RubyVM::InstructionSequence, which does not (and probably will not ever) exist on other Ruby implementations:</p>
<p>r53383: forwardable.rb: adjust backtrace by tail call<br>
r55376: forwardable.rb: optimize awy <strong>send</strong></p>
<p>This commit moves code around, so I wasn't sure if anything incompatible was introduced.</p>
<p>r55366: forwardable.rb: fix for non-module objects</p>
<p>The standard library of Ruby is shared by all Ruby implementations, and I believe every effort should be made to avoid using MRI-specific features. If such features must be used, there must always be a fallback path that works on all Rubies of a given compatibility level (2.4 in this case).</p> Ruby master - Bug #12860 (Closed): Splatting an argument does not obey left-to-right execution orderhttps://bugs.ruby-lang.org/issues/128602016-10-21T14:29:18Zheadius (Charles Nutter)headius@headius.com
<p>Ruby evaluates arguments left to right, but it does not appear to handle construction of the eventual argument list from left to right.</p>
<p>Take this example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="nb">p</span> <span class="n">args</span>
<span class="k">end</span>
<span class="n">ary</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">]</span>
<span class="n">foo</span><span class="p">(</span><span class="o">*</span><span class="n">ary</span><span class="p">,</span> <span class="n">ary</span><span class="p">.</span><span class="nf">shift</span><span class="p">)</span>
</code></pre>
<p>With left-to-right execution, the <code>ary</code> value should be splatted (1, 2), and THEN shifted (1) producing <code>args == [1, 2, 1]</code>.</p>
<p>However, on MRI, the shift occurs <em>before</em> the splat, so <code>args == [2, 1]</code>.</p>
<p>This is counter-intuitive. At the moment in time the splat is encountered, <code>ary</code> is still <code>[1, 2]</code>. So the first two arguments should be (1, 2). THEN the shift happens, producing a third argument of (1).</p>
<p>This affects JRuby running Rails because they have a small piece of code that depends on this unusual behavior: <a href="https://github.com/rails/rails/blob/master/activesupport/lib/active_support/callbacks.rb#L411-L425" class="external">https://github.com/rails/rails/blob/master/activesupport/lib/active_support/callbacks.rb#L411-L425</a></p>
<p>This code appears to have been introduced into Rails recently, and I will file a separate issue to change it to be more explicit about ordering.</p> Ruby master - Bug #12689 (Open): Thread isolation of $~ and $_https://bugs.ruby-lang.org/issues/126892016-08-19T06:37:18Zheadius (Charles Nutter)headius@headius.com
<p>We are debating what is correct behavior now, and what should be correct behavior in the future, for the thread-visibility of the special variables <code>%~</code> and <code>$_</code></p>
<p>We have several examples from <a href="https://github.com/jruby/jruby/issues/3031" class="external">https://github.com/jruby/jruby/issues/3031</a> that seem to exhibit conflicting behavior...or at least the behavior is unexpected in many cases.</p>
<pre><code>$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
nil
$ ruby23 -e 'def foo; proc { p $~; "foo" =~ /foo/ }; end; p = foo; Thread.new {p.call}.join; Thread.new{p.call}.join'
nil
#<MatchData "foo">
$ ruby23 -e 'p = proc { p $~; "foo" =~ /foo/ }; def foo(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo(p)'
nil
#<MatchData "foo">
$ ruby23 -e 'class Foo; P = proc { p $~; "foo" =~ /foo/ }; def foo; Thread.new {P.call}.join; Thread.new{P.call}.join; end; end; Foo.new.foo'
nil
#<MatchData "foo">
$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
nil
$ ruby23 -e 'def foo; p = proc { p $~; "foo" =~ /foo/ }; bar(p); end; def bar(p); Thread.new {p.call}.join; Thread.new{p.call}.join; end; foo'
nil
#<MatchData "foo">
</code></pre>
<p>These cases exhibit some oddities in whether $~ (and presumably $_) are shared across threads.</p>
<p>The immediate thought is that they should be both frame and thread-local...but ko1 points out that such a change would break cases like this:</p>
<pre><code>def foo
/foo/ =~ 'foo'
Proc.new{
p $~
}
end
Thread.new{
foo.call
}.join
</code></pre>
<p>So there's a clear conflict here. Users sometimes expect the $~ value to be shared across threads (at least for read, as in ko1's example) and sometimes do not want it shared at all (as in the case of <a href="https://github.com/jruby/jruby/issues/3031" class="external">https://github.com/jruby/jruby/issues/3031</a></p>
<p>Now we discuss.</p> Ruby master - Bug #12688 (Closed): Thread unsafety in autoloadhttps://bugs.ruby-lang.org/issues/126882016-08-19T05:13:43Zheadius (Charles Nutter)headius@headius.com
<p>I need clarification here. I expected, based on Ruby's assertion that autoloads are thread-safe, that the following code would never error. Instead, it gets a couple iterations in and raises NameError:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="kp">loop</span> <span class="k">do</span>
<span class="k">class</span> <span class="nc">Foo</span>
<span class="nb">autoload</span> <span class="ss">:Bar</span><span class="p">,</span> <span class="s1">'bar.rb'</span>
<span class="k">end</span>
<span class="n">go</span> <span class="o">=</span> <span class="kp">false</span>
<span class="n">threads</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">50</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span><span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="mi">1</span> <span class="k">until</span> <span class="n">go</span><span class="p">;</span> <span class="nb">print</span> <span class="s1">'.'</span><span class="p">;</span> <span class="no">Foo</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="ss">:Bar</span><span class="p">)</span> <span class="p">}}</span>
<span class="n">go</span> <span class="o">=</span> <span class="kp">true</span>
<span class="n">threads</span><span class="p">.</span><span class="nf">each</span><span class="p">(</span><span class="o">&</span><span class="ss">:join</span><span class="p">)</span>
<span class="nb">puts</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">send</span> <span class="ss">:remove_const</span><span class="p">,</span> <span class="ss">:Foo</span>
<span class="k">end</span>
</code></pre>
<p>And the output with Ruby 2.3.0:</p>
<pre><code>$ ruby23 -I. autoload_breaker.rb
..................................................
..................................................autoload_breaker.rb:7:in `const_get': uninitialized constant Foo::Bar (NameError)
Did you mean? Foo::Bar
from autoload_breaker.rb:7:in `block (3 levels) in <main>'
</code></pre>
<p>Is there something wrong with my script? Is my expectation incorrect?</p> Ruby master - Bug #12671 (Closed): Hash#to_proc result is not a lambda, but enforces arityhttps://bugs.ruby-lang.org/issues/126712016-08-12T01:25:55Zheadius (Charles Nutter)headius@headius.com
<pre><code> $ ruby23 -e 'pr = {foo:1}.to_proc; puts pr.lambda?; pr.call rescue puts $!; pr.call(1, 2) rescue puts $!'
false
wrong number of arguments (given 0, expected 1)
wrong number of arguments (given 2, expected 1)
</code></pre>
<p>I believe it should be marked as a lambda, since it enforces arity.</p> Ruby master - Feature #12481 (Feedback): Add Array#view to allow opt-in copy-on-write sharing of ...https://bugs.ruby-lang.org/issues/124812016-06-11T03:50:17Zheadius (Charles Nutter)headius@headius.com
<p>Currently the only way to get a copy-on-write view of an existing array is to construct a new array, via <code>#[]</code> or <code>partition</code> or similar methods. I propose a new way to use an existing array to get a view of another array: Array#view (name negotiable).</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">ary1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">]</span>
<span class="n">ary2</span> <span class="o">=</span> <span class="no">Array</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">ary2</span><span class="p">.</span><span class="nf">view</span><span class="p">(</span><span class="n">ary1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="c1"># => ary2 now holds [2, 3] without any allocation.</span>
</code></pre>
<p>This would be an OPTIONAL feature for VMs to implement, if they have copy-on-write array capabilities. Otherwise, it would just do the allocation necessary to hold a copy of the results.</p>
<p>Because of copy-on-write, there would be no visible connection between ary2 and ary1 above; any modifications to either would force a copy.</p>
<p>Consider the case of processing a large array of values N at a time; you could <code>ary2.view</code> in a tight loop, passing that to other functions, and never allocate a single object.</p>
<p>Official documentation would say something like "Make this array contain the contents of the target array starting at 'begin' for 'len' elements." COW would not be mentioned, but VMs could improve performance by using COW under the covers.</p>
<p>A similar feature String#view would also be useful for traversing a stream of bytes without doing any allocation. Think parsing.</p> Ruby master - Bug #12359 (Closed): Named captures "conflict" warning is unnecessary and limits us...https://bugs.ruby-lang.org/issues/123592016-05-09T14:46:20Zheadius (Charles Nutter)headius@headius.com
<p>There's currently a warning whenever a named capture in a regular expression has the same name as an already-declared local variable:</p>
<pre><code>$ ruby23 -v -e 'x = 1; /(?<x>foo)/ =~ "foo"'
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin14]
-e:1: warning: named capture conflicts a local variable - x
</code></pre>
<p>It also warns if you have two expressions using the same named capture:</p>
<pre><code>$ ruby23 -v -e '/(?<x>foo)/ =~ "foo"; /(?<x>foo)/ =~ "foo"'
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin14]
-e:1: warning: named capture conflicts a local variable - x
</code></pre>
<p>I do not believe either of these warnings are useful, since the only option for fixing them is to use a new variable name for every named capture.</p>
<p>I have a few more reasons, as well:</p>
<ul>
<li>Using the same named capture on both the "then" and "else" branches of an "if" statement will produce a warning. Changing one of the names requires additional work to join the to branches back together (since now there's no single variable resulting from them).</li>
<li>It is not possible to initialize a variable with the same name as a named capture.</li>
<li>The pattern that results from this warning is renaming a capture something like "name2" and then having "name = name2" after the match. Ugly.</li>
</ul>
<p>Risks of removing the warning:</p>
<ul>
<li>When a named capture writes to an already-declared variable, that may surprise a user.</li>
</ul>
<p>I think this risk is very low, especially since you'd have to turn on verbose mode to even see the warning.</p> Ruby master - Bug #12068 (Closed): raise overwrites exception cause even if exception is not newhttps://bugs.ruby-lang.org/issues/120682016-02-13T12:03:52Zheadius (Charles Nutter)headius@headius.com
<p>It appears that <code>raise</code> will overwrite the original cause of an exception even when re-raising the same one. I believe this is a bug, since loses the original cause and associates the re-raised exception with a completely unrelated cause.</p>
<p>Example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">begin</span>
<span class="k">begin</span>
<span class="k">raise</span> <span class="s1">'a'</span>
<span class="k">rescue</span> <span class="o">=></span> <span class="n">a</span>
<span class="k">begin</span>
<span class="k">raise</span> <span class="s1">'b'</span>
<span class="k">rescue</span> <span class="o">=></span> <span class="n">b</span>
<span class="nb">p</span> <span class="p">[</span><span class="n">b</span><span class="p">,</span> <span class="n">b</span><span class="p">.</span><span class="nf">cause</span><span class="p">]</span>
<span class="k">begin</span>
<span class="k">raise</span> <span class="s1">'c'</span>
<span class="k">rescue</span>
<span class="k">raise</span> <span class="n">b</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">rescue</span>
<span class="nb">p</span> <span class="p">[</span><span class="vg">$!</span><span class="p">,</span> <span class="vg">$!</span><span class="p">.</span><span class="nf">cause</span><span class="p">]</span>
<span class="k">end</span>
</code></pre>
<p>CRuby outputs:</p>
<pre><code>[] ~/projects/jruby $ ruby23 cause.rb
[#<RuntimeError: b>, #<RuntimeError: a>]
[#<RuntimeError: b>, #<RuntimeError: c>]
</code></pre>
<p>Here, the original cause of the "b" exception (the "a" exception) is lost, replaced during the re-raise with the "c" exception.</p>
<p>I believe JRuby 9.0.5.0's behavior is correct here:</p>
<pre><code>[] ~/projects/jruby $ rvm jruby-9.0.5.0 do ruby cause.rb
[#<RuntimeError: b>, #<RuntimeError: a>]
[#<RuntimeError: b>, #<RuntimeError: a>]
</code></pre> Ruby master - Bug #11822 (Closed): Semantics of Queue#pop after close are wronghttps://bugs.ruby-lang.org/issues/118222015-12-15T16:05:30Zheadius (Charles Nutter)headius@headius.com
<p>Current test/ruby/thread/test_queue.rb test_close has the following assertion that seems wrong to me:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">test_close</span>
<span class="p">[</span><span class="o">-></span><span class="p">{</span><span class="no">Queue</span><span class="p">.</span><span class="nf">new</span><span class="p">},</span> <span class="o">-></span><span class="p">{</span><span class="no">SizedQueue</span><span class="p">.</span><span class="nf">new</span> <span class="mi">3</span><span class="p">}].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">qcreate</span><span class="o">|</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">qcreate</span><span class="p">.</span><span class="nf">call</span>
<span class="n">assert_equal</span> <span class="kp">false</span><span class="p">,</span> <span class="n">q</span><span class="p">.</span><span class="nf">closed?</span>
<span class="n">q</span> <span class="o"><<</span> <span class="ss">:something</span>
<span class="n">assert_equal</span> <span class="n">q</span><span class="p">,</span> <span class="n">q</span><span class="p">.</span><span class="nf">close</span>
<span class="n">assert</span> <span class="n">q</span><span class="p">.</span><span class="nf">closed?</span>
<span class="n">assert_raise_with_message</span><span class="p">(</span><span class="no">ClosedQueueError</span><span class="p">,</span> <span class="sr">/closed/</span><span class="p">){</span><span class="n">q</span> <span class="o"><<</span> <span class="ss">:nothing</span><span class="p">}</span>
<span class="n">assert_equal</span> <span class="n">q</span><span class="p">.</span><span class="nf">pop</span><span class="p">,</span> <span class="ss">:something</span> <span class="c1"># <<< THIS ONE</span>
<span class="n">assert_nil</span> <span class="n">q</span><span class="p">.</span><span class="nf">pop</span>
<span class="n">assert_nil</span> <span class="n">q</span><span class="p">.</span><span class="nf">pop</span>
<span class="c1"># non-blocking</span>
<span class="n">assert_raise_with_message</span><span class="p">(</span><span class="no">ThreadError</span><span class="p">,</span> <span class="sr">/queue empty/</span><span class="p">){</span><span class="n">q</span><span class="p">.</span><span class="nf">pop</span><span class="p">(</span><span class="n">non_block</span><span class="o">=</span><span class="kp">true</span><span class="p">)}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>Once a queue is closed, I don't think it should ever return a result anymore. The queue should be cleared and pop should always return nil.</p>
<p>In r52691, ko1 states that "deq'ing on closed queue returns nil, always." This test does not match that behavior.</p> Ruby master - Feature #11539 (Open): Support explicit declaration of volatile instance variableshttps://bugs.ruby-lang.org/issues/115392015-09-18T21:47:40Zheadius (Charles Nutter)headius@headius.com
<a name="SUMMARY"></a>
<h1 >SUMMARY<a href="#SUMMARY" class="wiki-anchor">¶</a></h1>
<p>We seek to add a mechanism for explicitly declaring the volatility of a given instance variable and a formalization of current volatility semantics.</p>
<a name="BACKGROUND"></a>
<h1 >BACKGROUND<a href="#BACKGROUND" class="wiki-anchor">¶</a></h1>
<p>Concurrency is becoming more and more important in Ruby, and currently there are no guarantees that an instance variable is volatile.</p>
<p>Volatility in Java allows code to specify that reads and writes of a given field (Java's instance variables) should not be reordered with respect to each other. This is necessary to guarantee that concurrent reads and writes of those fields do not happen in unexpected orders, and it is a critical feature for building concurrency libraries in Java. Volatility is also used to provide hardware-level atomic libraries, a key component of lock-free data structures.</p>
<p>The Java memory model describes, among other things, how volatility works: <a href="http://www.cs.umd.edu/~pugh/java/memoryModel/" class="external">http://www.cs.umd.edu/~pugh/java/memoryModel/</a></p>
<p>In CRuby, variables are implicitly volatile due to the fact that the GIL enforces memory barriers when switching threads. However, there is a growing desire to bring parallel threading to CRuby, and this will need to be addressed sooner rather than later.</p>
<p>In JRuby, this is a much bigger concern. Currently, JRuby uses volatile semantics for instance variable writes (not for reads) by default, because of the lack of a standard Ruby memory model. However, this forces undue performance constraints on all users: volatile instance variable writes, for example, have been measured to be 30% slower than non-volatile instance variable writes, and this is taking into account all the usual lookup, caching, and guard logic that surrounds them.</p>
<a name="PURPOSE"></a>
<h1 >PURPOSE<a href="#PURPOSE" class="wiki-anchor">¶</a></h1>
<p>This proposal seeks to formalize the fact that Ruby instance variables are not and have never been <em>guaranteed</em> to be volatile, and provide an explicit mechanism whereby users can opt-in to explicit volatility. This will provide immediate benefits in JRuby (reduced overhead of accessing instance variables) and eventual benefits for CRuby (explicit memory model surrounding instance variables for future parallel execution).</p>
<p>We believe that it should be possible to write efficient concurrent data structures in pure Ruby, and the first step toward this goal would be to have volatility as a first-class citizen of Ruby's object model.</p>
<a name="PROPOSAL"></a>
<h1 >PROPOSAL<a href="#PROPOSAL" class="wiki-anchor">¶</a></h1>
<ol>
<li>Ruby instance variables would be explicitly defined as non-volatile by default.</li>
<li>A new method or syntax would be provided to explicitly opt-in to volatility. Some options:</li>
</ol>
<a name="FORM-1"></a>
<h2 >FORM 1:<a href="#FORM-1" class="wiki-anchor">¶</a></h2>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># This form builds off of existing modifer methods like Module#private.</span>
<span class="n">volatile</span> <span class="ss">:foo</span>
<span class="c1"># No accessors would be created, leaving private variables private, but this would work too:</span>
<span class="n">volatile</span> <span class="nb">attr_accessor</span> <span class="ss">:foo</span>
</code></pre>
<a name="FORM-2"></a>
<h2 >FORM 2:<a href="#FORM-2" class="wiki-anchor">¶</a></h2>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># This form overloads existing attr_accessor with keyword arguments, leaving room for other models like "atomic"</span>
<span class="c1"># Note that this form is less desirable since it requires creating accessor methods to get volatility.</span>
<span class="nb">attr_accessor</span> <span class="ss">:foo</span><span class="p">,</span> <span class="ss">model: </span><span class="n">volatile</span>
</code></pre>
<a name="IMPLEMENTATION"></a>
<h1 >IMPLEMENTATION<a href="#IMPLEMENTATION" class="wiki-anchor">¶</a></h1>
<p>The implementation in MRI would be minimal, given that instance variables are implicitly volatile right now. The new Module#volatile method would be a no-op in MRI until there's need for true volatility.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># Implememtation of volatile for current (non-parallel) MRI:</span>
<span class="k">class</span> <span class="nc">Module</span>
<span class="k">def</span> <span class="nf">volatile</span><span class="p">(</span><span class="o">*</span><span class="n">syms</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>The implementation in JRuby would be trivial, since we already have logic in place to make instance variable reads and writes volatile. We'd simply make them non-volatile globally and add this method to turn volatility on.</p>
<p>So basically the work is already done in both implementations (other than wiring up the Module#volatile method).</p>
<a name="RISKS"></a>
<h1 >RISKS<a href="#RISKS" class="wiki-anchor">¶</a></h1>
<p>There is risk that the new <code>Module#volatile</code> would conflict with some library's addition to <code>Module</code> of the same name, or conflict with some library's metaclass-decorating module that provides a method of the same name. However, we do not believe this is a high risk. We have not done any searches at this time to see if there are such conflicts.</p>
<a name="NAMING"></a>
<h1 >NAMING<a href="#NAMING" class="wiki-anchor">¶</a></h1>
<p>We believe <code>volatile</code> is the best name for this feature, since it is the accepted term in most programming communities (C++ register volatility, Java and C# volatile fields, etc).</p>
<a name="TIMEFRAME"></a>
<h1 >TIMEFRAME<a href="#TIMEFRAME" class="wiki-anchor">¶</a></h1>
<p>As soon as possible. We would like to see this in Ruby 2.3 ideally. Given that it is a no-op in MRI and nearly available in JRuby, this should be feasible.</p>
<a name="RELATED-WORK"></a>
<h1 >RELATED WORK<a href="#RELATED-WORK" class="wiki-anchor">¶</a></h1>
<p>See Java, C# as examples of <code>volatile</code> in real-world practice. Java has an extensive set of lock-free concurrent data structures built entirely in Java atop the ability to declare field volatility.</p>
<p>The concurrent-ruby project (<a href="https://github.com/ruby-concurrency/concurrent-ruby" class="external">https://github.com/ruby-concurrency/concurrent-ruby</a>) implements this feature as a separate class hierarchy, Synchronization::Object. Classes descending from this hierarchy gain the ability to request construction-time memory fencing and instance variable volatility. While this approach works, we believe it would be better to have as a core Ruby feature that can be applied to any instance variable in any class.</p>
<p>C++ also has a notion of volatility as a compiler hint to not reorder stores and loads of a memory location: <a href="http://en.cppreference.com/w/cpp/language/cv" class="external">http://en.cppreference.com/w/cpp/language/cv</a></p> Ruby master - Feature #11195 (Closed): Add "no_proxy" parameter to Net::HTTP.newhttps://bugs.ruby-lang.org/issues/111952015-05-28T22:37:44Zheadius (Charles Nutter)headius@headius.com
<p>Net::HTTP.new currently accepts parameters for proxy host, proxy port, proxy username, and proxy password.</p>
<p>It does not accept an argument for non-proxied addresses, and as a result specifying a proxy host will use that proxy unconditionally.</p>
<p>Compare this with the default ENV behavior, which <em>does</em> honor the "no_proxy" env var. So the only way you can set up a proxy AND have it honor non-proxied addresses is to use ENV.</p>
<p>I propose that we add one more trailing argument for non-proxied hosts, since this is an unavoidable part of http proxying.</p>
<p>I would have made the change myself, but it involved a bit more work than just adding the param; the logic for honoring "no_proxy" only exists in the ENV logic, and when specified directly there's no affordance for non-proxied addresses in the rest of net/http.</p>
<p>This might be easier if we make uri/generic.rb's find_proxy accept an optional hash from which to look up these values. Then we just store them in Net::HTTP as a local version of ENV.</p> Ruby master - Bug #11194 (Closed): Refactor env-sourced proxy logic for uri/generichttps://bugs.ruby-lang.org/issues/111942015-05-28T22:23:35Zheadius (Charles Nutter)headius@headius.com
<p>For <a href="https://github.com/jruby/jruby/issues/2983" class="external">https://github.com/jruby/jruby/issues/2983</a> we are modifying uri/generic.rb's find_proxy logic to also check the JVM properties used to configure proxies.</p>
<p>The diff is here: <a href="https://gist.github.com/headius/6272d168cf165ddf675f" class="external">https://gist.github.com/headius/6272d168cf165ddf675f</a></p>
<p>There are two changes:</p>
<ol>
<li>The logic for looking up proxy host and non-proxy addresses from ENV has been pulled out into private utility methods. This change should be zero-sum, and I'd like to commit it to HEAD.</li>
<li>Additional logic has been added for looking up JVM properties.</li>
</ol>
<p>I will proceed with the refactor in (1) regardless, but I need community help:</p>
<ul>
<li>Any concerns about adding these methods?</li>
<li>Could we perhaps check RUBY_ENGINE and put the JRuby logic in the canonical uri/generic.rb?</li>
</ul>
<p>Currently we maintain our own fork of stdlib, and if ruby-core folks don't mind we'd REALLY like to get rid of it. That requires adding a few RUBY_ENGINE checks to MRI's stdlib.</p>
<p>Thoughts?</p> Ruby master - Bug #11127 (Third Party's Issue): Symbol#== allocates at least one string, possibly...https://bugs.ruby-lang.org/issues/111272015-05-07T21:00:16Zheadius (Charles Nutter)headius@headius.com
<p>Symbol#== is implemented by Comparable, which uses Symbol#<=>, which coerces the symbol to a String so it can use String#<=> logic.</p>
<p>Most people don't know this, but it adds many hidden allocations to any Symbol-heavy code that that's doing lots of == comparisons.</p>
<p>I propose that Symbol should define its own == that doesn't have to do the extra work for <=>. The other Comparable methods are not important because Symbols usually compared using == or ===.</p> Ruby master - Bug #11119 (Closed): Anonymous classes and modules have terrible #name and #inspect...https://bugs.ruby-lang.org/issues/111192015-05-04T16:07:39Zheadius (Charles Nutter)headius@headius.com
<p>MRI lazily determines the name of a class or module by walking all defined constants starting from <code>Object</code> and looking for the namespace in question. This allows deferring the full name calclation until the class/module is finished being defined. However, if the class or module is <em>never</em> accessible via <code>Object</code>, then this system-walking occurs for every call to <code>#name</code> or <code>#inspect</code> on the <code>class</code>/<code>module</code> and every call to the default <code>#inspect</code> on instances of the class.</p>
<p>A simple benchmark:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="k">module</span> <span class="nn">B</span>
<span class="k">module</span> <span class="nn">X</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">a</span>
<span class="n">c</span> <span class="o">=</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span>
<span class="n">c2</span> <span class="o">=</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span>
<span class="n">c</span><span class="p">.</span><span class="nf">class_eval</span> <span class="s1">'A = c2'</span>
<span class="n">c2</span><span class="p">.</span><span class="nf">class_eval</span> <span class="s1">'A = c'</span>
<span class="n">c</span>
<span class="k">end</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">a</span>
<span class="n">x</span> <span class="o">=</span> <span class="no">B</span><span class="o">::</span><span class="no">X</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="nb">puts</span> <span class="s1">'named'</span>
<span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="mi">1_000_000</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">x</span><span class="p">.</span><span class="nf">name</span> <span class="p">}</span> <span class="p">}</span>
<span class="nb">puts</span> <span class="s1">'anon'</span>
<span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="mi">1_000_000</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">c</span><span class="p">.</span><span class="nf">name</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">cobj</span> <span class="o">=</span> <span class="n">c</span><span class="p">.</span><span class="nf">new</span>
<span class="nb">puts</span> <span class="s1">'anon obj'</span>
<span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="mi">1_000_000</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">cobj</span><span class="p">.</span><span class="nf">inspect</span> <span class="p">}</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre>
<p>Results on MRI 2.2 and JRuby 1.7 HEAD:</p>
<p>MRI:</p>
<pre><code>named
0.210000 0.000000 0.210000 ( 0.205585)
anon
14.170000 0.050000 14.220000 ( 14.259003)
anon obj
15.750000 0.060000 15.810000 ( 15.864806)
</code></pre>
<p>JRuby:</p>
<pre><code>named
0.250000 0.000000 0.250000 ( 0.253000)
anon
0.270000 0.000000 0.270000 ( 0.264000)
anon obj
0.450000 0.000000 0.450000 ( 0.447000)
</code></pre>
<p>The effect worsens linearly with the size of the system. Running in a freshly-generated Rails app's console:</p>
<pre><code>named
0.260000 0.020000 0.280000 ( 0.272182)
anon
240.900000 0.800000 241.700000 (242.384455)
anon obj
257.070000 1.110000 258.180000 (261.986562)
</code></pre>
<p>I believe MRI needs to give up on looking for the object after the first failed namespace traversal, or else eagerly build this name the way other implementations do (and accept some changes).</p> Ruby master - Bug #11045 (Closed): ruby/test_m17n_comb.rb test_str_crypt tests platform-specific ...https://bugs.ruby-lang.org/issues/110452015-04-07T20:36:53Zheadius (Charles Nutter)headius@headius.com
<p>I think we should just remove the glibc version guards in test_str_crypt in test/ruby/test_m17n_comb.rb.</p>
<ol>
<li>JRuby can't determine glibc version without making a glibc-specific call (because we have no build-time configure script or similar)</li>
<li>On platforms that use glibc but don't report version, this guard won't work.</li>
<li>Do we really want to be testing linux-specific crypt behavior?</li>
</ol>
<p>I propose that we just make the test skip non-alpha salt unconditionally.</p> Ruby master - Feature #11026 (Open): How atomic should dynamic regexp with "once" flag be?https://bugs.ruby-lang.org/issues/110262015-04-02T22:02:39Zheadius (Charles Nutter)headius@headius.com
<p>The <code>test_once_multithread</code> test in ruby/test_regexp.rb tries to confirm that a dynamic "once" regexp is created atomically. Here's the test:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">test_once_multithread</span>
<span class="n">m</span> <span class="o">=</span> <span class="no">Mutex</span><span class="p">.</span><span class="nf">new</span>
<span class="n">pr3</span> <span class="o">=</span> <span class="nb">proc</span><span class="p">{</span><span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="sr">/</span><span class="si">#{</span><span class="n">m</span><span class="p">.</span><span class="nf">unlock</span><span class="p">;</span> <span class="nb">sleep</span> <span class="mf">0.5</span><span class="p">;</span> <span class="n">i</span><span class="si">}</span><span class="sr">/o</span>
<span class="p">}</span>
<span class="n">ary</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">n</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">th1</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span><span class="p">{</span><span class="n">m</span><span class="p">.</span><span class="nf">lock</span><span class="p">;</span> <span class="n">ary</span> <span class="o"><<</span> <span class="n">pr3</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">n</span><span class="o">+=</span><span class="mi">1</span><span class="p">)}</span>
<span class="n">th2</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span><span class="p">{</span><span class="n">m</span><span class="p">.</span><span class="nf">lock</span><span class="p">;</span> <span class="n">ary</span> <span class="o"><<</span> <span class="n">pr3</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">n</span><span class="o">+=</span><span class="mi">1</span><span class="p">)}</span>
<span class="n">th1</span><span class="p">.</span><span class="nf">join</span><span class="p">;</span> <span class="n">th2</span><span class="p">.</span><span class="nf">join</span>
<span class="n">assert_equal</span><span class="p">([</span><span class="sr">/1/</span><span class="p">,</span> <span class="sr">/1/</span><span class="p">],</span> <span class="n">ary</span><span class="p">)</span>
<span class="k">end</span>
</code></pre>
<p>The logic here works as follows:</p>
<ul>
<li>Each thread locks the mutex, and then proceeds to execute the regexp with a different dynamic value</li>
<li>Executing the regexp unlocks the mutex and applies the dynamic value</li>
</ul>
<p>The test tries to guarantee that only the first thread to start processing the dynamic regexp will finally cache it, but I think this may be overreaching.</p>
<p>In order to make this test pass in a threaded implementation, the entire body of the regexp would need to be evaluated under synchronization, blocking other threads from doing the same. This would also require a synchronization lock to be acquired for every subsequent access of that regular expression, to ensure that multiple threads arriving at the code at the same time don't try to run the code at the same time or update the cached value twice. This could possibly be reduced with double-checked locking, but having this much synchronization overhead for a simple Ruby expression seems excessive.</p>
<p>Also, this synchronization would only protect threads from running the <code>//o</code> body twice; it does not guarantee ordering of those threads outside of the <code>//o</code>, so if any users depended on this "first thread wins" behavior, they'd still be surprised because they can't guarantee which thread hits the code first.</p>
<p>I believe this test should <em>only</em> confirm that all threads see the same regexp result, rather than trying to test that only one thread ever evaluates the regexp.</p>
<p>As far as specified behavior, I believe the only guarantee <code>//o</code> should make is that it will cache exactly one value and never execute again after that value has been cached, rather than making guarantees about the atomicity of dynamic operations within that regexp. In short, it should guarantee all threads see the same <code>//o</code> expression.</p>
<p>Thoughts?</p> Ruby master - Bug #11015 (Closed): IO.copy_stream does not advance Tempfile destinationhttps://bugs.ruby-lang.org/issues/110152015-03-29T23:44:14Zheadius (Charles Nutter)headius@headius.com
<p>The following script, reported to JRuby in <a href="https://github.com/jruby/jruby/issues/2762" class="external">https://github.com/jruby/jruby/issues/2762</a>, shows that MRI does not advance the position of a <code>Tempfile</code> when using <code>copy_stream</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s2">"stringio"</span>
<span class="nb">require</span> <span class="s2">"tempfile"</span>
<span class="n">file</span> <span class="o">=</span> <span class="no">Tempfile</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)</span>
<span class="no">IO</span><span class="p">.</span><span class="nf">copy_stream</span><span class="p">(</span><span class="no">StringIO</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">),</span> <span class="n">file</span><span class="p">)</span>
<span class="n">file</span><span class="p">.</span><span class="nf">eof?</span> <span class="c1">#=> false (it's on the beginning)</span>
</code></pre>
<p>Other types of streams leave the destination where it is after the copy, so I believe this is a bug.</p> Ruby master - Bug #10741 (Closed): const_defined? triggers autoload where it did not beforehttps://bugs.ruby-lang.org/issues/107412015-01-14T21:09:20Zheadius (Charles Nutter)headius@headius.com
<p>This should not cause b.rb to load, but it does in 2.2:</p>
<p>~/projects/jruby $ cat a.rb<br>
module Foo<br>
autoload :Bar, 'b.rb'<br>
end</p>
<p>p Foo.const_defined?('Bar')</p>
<p>~/projects/jruby $ cat b.rb<br>
puts "in b"<br>
module Foo<br>
Bar = 1<br>
end</p>
<p>~/projects/jruby $ rvm ruby-2.0 do ruby -I. a.rb<br>
true</p>
<p>~/projects/jruby $ rvm ruby-2.1 do ruby -I. a.rb<br>
in b<br>
true</p>
<p>~/projects/jruby $ rvm ruby-2.2 do ruby -I. a.rb<br>
in b<br>
true</p>
<p>This is likely caused by the :: support in const_get/const_defined not doing autoload-free traversal.</p> Ruby master - Bug #10686 (Closed): Memory leaking from torture test of symbol GChttps://bugs.ruby-lang.org/issues/106862015-01-01T00:31:37Zheadius (Charles Nutter)headius@headius.com
<p>The following code appears to grow without bounds when running on MRI 2.2p0 (and grows <em>very</em> fast...hold on to your RAM):</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">x</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="kp">loop</span> <span class="p">{</span> <span class="p">(</span><span class="n">x</span> <span class="o">+=</span> <span class="mi">1</span><span class="p">).</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">to_sym</span> <span class="p">}</span>
</code></pre>
<p>I asked ko1 about this on Twitter and he said it appears to be leaking strings somewhere.</p> Ruby master - Bug #10685 (Closed): Segfault from keyword rest arg when coercing an object that de...https://bugs.ruby-lang.org/issues/106852014-12-31T23:39:59Zheadius (Charles Nutter)headius@headius.com
<p>This spec appears to cause the segfault mentioned in <a href="http://rubini.us/2014/12/31/matz-s-ruby-developers-don-t-use-rubyspec/" class="external">http://rubini.us/2014/12/31/matz-s-ruby-developers-don-t-use-rubyspec/</a> today.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="n">it</span> <span class="s2">"calls #to_hash on the last element if keyword arguments are present"</span> <span class="k">do</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">mock</span><span class="p">(</span><span class="s2">"destructure block keyword arguments"</span><span class="p">)</span>
<span class="n">obj</span><span class="p">.</span><span class="nf">should_receive</span><span class="p">(</span><span class="ss">:to_hash</span><span class="p">).</span><span class="nf">and_return</span><span class="p">({</span><span class="ss">x: </span><span class="mi">9</span><span class="p">})</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">m</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="n">obj</span><span class="p">])</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="o">*</span><span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">,</span> <span class="o">**</span><span class="n">k</span><span class="o">|</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">,</span> <span class="n">k</span><span class="p">]</span> <span class="p">}</span>
<span class="n">result</span><span class="p">.</span><span class="nf">should</span> <span class="o">==</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">],</span> <span class="mi">3</span><span class="p">,</span> <span class="p">{</span><span class="ss">x: </span><span class="mi">9</span><span class="p">}]</span>
<span class="k">end</span>
</code></pre>
<p>I tried to boil it down to a shorter example and was not able to isolate it from RubySpec/mspec entanglement.</p>
<p>Here's the full dump including the last spec line that starts to run (the spec above): <a href="https://gist.github.com/headius/5d52df923eb615729ca3" class="external">https://gist.github.com/headius/5d52df923eb615729ca3</a></p>
<p>I ran it using mspec gem from rubyspec's "archive" branch using "mspec ci -f s language".</p> Ruby master - Feature #10682 (Closed): Add "excludes" support to test suite, for alternative impl...https://bugs.ruby-lang.org/issues/106822014-12-31T19:12:14Zheadius (Charles Nutter)headius@headius.com
<p>JRuby uses MRI's test suite as our primary compatibility suite. We would like to enhance the suite to support excluding tests.</p>
<p>Before the juggling of minitest versions in stdlib, JRuby was using minitest-excludes to exclude tests we knew didn't pass. The EXCLUDE_DIR env pointed at a dir with .rb files named after the test classes and containing "exclude" lines as seen here: <a href="https://github.com/jruby/jruby/blob/jruby-1_7/test/externals/ruby1.9/excludes/TestBignum.rb" class="external">https://github.com/jruby/jruby/blob/jruby-1_7/test/externals/ruby1.9/excludes/TestBignum.rb</a></p>
<p>This allowed us to maintain a "high water mark" for compatibility as MRI's suite evolved. It is largely the same feature as mspec/RubySpec's "tags".</p>
<p>Unfortunately minitest 5 does not support excludes, and MRI's suite has frozen a version of minitest 4 to use that is incompatible with the minitest-excludes library. In addition, many tests have returned to running with test/unit.</p>
<p>I provide here a patch for the exclude support we added to our copy of MRI's suite: <a href="https://gist.github.com/headius/4226cd94bbcf7b150e65" class="external">https://gist.github.com/headius/4226cd94bbcf7b150e65</a></p>
<p>The only significant changes from minitest-excludes:</p>
<ul>
<li>The env var is now "EXCLUDES". This is negotiable, but I don't know if enough people are using (or ever will use again) minitest-excludes, so the compatibility issue is moot.</li>
<li>There's no require needed to activate the excludes; if the env var is present, it will attempt to load them.</li>
</ul>
<p>I'm willing to tune and improve this as necessary. JRuby master (9k, 9.0.0.0, 2.2-compatible) has been running MRI's tests this way for several months.</p> Ruby master - Feature #10499 (Closed): Eliminate implicit magic in Proc.new and Kernel#prochttps://bugs.ruby-lang.org/issues/104992014-11-12T14:18:35Zheadius (Charles Nutter)headius@headius.com
<p>Proc.new and Kernel#proc have a little known feature: if called without a block, they capture whatever block was passed to the current method.</p>
<p>I propose that this feature should be removed, finally, since it:</p>
<ul>
<li>Doesn't enhance readability (where is this block coming from?)</li>
<li>Doesn't reflect any other behavior in Ruby</li>
<li>Can lead to bugs (call either without a block accidentally and you aren't sure what you'll get)</li>
</ul>
<p>I believe this was an implementation artifact in MRI, since the most recently-pushed block would still be on global stacks, which is where the logic for proc and Proc.new looked for it.</p>
<p>All argument syntaxes now support &block, which I believe is the correct way to clearly, explicitly capture the incoming block into an object.</p>
<p>Thoughts?</p> Ruby master - Bug #10494 (Closed): ioctl returns EINVAL instead of ENOTTY for pipes on older linu...https://bugs.ruby-lang.org/issues/104942014-11-11T01:26:04Zheadius (Charles Nutter)headius@headius.com
<p>Yeah I know the title is a mouthful. I'll try to explain.</p>
<p>BACKGROUND:</p>
<p>Starting sometime recently, MRI's test suite started to use a custom version of test/unit contained in test/lib. Inside unit.rb, there's code that uses io/console to determine the window size for some fancy formatting.</p>
<p><a href="https://github.com/ruby/ruby/blob/trunk/test/lib/test/unit.rb#L404" class="external">https://github.com/ruby/ruby/blob/trunk/test/lib/test/unit.rb#L404</a></p>
<p>The surrounding rescue captures LoadError (io/console doesn't exist), NoMethodError (winsize is not defined), ENOTTY (fd is not a tty), and EBADF (fd is no longer valid), and provides suitable fallback code for console width.</p>
<p>PROBLEM:</p>
<p>However, when we ran the MRI suite with JRuby on Travis, we got the following error:</p>
<pre><code> [exec] Run options: -q --
[exec]
[exec] # Running tests:
[exec]
[exec]
[exec] Errno::EINVAL: Invalid argument - ioctl(TIOCGWINSZ) winsize at /home/travis/build/jruby/jruby/lib/ruby/stdlib/io/console.rb:120
[exec] terminal_width at /home/travis/build/jruby/jruby/test/mri/lib/test/unit.rb:404
[exec] put_status at /home/travis/build/jruby/jruby/test/mri/lib/test/unit.rb:433
</code></pre>
<p>This is from within a Maven build target we use to run our test suites with the proper environment. We could not reproduce this on current Linux versions.</p>
<p>CAUSE:</p>
<p>For those unfamiliar with the JVM, the standard process-launching APIs <em>always</em> set the child's stdio to pipes that you can then read, a la popen in MRI. Long story short, it turns out that Linux incorrectly returned EINVAL instead of ENOTTY when calling ioctl with a pipe up until this fix in May of 2012: <a href="https://lkml.org/lkml/2012/5/25/142" class="external">https://lkml.org/lkml/2012/5/25/142</a></p>
<p>This is after the release of Ubuntu 12.04 that Travis runs, and I'm guessing they have not updated the kernels.</p>
<p>REPRODUCTION (for Linux kernels prior to this patch):</p>
<p>I was able to reproduce this EINVAL on a Travis dev box (thanks asarih!) with both JRuby and MRI using the following command line:</p>
<pre><code>$ rvm ruby-2.1 do ruby -rio/console -e 'p $stdout.winsize' | cat
-e:1:in `winsize': Invalid argument (Errno::EINVAL)
from -e:1:in `<main>'
</code></pre>
<p>SOLUTION:</p>
<p>????????</p>
<p>I'm not sure.</p>
<ul>
<li>
<p>We could just patch the test library. I have done this for JRuby here: <a href="https://github.com/jruby/jruby/commit/29122372da5ba2f359239aa916e074a57fe96b70" class="external">https://github.com/jruby/jruby/commit/29122372da5ba2f359239aa916e074a57fe96b70</a></p>
</li>
<li>
<p>We could modify io/console #winsize and other methods to raise this incorrect EINVAL as ENOTTY, since we should know in each of those methods that we have the right request and data structure (ergo EINVAL should not happen on non-buggy systems)</p>
</li>
</ul>
<p>We need to do one or the other for the test suite to be runnable on these earlier Linux kernels when stdout is a pipe.</p> Ruby master - Bug #10478 (Closed): WEBrick's server.rb contains a return that should be a breakhttps://bugs.ruby-lang.org/issues/104782014-11-05T00:19:23Zheadius (Charles Nutter)headius@headius.com
<p>Commit 46253 (git hash 6f226d9) added the ability to shut down the running WEBrick server more gracefully, by monitoring a shutdown pipe. However, the logic to actually escape the server loop was written to use "return" instead of break", which causes a LocalJumpError when run inside a threaded server:</p>
<ol>
<li>Error:<br>
TestNetHTTPContinue#test_expect_continue:<br>
LocalJumpError: unexpected return<br>
/Users/headius/projects/jruby/lib/ruby/stdlib/webrick/server.rb:216:in `start'</li>
</ol>
<p>It is not entirely clear why this does not show up for MRI running the same tests (JRuby master is now running MRI trunk stdlib and tests), but we believe this return should be a "break". A break here exits the server loop and allows the rest of the server logic to finish gracefully without an error bubbling out.</p>
<p>We have made this change in JRuby here: <a href="https://github.com/jruby/jruby/commit/ce7e292dcc899111a908467f90fd0c639cfbba18" class="external">https://github.com/jruby/jruby/commit/ce7e292dcc899111a908467f90fd0c639cfbba18</a></p>
<p>I am marking this urgent, because MRI 2.2 is almost done, and JRuby plans to release shortly after with expected 2.2 compatibility.</p>
<p>Assigning to akr because he did the original commit.</p> Ruby master - Bug #10231 (Closed): Process.detach(pid) defines new singleton classes every callhttps://bugs.ruby-lang.org/issues/102312014-09-11T20:02:34Zheadius (Charles Nutter)headius@headius.com
<p>The logic for Process.detach(pid) adds a singleton "pid" method to the thread it returns for every call. This is bad for method caches (MRI still flushes them all for this, I believe) and memory churn (singleton classes are not small).</p>
<p>The offending line of code is here: <a href="https://github.com/ruby/ruby/blob/trunk/process.c#L1041" class="external">https://github.com/ruby/ruby/blob/trunk/process.c#L1041</a></p>
<p>I would suggest that Process.detach should return a subclass of Thread that has the pid method defined ahead of time.</p>
<p>It also stores the value in thread local storage, rather than as an instance variable. I'm not sure why.</p> Ruby master - Feature #10042 (Feedback): Deprecate postfix rescue syntax for removal in 3.0https://bugs.ruby-lang.org/issues/100422014-07-15T23:58:27Zheadius (Charles Nutter)headius@headius.com
<p>The postfix rescue notation is convenient...but almost always is a really bad antipattern.</p>
<p>An example of the notation:</p>
<pre><code>Integer(f) rescue f # returns f if it is not parseable as an Integer
</code></pre>
<p>It silently ignores all StandardError raised by a piece of code...which often covers <em>many</em> more exceptions than the user <em>wants</em> to be ignoring.</p>
<p>It also hides the cost of constructing and throwing away all those ignored exceptions.</p>
<p>I believe Matz has even said in the past that he regrets adding the feature.</p>
<p>In any case, I propose that "rescue nil" should be deprecated with a warning (either always on or only when verbose) and we should plan to remove it in 3.0.</p>
<p>Who's with me?!</p> Ruby master - Feature #9725 (Open): Do not inspect NameError target object unless verbosehttps://bugs.ruby-lang.org/issues/97252014-04-10T22:19:37Zheadius (Charles Nutter)headius@headius.com
<p>At least once every few months, we get an error report of JRuby raising a memory error where MRI does not due to <code>NameError</code>'s <code>Message</code> object holding a reference to an object that's too large to inspect. I propose that this inspection of the target object should only be done in verbose mode.</p>
<a name="Background"></a>
<h2 >Background:<a href="#Background" class="wiki-anchor">¶</a></h2>
<p><code>NameError</code> is raised when a variable-like method call fails to find a defined method. The resulting exception is created with a hidden <code>NameError::Message</code> that holds the object in which the method could not be found.</p>
<p>When name error needs to render its message, such as when it bubbles out or when <code>#message</code> is called, it does <code>to_str</code> on the <code>NameError::Message</code>, which ends up inspecting the target object. If this object's inspect output is large (or infinite) it can end up consuming a large amount of memory.</p>
<a name="Problems"></a>
<h2 >Problems:<a href="#Problems" class="wiki-anchor">¶</a></h2>
<ul>
<li>If the amount of memory required to render a <code>NameError</code> exceeds available memory, a very confusing and misleading memory error can be raised instead.</li>
<li>If the target object is considered sensitive data, it will end up bubbling out through potentially untrustworthy code. It is an encapsulation flaw, basically.</li>
<li>A <code>NameError</code> that gets held in memory will also prevent GC of the object it references.</li>
</ul>
<a name="Solutions"></a>
<h2 >Solutions:<a href="#Solutions" class="wiki-anchor">¶</a></h2>
<ul>
<li>
<code>NameError</code> should not capture the target object.</li>
<li>
<code>NameError</code> should build a message based on the target object <em>at creation time</em>, and only include information useful to indicate the type of object.</li>
<li>(Optional) If verbose mode is set, <code>NameError</code> can just do what it does now.</li>
</ul> Backport200 - Backport #9649 (Closed): Segfault running thread_safe torture tests under Ruby 2.0https://bugs.ruby-lang.org/issues/96492014-03-17T12:54:58Zheadius (Charles Nutter)headius@headius.com
<p>We received the following report on the thread_safe project: <a href="https://github.com/headius/thread_safe/issues/32" class="external">https://github.com/headius/thread_safe/issues/32</a></p>
<p>It does not appear that any extension code is involved in the backtrace, so I believe the most likely scenario is that the problem lies in MRI.</p>
<p>I have not received any reports of segfaults other than this one.</p> Ruby master - Bug #9363 (Rejected): test_http.rb and others must be run as roothttps://bugs.ruby-lang.org/issues/93632014-01-05T02:41:39Zheadius (Charles Nutter)headius@headius.com
<p>It seems like test/net/test_http.rb (and perhaps other network-related tests) must be run as root to test properly...or I am not seeing how to run them without using privileged ports.</p>
<p>We would like to use these tests in JRuby, and currently that's not possible since they bind to privileged.</p>
<p>If this is the case, we should fix the tests.</p>
<p>If this is not the case, please help me figure out how to run these tests without restricted ports.</p> Ruby master - Bug #9361 (Closed): gem install --ignore-dependencies fails to install local gemshttps://bugs.ruby-lang.org/issues/93612014-01-04T06:52:03Zheadius (Charles Nutter)headius@headius.com
<p>$ rvm ruby-2.1 do gem install --ignore-dependencies /Users/headius/projects/jruby/lib/ruby/gems/shared/cache/dicks-0.03.gem<br>
ERROR: While executing gem ... (NoMethodError)<br>
undefined method `full_name' for nil:NilClass</p>
<p>This is also impacting JRuby master.</p>
<p>Also filed against RubyGems: <a href="https://github.com/rubygems/rubygems/issues/771" class="external">https://github.com/rubygems/rubygems/issues/771</a></p> Ruby master - Feature #9043 (Open): Add String#f method as shortcut for #freezehttps://bugs.ruby-lang.org/issues/90432013-10-22T21:16:59Zheadius (Charles Nutter)headius@headius.com
<p>We have String#b to create a binary-encoded String, and we have the "f" suffix (going away, hopefully) and the "literal".freeze optimization (<a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Use String#freeze and compiler tricks to replace "str"f suffix (Closed)" href="https://bugs.ruby-lang.org/issues/8992">#8992</a>). I think it would be reasonable to add String#f as a shorter method for producing a frozen string.</p>
<p>If String#f is added the "literal".freeze optimization could be dropped in favor of "literal".f. This would provide something very close to the original "literal"f syntax but in a backward-compatibility-friendly way (class String; alias f freeze; end).</p> Ruby master - Feature #9042 (Closed): Remove "f" suffix in favor of VM optimization of "literal"....https://bugs.ruby-lang.org/issues/90422013-10-22T21:14:22Zheadius (Charles Nutter)headius@headius.com
<p>The "f" suffix has been added to Ruby trunk to allow creating pre-frozen literal strings. However, <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Use String#freeze and compiler tricks to replace "str"f suffix (Closed)" href="https://bugs.ruby-lang.org/issues/8992">#8992</a> provides a way the VM could simply optimize #freeze called on a literal string in the same way. If <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Use String#freeze and compiler tricks to replace "str"f suffix (Closed)" href="https://bugs.ruby-lang.org/issues/8992">#8992</a> is accepted, the backward-incompatible "f" syntax should be removed.</p> Ruby master - Feature #8992 (Closed): Use String#freeze and compiler tricks to replace "str"f suffixhttps://bugs.ruby-lang.org/issues/89922013-10-08T02:56:29Zheadius (Charles Nutter)headius@headius.com
<p>BACKGROUND:</p>
<p>In <a href="https://bugs.ruby-lang.org/issues/8579" class="external">https://bugs.ruby-lang.org/issues/8579</a> @charliesome introduced the "f" suffix for creating already-frozen strings. A string like "str"f would have the following characteristics:</p>
<ul>
<li>It would be frozen before the expression returned</li>
<li>It would be the same object everywhere, pulling from a global "fstring" table</li>
</ul>
<p>To avoid memory leaks, these pooled strings would remove themselves from the "fstring" table on GC.</p>
<p>However, there are problems with this new syntax:</p>
<ul>
<li>It will never parse in Ruby 2.0 and earlier.</li>
<li>It's not particularly attractive, though this is a subjective matter.</li>
<li>It does not lend itself well to use in other scenarios, such as for arrays and hashes (<a href="http://bugs.ruby-lang.org/issues/8909" class="external">http://bugs.ruby-lang.org/issues/8909</a> )</li>
</ul>
<p>PROPOSAL:</p>
<p>I propose that we eliminate the new "f" suffix and just make the compiler smart enough to see literal strings with .frozen the same way.</p>
<p>So this code:</p>
<p>str = "mystring".freeze</p>
<p>Would be equivalent in the compiler to this code:</p>
<p>str = "mystring"f</p>
<p>And the fstring table would still be used to return pooled instances.</p>
<p>IMPLEMENTATION NOTES:</p>
<p>The fstring table already exists on master and would be used for these pooled strings. An open question is whether the compiler should forever optimize "str".frozen to return the pooled version or whether it should check (inline-cache style) whether String#freeze has been replaced. I am ok with either, but the best potential comes from ignoring String#freeze redefinitions...or making it impossible to redefine String#freeze.</p>
<p>BONUS BIKESHEDDING:</p>
<p>If we do not want to overload the existing .freeze method in this way, we could follow suggestions in <a href="http://bugs.ruby-lang.org/issues/8977" class="external">http://bugs.ruby-lang.org/issues/8977</a> to add a new "frozen" method (or some other name) that the compiler would understand.</p>
<p>If it were "frozen", the following two lines would be equivalent:</p>
<p>str = "mystring".frozen<br>
str = "mystring"f</p>
<p>In addition, using .frozen on any string would put it in the fstring table and return that pooled version.</p>
<p>I also propose one alternative method name: the unary ~ operator.</p>
<p>There is no ~ on String right now, and it has no meaning for strings that we'd be overriding. So the following two lines would be equivalent:</p>
<p>str = ~"mystring"<br>
str = "mystring"f</p>
<p>JUSTIFICATION:</p>
<p>Making the compiler aware of normal method-based String freezing has the following advantages:</p>
<ul>
<li>It will parse in all versions of Ruby.</li>
<li>It will be equivalent in all versions of Ruby other than the fstring pooling.</li>
<li>It extends neatly to Array and Hash; the compiler can see Array or Hash with literal elements and return the same object.</li>
<li>It does not require a pragma (<a href="http://bugs.ruby-lang.org/issues/8976" class="external">http://bugs.ruby-lang.org/issues/8976</a> )</li>
<li>It looks like Ruby.</li>
</ul> Ruby master - Feature #8960 (Assigned): Add Exception#backtrace_locationshttps://bugs.ruby-lang.org/issues/89602013-09-27T20:11:45Zheadius (Charles Nutter)headius@headius.com
<p>All parties agreed this would be useful to add in <a href="https://bugs.ruby-lang.org/issues/7895" class="external">https://bugs.ruby-lang.org/issues/7895</a> and ko1 suggested I file a feature against ruby-trunk. So here it is.</p>
<p>I might be able to come up with a patch, but I'm not sure when. Help wanted.</p>
<p>Can we agree this will go into 2.1?</p> Ruby master - Feature #8909 (Rejected): Expand "f" frozen suffix to literal arrays and hasheshttps://bugs.ruby-lang.org/issues/89092013-09-14T17:40:25Zheadius (Charles Nutter)headius@headius.com
<p>The "f" suffix to declare a frozen string was recently accepted into 2.1, and I think it's a spectacular addition. I would very much like to see it work for literal arrays and hashes too:</p>
<p>[1, 2, 3, 4, 5]f</p>
<p>{foo: 1, bar: 2, baz: 3}f</p>
<p>There are many, many cases where this could reduce allocation (frozen array with literal elements would only need to be allocated once) and improve thread-safety (explicitly create frozen arrays and hashes when creating structures that might be used across threads).</p>
<p>Is there any reason why we could not do this? I believe both of the above syntaxes would be invalid today, as was the case with the String "f" suffix, and hopefully that means the work to add this syntax would be similar.</p> Ruby master - Bug #8875 (Rejected): Select is not usable with SSLSockethttps://bugs.ruby-lang.org/issues/88752013-09-08T02:17:06Zheadius (Charles Nutter)headius@headius.com
<p>Because of the various levels of buffering SSLSocket employs, it is not possible to reliably use IO.select to check when it has data available.</p>
<p>SSLSocket wraps a normal IO that it uses for reading and writing unencrypted data. This IO has its own buffers, at the OS/libc level.</p>
<p>Select normally operates against IO, checking whether data has been buffered or is available on the wire. However, in order to decrypt data on the wire, SSLSocket often needs to read more data than it needs, potentially draining the stream. This is problem #1.</p>
<p>This problem can be mitigated by making IO.select know that it's an SSLSocket and that it may have its own buffers.</p>
<p>However, there's another layer of buffering that happens in openssl/buffering.rb, where read, readpartial, read_nonblock, and methods that call them eventually hit fill_rbuf, which can easily drain both the IO buffers and the SSLSocket buffers into a Ruby-land buffer IO.select does not know about.</p>
<p>An example script is here: <a href="https://gist.github.com/headius/6477345" class="external">https://gist.github.com/headius/6477345</a></p>
<p>In investigating why this hangs on JRuby (under the original assumption that it was a JRuby issue) I realized that fill_rbuff is reading 16k bytes at a time to try to fill its internal buffer. This effectively drains all data in all buffers visible to IO.select, causing select to hang after the first read.</p>
<p>ruby-head (a few months old), Ruby 1.9.3p253, Ruby 1.8.7p358, JRuby (all versions), and Rubinius (all versions) are affected, because we all share buffering.rb which is where the problem lies.</p>
<p>This may be a known issue, but we continue to get bug reports from Ruby users claiming JRuby is failing to support select + SSLSocket correctly. I'd like to figure out if there's anything we as a community can do to fix this.</p> Ruby master - Feature #8839 (Assigned): Class and module should return the class or module that w...https://bugs.ruby-lang.org/issues/88392013-08-31T02:57:19Zheadius (Charles Nutter)headius@headius.com
<p>With the change for <a href="https://bugs.ruby-lang.org/issues/3753" class="external">https://bugs.ruby-lang.org/issues/3753</a>, "def" forms now return the symbolic name of the method defined. Because class and module bodies just return the last expression in their bodies, this means they will now usually end up returning a symbol for the last method defined. This does not seem useful or correct.</p>
<p>I think class and module should return a reference to the class or module just opened. This would make the return value useful and consistent.</p> Ruby master - Feature #8556 (Rejected): MutexedDelegator as a trivial way to make an object threa...https://bugs.ruby-lang.org/issues/85562013-06-21T22:27:22Zheadius (Charles Nutter)headius@headius.com
<p>I propose adding <code>MutexedDelegator</code> as a simple way to wrap any object with a thread-safe wrapper, via existing delegation logic in <code>delegate.rb</code>.</p>
<p><code>Delegator</code> provides a way to pass method calls through to a wrapped object. <code>SimpleDelegator</code> is a trivial implementation that just holds the object in an instance variable. <code>MutexedDelegator</code> would extend <code>SimpleDelegator</code> and only override initialize and <code>method_missing</code> as follows:</p>
<pre><code> class MutexedDelegator < SimpleDelegator
def initialize(*)
super
@mutex = Mutex.new
end
def method_missing(m, *args, &block)
target, mutex = self.__getobj__, @mutex
begin
mutex.lock
target.__send__(m, *args, &block)
ensure
mutex.unlock
end
end
end
</code></pre>
<p>The only changes here are:</p>
<ul>
<li>
<code>Mutex#lock</code> and <code>unlock</code> logic wrapping the send</li>
<li>No <code>respond_to?</code> check; I'm not sure why it's there to begin with, since if we're in <code>method_missing</code> the <code>super()</code> call will fail just like a normal <code>method_missing</code> failure anyway</li>
<li>No backtrace manipulation. This does not work on JRuby and Rubinius anyway, and in this case I feel that the delegator should not hide itself, since there's real behavior change happening.</li>
</ul>
<p>This is a trivial addition to stdlib that would make it simple to synchronize all calls to a given object in the same way as the JDK's <code>Collections.synchronizedSet</code>/<code>Map</code>/<code>List</code> calls.</p> Ruby master - Bug #8488 (Rejected): Refactor rbinstall.rb for maintenance, clarity, reusehttps://bugs.ruby-lang.org/issues/84882013-06-04T21:27:55Zheadius (Charles Nutter)headius@headius.com
<p>tool/rbinstall.rb is used to install many parts of Ruby at "make install" time, but it has grown to a rather cumbersome size. It also houses some logic that could/should be reused by other implementations, like the default gem installation.</p>
<p>I would like to do some cleanup and refactoring of this file to make it easier to maintain. I'd also like to factor out the default gems logic into a separate piece of code that can be used outside of rbinstall, for e.g. JRuby.</p>
<p>A number of folks have contributed to this file, but kou and nobu seem to be the primary maintainers.</p> Ruby master - Bug #8274 (Closed): No tests for PKCS7::write_smimehttps://bugs.ruby-lang.org/issues/82742013-04-16T14:53:06Zheadius (Charles Nutter)headius@headius.com
<p>There are no tests for OpenSSL::PKCS7::write_smime.</p>
<p>That is all :-)</p>
<p>A recent pull request for JRuby attempts to implement write_smime, but we have no way to confirm it is working correctly (and I do not understand it well enough to implement a test).</p>
<p><a href="https://github.com/jruby/jruby/pull/634" class="external">https://github.com/jruby/jruby/pull/634</a></p> Ruby master - Feature #8271 (Assigned): Proposal for moving to a more visible, formal process for...https://bugs.ruby-lang.org/issues/82712013-04-16T05:35:58Zheadius (Charles Nutter)headius@headius.com
<p>Proposal for moving to a more visible, formal process for feature requests.</p>
<ol>
<li>Introduction</li>
</ol>
<p>In order to make it clear that an issue or change to MRI is a visible feature change all implementations will need to consider implementing, I propose that we move all feature requests into a separate Redmine project. I see the following benefits to this arrangement:</p>
<ul>
<li>Features are always in one place. One-stop-shopping for implementers to track changes to "Ruby" that are not specific to MRI.</li>
<li>No confusion about where feature requests should be filed. Currently, people usually file feature requests against "trunk", but sometimes against version-specific projects. It's also valid to say that a feature improvement or clarification is not specific to trunk. Tracking features separate from "trunk" and version-specific Redmine projects keeps the process localized to one Redmine project.</li>
<li>Ability to add fields to "feature" issues that do not have relevance for "bugs". For example, bugs do not usually need approval from matz, but features could have an "approved by matz" field. We could also have other metadata tracking other implementations, such as "approved by implementations" or "affects implementations" with drop-downs for known impls. One-stop-shopping to know whether a given impl is affected and/or has agreed to add the feature.</li>
<li>More visible process for folks in the community that can't follow the current process or don't believe there's a process in place.</li>
</ul>
<p>I propose that the project be called CommonRuby (already created and under some use) and be a top-level entry on bugs.ruby-lang.org.</p>
<ol start="2">
<li>Processes</li>
</ol>
<p>For issues that are obviously new features (i.e. user knows to select "feature" in the current tracker), issues would be filed directly in CommonRuby. Discussion proceeds exactly as the current process, perhaps with additional issue fields added that allow tracking matz approval, etc, as stated in §1.</p>
<p>Issues that are approved for a Ruby version will have fields/metadata to indicate at which version the feature is available. This may mean specifying multiple releases if, for example, 2.0.1 and 1.9.3p400 would both see a feature added (just saying 1.9.3p400 is insufficient since the feature does not exist in 2.0.0. This avoids having to track features through the backport process to know if there are multiple releases that contain the feature.</p>
<p>For issues that start out as bugs, but later become features or feature changes, those issues would be tranferred into CommonRuby at the point where it's obvious they're feature-related.</p>
<ol start="3">
<li>Detriments</li>
</ol>
<p>Benefits are stated in the introduction above.</p>
<p>Possible detriments with mitigation:</p>
<ul>
<li>Confusion by users about where to file features.</li>
</ul>
<p>This would be mitigated by adding more information to bug-related home pages about the CommonRuby project. The "feature" value in current "trunk" project could either be removed (after migrating features to CommonRuby) or modified to error/warn or switch the issue to CommonRuby programmatically.</p>
<ul>
<li>More complex process.</li>
</ul>
<p>I believe this process is no more complicated than the current process. It also makes the process of evolving "common Ruby" more isolated from MRI development and may make it easier for users to track that evolution.</p>
<ol start="4">
<li>Further justification</li>
</ol>
<p>A lot of noise has been made over the past several months about Ruby lacking a process for new and changing features. The design process proposed by Brian Shirai (née Ford) gained some measure of popularity, but requires a complete retooling and reworking of current processes, making it infeasible for short-term implementation. Other process-change proposals have been kicked around on ruby-core, but the truth is that there <em>is</em> a current process, even if it's not particularly visible. By implementing my proposal, the process would become more obvious and transparent without major impact to MRI's development or Ruby's evolutionary processes.</p>
<ol start="5">
<li>Prior art</li>
</ol>
<p>The PEP (Python Enhancement Proposal) and JSR (Java Specification Request) processes are partial inspiration for this proposal. The latter governs all visible feature changes to Python independent of bug reports to the main "CPython" implementation. The latter governs (through a heavy and overly-strict process) changes to "Java" independent of individual JVM implementations. Both processes have been very successful at isolating spec changes from implementation changes, although the JSR process tends to move very slowly and be less transparent than it should be.</p>
<ol start="6">
<li>Conclusion</li>
</ol>
<p>Ruby does not lack a process for adding or changing features, but it does lack visibility into that process and in many cases fails to provide tools to non-MRI implementations to participate. Moving feature requests and discussion to a CommonRuby project independent of MRI will make the process more transparent and easier to follow (for users and implementers) while having minimal impact on the current process.</p> Ruby master - Feature #8257 (Closed): Exception#cause to carry originating exception along with n...https://bugs.ruby-lang.org/issues/82572013-04-12T01:40:03Zheadius (Charles Nutter)headius@headius.com
<p>Often when a lower-level API raises an exception, we would like to re-raise a different exception specific to our API or library. Currently in Ruby, only our new exception is ever seen by users; the original exception is lost forever, unless the user decides to dig around our library and log it. We need a way to have an exception carry a "cause" along with it.</p>
<p>Java has getCause/setCause and standard constructors that take a cause exception. Printing out an exception's backtrace then reports both that exception and any "cause" exception.</p>
<p>Rubinius has added a similar feature: <a href="https://gist.github.com/dbussink/b2e01e51d0c50b27004f" class="external">https://gist.github.com/dbussink/b2e01e51d0c50b27004f</a></p>
<p>The changes required for this feature are pretty benign:</p>
<ul>
<li>Exception#cause and #cause= accessors.</li>
<li>A new set of Kernel#raise overloads that accept (as a trailing argument, probably) the "cause" exception.</li>
<li>Modifications to backtrace-printing logic to also display backtrace information from the "cause" exception (and in turn, from any nested "cause" exceptions).</li>
</ul>
<p>There's some discussion here about alternatives to #cause, none of which are quite as elegant as having it built in: <a href="http://www.skorks.com/2013/04/ruby-why-u-no-have-nested-exceptions/" class="external">http://www.skorks.com/2013/04/ruby-why-u-no-have-nested-exceptions/</a></p> Backport200 - Backport #8101 (Closed): Backport fix for time.send(:initialize) reinitializing and...https://bugs.ruby-lang.org/issues/81012013-03-16T01:28:26Zheadius (Charles Nutter)headius@headius.com
<p>I filed <a href="https://bugs.ruby-lang.org/issues/8099" class="external">https://bugs.ruby-lang.org/issues/8099</a> to get Time to prevent reinitialization, and nobu fixed it in r39766. I am requesting that it be backported to 1.9.3 along with tests I will be adding shortly.</p> Ruby master - Feature #7701 (Closed): Non-optional (required) keyword argshttps://bugs.ruby-lang.org/issues/77012013-01-16T08:57:32Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
I would like to see keyword args expanded to include a non-optional form, to force callers to pass in keyword arguments.</p>
<p>Currently, we have required, optional, and rest positional args but only optional and rest keyword args. Consistency is one small reason to add required keyword args.</p>
<p>They would likely take the form of keyword with no default value:</p>
<p>def foo(a:, b:)<br>
...<br>
end</p>
<p>foo(a: 1, b: 2) # ok<br>
foo(a: 1) # ArgumentError</p>
<p>Justifications:</p>
<ul>
<li>
<p>Consistency with positional args. A weak justification, I know.</p>
</li>
<li>
<p>Avoiding a lot of boilerplate code by users wishing to enforce keywords being passed in. Example from tenderlove:</p>
<p>def foo(a: raise('pass a'), b: raise('pass b'))</p>
</li>
<li>
<p>Building a rich API atop keyword args would be easier (i.e. require fewer manual checks) if you could force some keywords to be passed in. Having to check everywhere when you require a keyword argument is unpleasant.</p>
</li>
<li>
<p>Keyword args already enforces that no <em>additional</em> keyword args can be passed (without **), and it seems lopsided to have no way to enforce a minimum set of keyword args.<br>
=end</p>
</li>
</ul> Ruby master - Bug #7530 (Closed): Concurrent loads fail with mutex errorshttps://bugs.ruby-lang.org/issues/75302012-12-07T08:21:28Zheadius (Charles Nutter)headius@headius.com
<p>I have no idea what's going on here.</p>
<pre><code>jruby-1.7.0 ~/projects/vts-jruby $ cat bench_load_path.rb
require 'benchmark'
FAKE_PATHS = ARGV[0].to_i || 100
THREADS = 8
ITERATIONS_PER_THREAD = 1000
FAKE_PATHS.times do |i|
$:.unshift "foo#{i}"
end
$: << '.'
system 'touch __load_path_bench_script__.rb'
puts Benchmark.measure {
@threads = THREADS.times.map do
Thread.new {
ITERATIONS_PER_THREAD.times do
require '__load_path_bench_script__'
$".pop
end
}
end
@threads.each { |t| t.join }
}
system 'rm __load_path_bench_script__.rb'
jruby-1.7.0 ~/projects/vts-jruby $ ruby-2.0.0 -rubygems bench_load_path.rb 100
/usr/local/lib/ruby/2.0.0/rubygems/custom_require.rb:36:in `require': wrong argument type false (expected mutex) (TypeError)
from /usr/local/lib/ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:17:in `block (3 levels) in <main>'
jruby-1.7.0 ~/projects/vts-jruby $ ruby-2.0.0 --disable-gems bench_load_path.rb 100
bench_load_path.rb:18:in `require': wrong argument type false (expected mutex) (TypeError)
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:17:in `block (3 levels) in <main>'
With verbose on, I get this error and a bunch of circular require warnings, even though I'm not doing any circular requires here.
ruby-2.0.0-preview2 ~/projects/vts-jruby $ ruby -v bench_load_path.rb
ruby 2.0.0dev (2012-12-01 trunk 38126) [x86_64-darwin11.4.2]
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36: warning: loading in progress, circular require considered harmful - /Users/headius/projects/vts-jruby/__load_path_bench_script__.rb
from bench_load_path.rb:17:in `block (3 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
/Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require': Attempt to unlock a mutex which is locked by another thread (ThreadError)
from /Users/headius/.rvm/rubies/ruby-2.0.0-preview2/lib/ruby/site_ruby/2.0.0/rubygems/custom_require.rb:36:in `require'
from bench_load_path.rb:18:in `block (4 levels) in <main>'
from bench_load_path.rb:17:in `times'
from bench_load_path.rb:17:in `block (3 levels) in <main>'
</code></pre> Ruby master - Bug #7512 (Closed): Test for HMAC signing with UTF-8 Stringhttps://bugs.ruby-lang.org/issues/75122012-12-05T09:55:17Zheadius (Charles Nutter)headius@headius.com
<p>From <a href="https://jira.codehaus.org/browse/JRUBY-7017" class="external">https://jira.codehaus.org/browse/JRUBY-7017</a></p>
<p>Patch: <a href="https://gist.github.com/9bf14142e174891db0dc" class="external">https://gist.github.com/9bf14142e174891db0dc</a></p> Ruby master - Bug #7406 (Closed): English.rb says that $_ is thread-local, but it is frame-localhttps://bugs.ruby-lang.org/issues/74062012-11-20T07:28:41Zheadius (Charles Nutter)headius@headius.com
<p>system ~/projects/jruby/gist-4110634 $ jirb<br>
irb(main):001:0> $_ = 'foo'<br>
=> "foo"<br>
irb(main):002:0> def blah; p $<em>; end<br>
=> nil<br>
irb(main):003:0> p $</em><br>
"foo"<br>
=> "foo"<br>
irb(main):004:0> blah<br>
nil<br>
=> nil</p>
<p>Not thread-local.</p> Ruby master - Bug #7282 (Closed): Invalid UTF-8 from emoji allowed through silentlyhttps://bugs.ruby-lang.org/issues/72822012-11-06T11:51:31Zheadius (Charles Nutter)headius@headius.com
<p>On my system, where the default encoding is UTF-8, the following should not parse:</p>
<p>ruby-2.0.0 -e 'p "Hello, \x96 world!"}'</p>
<p>But it does. And it is apparently marked as "ok" as far as code range goes, because encoding to UTF-8 does not catch the problem:</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e 'p "{"sample": "Hello, \x96 world!"}".encode("UTF-8")'<br>
"{"sample": "Hello, \x96 world!"}"</p>
<p>system ~/projects/jruby $ ruby-2.0.0 -e 'p "{"sample": "Hello, \x96 world!"}".encode("UTF-8")'<br>
"{"sample": "Hello, \x96 world!"}"</p>
<p>Nor does character-walking:</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e '"Hello, \x96 world!".each_char {|x| print x}'<br>
Hello, ? world!<br>
system ~/projects/jruby $ ruby-2.0.0 -e '"Hello, \x96 world!".each_char {|x| print x}'<br>
Hello, ? world!</p>
<p>Nor does []:</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e 'p "Hello, \x96 world!"[7]'<br>
"\x96"</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e 'p "Hello, \x96 world!"[8]'<br>
" "</p>
<p>system ~/projects/jruby $ ruby-2.0.0 -e 'p "Hello, \x96 world!"[7]'<br>
"\x96"</p>
<p>system ~/projects/jruby $ ruby-2.0.0 -e 'p "Hello, \x96 world!"[8]'<br>
" "</p>
<p>But the malformed String does get caught by transcoding to UTF-16:</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e 'p "{"sample": "Hello, \x96 world!"}".encode("UTF-16")'<br>
-e:1:in <code>encode': "\x96" on UTF-8 (Encoding::InvalidByteSequenceError) from -e:1:in </code>'</p>
<p>system ~/projects/jruby $ ruby-2.0.0 -e 'p "{"sample": "Hello, \x96 world!"}".encode("UTF-16")'<br>
-e:1:in <code>encode': "\x96" on UTF-8 (Encoding::InvalidByteSequenceError) from -e:1:in </code>'</p>
<p>Or by doing a simple regexp match:</p>
<p>system ~/projects/jruby $ ruby-1.9.3 -e '"Hello, \x96 world!".match /.+/'<br>
-e:1:in <code>match': invalid byte sequence in UTF-8 (ArgumentError) from -e:1:in </code>match'<br>
from -e:1:in `'</p>
<p>system ~/projects/jruby $ ruby-2.0.0 -e '"Hello, \x96 world!".match /.+/'<br>
-e:1:in <code>match': invalid byte sequence in UTF-8 (ArgumentError) from -e:1:in </code>match'<br>
from -e:1:in `'</p>
<p>And of course I am ignoring the fact that it should never have parsed to begin with.</p>
<p>This kind of inconsistency in rejecting malformed UTF-8 does not inspire a lot of confidence.</p>
<p>JRuby allows it through the parser (this is a bug) but does fail in other places because the string is malformed.</p> Ruby master - Bug #7161 (Closed): Perf fix: use symbols instead of strings for const/ivar access ...https://bugs.ruby-lang.org/issues/71612012-10-15T04:35:29Zheadius (Charles Nutter)headius@headius.com
<p>From pull request: <a href="https://github.com/ruby/ruby/pull/195" class="external">https://github.com/ruby/ruby/pull/195</a></p>
<p>Fixes a number of places where literal, non-dynamic strings are used as the first argument for constant and instance variable get, set, and defined? methods. This reduces object overhead in all cases.</p>
<p>Notable fixes:</p>
<ul>
<li>Setting @original_filename and @content_type in read_multipart (cgi/core.rb). This is called for every multipart read.</li>
<li>Setting <a class="user active user-mention" href="https://bugs.ruby-lang.org/users/8026">@uri (Uri Gorelik)</a> and @ref in DrbObject.new_with (drb/drb.rb). Called for every "with" instantiation of a DrbObject.</li>
<li>Setting BINDING_QUEUE in IRB::Workspace#initialize (irb/workspace.rb). Called for every new IRB workspace.</li>
<li>Getting @mon_mutex in ConditionVariable#wait (monitor.rb). Called for every #wait. This fix makes the method require zero object allocation (other than imposed by the runtime).</li>
</ul>
<p>The other fixes are rarely called, but fixed for consistency.</p>
<p>make test-all passes the same with or without this patch.</p> Backport193 - Backport #7046 (Rejected): ERB#run and ERB#result are not safe for concurrent usehttps://bugs.ruby-lang.org/issues/70462012-09-22T07:59:48Zheadius (Charles Nutter)headius@headius.com
<p>ERB#run and ERB#result both accept an optional binding under which to execute the template. However, if none is given, they both use TOPLEVEL_BINDING by default. Given that by default, the _erbout variable is used for the String into which ERB output gets appended, this causes concurrent template execution on the same thread or separate threads to modify the same buffer. On JRuby, this led to overflow errors when in-progress writes saw their buffers suddenly altered.</p>
<p>This also causes any variables or values evaluated at TOPLEVEL to remain referenced.</p>
<p>I have provided a patch (<a href="https://gist.github.com/3764377" class="external">https://gist.github.com/3764377</a>) that is still very close to the toplevel binding, but instead uses the following logic each call to get a new, isolated binding in which to run the template:</p>
<p>eval "proc{binding}.call", TOPLEVEL_BINDING</p>
<p>This provides visibility to all values at TOPLEVEL, isolates runs to reduce concurrency issues, and guarantees any values stored in the binding will be thrown away after execution.</p>
<p>This fix should be backported to 1.9.3 at minimum.</p> Ruby master - Bug #7037 (Closed): float formatting inconsistently rounds half to evenhttps://bugs.ruby-lang.org/issues/70372012-09-19T16:05:33Zheadius (Charles Nutter)headius@headius.com
<p>MRI does not appear to consistently round half to even. I'm not sure what rounding strategy this is, but it rounds xx05 and xx15 to odd for xxx, and other values to even:</p>
<p>irb(main):001:0> "%1.1f" % 1.05<br>
=> "1.1"<br>
irb(main):002:0> "%1.1f" % 1.15<br>
=> "1.1"<br>
irb(main):003:0> "%1.1f" % 1.25<br>
=> "1.2"<br>
irb(main):004:0> "%1.1f" % 1.35<br>
=> "1.4"<br>
irb(main):005:0> "%1.1f" % 1.45<br>
=> "1.4"<br>
irb(main):006:0> "%1.1f" % 1.55<br>
=> "1.6"</p>
<p>None of the tie-breaking strategies I could find (<a href="http://en.wikipedia.org/wiki/Rounding#Tie-breaking" class="external">http://en.wikipedia.org/wiki/Rounding#Tie-breaking</a>) seem to support MRI's model.</p>
<p>If MRI is indeed using "half even", xx05 should round to xx0 and xx15 should round to xx2. An example with Java's BigDecimal appears to support this:</p>
<p>irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s<br>
=> "1.0"<br>
irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s<br>
=> "1.2"<br>
irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s<br>
=> "1.2"<br>
irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s<br>
=> "1.4"</p>
<p>We would like clarification about the proper rounding tie-breaker strategy to use so we can fix this JRuby issue properly: <a href="http://jira.codehaus.org/browse/JRUBY-6889" class="external">http://jira.codehaus.org/browse/JRUBY-6889</a></p> Ruby master - Feature #7035 (Closed): defined? should return cached, frozen stringshttps://bugs.ruby-lang.org/issues/70352012-09-18T02:57:07Zheadius (Charles Nutter)headius@headius.com
<p>Yehuda and I have been looking into allocation rates in Rails under both MRI and JRuby, and one of the big standouts is defined? logic returning a new String every time (for success cases). We could think of no reason why defined? needs to return a new String, and neither of us know of any code in the wild that takes the resulting string and modifies it.</p>
<p>For systems that use defined? heavily, it would seem best to only ever return the same instance of a cached, frozen String, rather than a new String every time that is only used for its boolean-ness and thrown away. Eliminating these extra Strings would reduce allocation and GC burden on MRI, with only the tiniest behavioral change nobody will ever notice.</p>
<p>An alternative with a larger behavioral change would be to have defined? always return Symbol, rather than String. The additional danger here is if anyone is using the String result as a String for comparison purposes, which I have definitely seen in the wild.</p>
<p>In any case, I could come up with no justification for returning a new String every time, so I have implemented the cached, frozen logic in JRuby: <a href="https://github.com/jruby/jruby/commit/b03d0bc89aefca13deaff7a568e5d9118a9ca2a8" class="external">https://github.com/jruby/jruby/commit/b03d0bc89aefca13deaff7a568e5d9118a9ca2a8</a></p> Ruby master - Feature #6668 (Rejected): Multiple assignment should not return an Array objecthttps://bugs.ruby-lang.org/issues/66682012-06-30T02:11:05Zheadius (Charles Nutter)headius@headius.com
<p>Currently, when doing multiple assignment, the entire expression must return the right-hand side as an array.</p>
<p>system ~ $ ruby -e "ret = (a, b, c = 1, 2, 3); p ret"<br>
[1, 2, 3]</p>
<p>This is an artifact of MRI's implementation, since multiple assignment was traditionally implemented by taking the array node on the right-hand side, standing it up as a full Ruby Array, and then peeling elements off for assignment on the left-hand side. It is also a performance issue, since it requires constructing the RHS array even when it is never used (unless you are able to do various compiler tricks). I propose removing it.</p>
<p>Justification:</p>
<ul>
<li>The feature is rarely used; most people don't even know it exists.</li>
<li>The impact of creating the RHS array is significant; JRuby can optimize it away in cases where the line is not used as an expression, and the performance difference is huge: <a href="https://gist.github.com/3019255" class="external">https://gist.github.com/3019255</a>
</li>
<li>It is counter-intuitive to have an automatic performance hit just from grouping assignments. "a,b = 1,2" should have the exact same performance as "a = 1; b = 2"</li>
</ul>
<p>Note that while JRuby can eliminate the array creation in non-expression cases, those are somewhat rare since many times masgn is used at the end of a method body, as for initializers:</p>
<p>class Foo<br>
def initialize(a, b, c)<br>
<a class="user active user-mention" href="https://bugs.ruby-lang.org/users/52980">@A (A A)</a>, @b, @c = a, b, c<br>
end<br>
end</p>
<p>JRuby and other implementations may get smart enough in our optimizers to eliminate the array in all cases where it's not needed, but this is a very large burden on the optimization subsystem. It may also not be possible to do in all cases (or not possible to do in even a majority of cases).</p>
<p>Multiple assignment should not return RHS as an array. I do not care what it returns.</p> Ruby master - Feature #6648 (Assigned): Provide a standard API for retrieving all command-line fl...https://bugs.ruby-lang.org/issues/66482012-06-26T07:56:37Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal:</p>
<ul>
<li>Scanning globals and hoping they have not been tweaked to new settings</li>
<li>Using external wrappers to launch Ruby</li>
<li>???</li>
</ul>
<p>Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings.</p>
<p>A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line.</p>
<p>JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags.</p>
<p>I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like...</p>
<p>class << RubyVM<br>
def vm_args; end # returns array of command line args <em>not</em> passed to the target script</p>
<pre><code>def script; end # returns the script being executed...though this overlaps with $0
def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args)
</code></pre>
<p>end<br>
=end</p> Ruby master - Feature #6647 (Closed): Exceptions raised in threads should be loggedhttps://bugs.ruby-lang.org/issues/66472012-06-26T07:21:41Zheadius (Charles Nutter)headius@headius.com
<p>Many applications and users I have dealt with have run into bugs due to Ruby's behavior of quietly swallowing exceptions raised in threads. I believe this is a bug, and threads should always at least log exceptions that bubble all the way out and terminate them.</p>
<p>The implementation should be simple, but I'm not yet familiar enough with the MRI codebase to provide a patch. The exception logging should be logged in the same way top-level exceptions get logged, but perhaps with information about the thread that was terminated because of the exception.</p>
<p>Here is a monkey patch that simulates what I'm hoping to achieve with this bug:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="o"><<</span> <span class="no">Thread</span>
<span class="k">alias</span> <span class="n">old_new</span> <span class="n">new</span>
<span class="k">def</span> <span class="nf">new</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="n">old_new</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="k">do</span> <span class="o">|*</span><span class="n">bargs</span><span class="o">|</span>
<span class="k">begin</span>
<span class="n">block</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="o">*</span><span class="n">bargs</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=></span> <span class="n">e</span>
<span class="k">raise</span> <span class="k">if</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">abort_on_exception</span> <span class="o">||</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">abort_on_exception</span>
<span class="nb">puts</span> <span class="s2">"Thread for block </span><span class="si">#{</span><span class="n">block</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2"> terminated with exception: </span><span class="si">#{</span><span class="n">e</span><span class="p">.</span><span class="nf">message</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">puts</span> <span class="n">e</span><span class="p">.</span><span class="nf">backtrace</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span><span class="o">|</span><span class="n">line</span><span class="o">|</span> <span class="s2">" </span><span class="si">#{</span><span class="n">line</span><span class="si">}</span><span class="s2">"</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="mi">1</span> <span class="o">/</span> <span class="mi">0</span> <span class="p">}.</span><span class="nf">join</span>
<span class="nb">puts</span> <span class="s2">"After thread"</span>
</code></pre>
<p>Output:</p>
<pre><code>system ~/projects/jruby $ ruby thread_error.rb
Thread for block #<Proc:0x000000010d008a80@thread_error.rb:17> terminated with exception: divided by 0
thread_error.rb:17:in `/'
thread_error.rb:17
thread_error.rb:7:in `call'
thread_error.rb:7:in `new'
thread_error.rb:5:in `initialize'
thread_error.rb:5:in `old_new'
thread_error.rb:5:in `new'
thread_error.rb:17
After thread
</code></pre> Ruby master - Feature #6590 (Third Party's Issue): Dealing with bigdecimal, etc gems in JRubyhttps://bugs.ruby-lang.org/issues/65902012-06-14T13:21:14Zheadius (Charles Nutter)headius@headius.com
<p>Hello!</p>
<p><a href="http://jira.codehaus.org/browse/JRUBY-6704" class="external">http://jira.codehaus.org/browse/JRUBY-6704</a></p>
<p>We have a need to make the "bigdecimal" gem work for JRuby, so that<br>
distros (like Red Hat, mentioned in the above bug) and users can have<br>
the same gemspec for JRuby with updated bigdecimal gem references.</p>
<p>I'm not sure the best way to proceed.</p>
<p>The bigdecimal source and gemspec all come from MRI source, and are<br>
not versioned separately. That means we can't simply share a<br>
repository for the JRuby bits. We could maintain a forked version in<br>
our forked "jruby/ruby" repository, but that wouldn't be part of the<br>
bigdecimal release cycle then.</p>
<p>So I'm looking to you for guidance.</p>
<p>The BigDecimal lib is here in JRuby:<br>
<a href="https://github.com/jruby/jruby/tree/master/src/org/jruby/ext/bigdecimal" class="external">https://github.com/jruby/jruby/tree/master/src/org/jruby/ext/bigdecimal</a></p>
<p>It might be simplest if for now there's a dummy "bigdecimal" gem<br>
pushed for JRuby that does not contain anything, since we would have<br>
other complications if we tried to remove BigDecimal from JRuby proper<br>
(it is referenced elsewhere int eh code, etc).</p>
<p>Thoughts?</p>
<ul>
<li>Charlie</li>
</ul> Ruby master - Bug #6398 (Closed): Add tests for Zlib.inflate and Zlib.deflatehttps://bugs.ruby-lang.org/issues/63982012-05-04T06:09:39Zheadius (Charles Nutter)headius@headius.com
<p>These just dispatch the same as calling Zlib::Deflate.deflate and Zlib::Inflate.inflate, but the Zlib-bound versions are not tested anywhere. As a result, we missed adding them in JRuby until now.</p>
<p>Patch for test: <a href="https://gist.github.com/2589529" class="external">https://gist.github.com/2589529</a></p>
<p>I request that this be backported to 1.9.3, since JRuby runs the 1.9.3 suite.</p> Ruby master - Feature #6309 (Assigned): Add a reference queue for weak referenceshttps://bugs.ruby-lang.org/issues/63092012-04-17T17:10:15Zheadius (Charles Nutter)headius@headius.com
<p>Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.</p>
<p>A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.</p>
<p>A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: <a href="https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb" class="external">https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb</a></p>
<p>Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.</p>
<p>_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.</p>
<p>It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.</p> Ruby master - Feature #6308 (Assigned): Eliminate delegation from WeakRefhttps://bugs.ruby-lang.org/issues/63082012-04-17T17:02:28Zheadius (Charles Nutter)headius@headius.com
<p>WeakRef's delegation features are a really awful pattern that should not be allowed in future versions of Ruby.</p>
<p>WeakRef makes no guarantees as to the liveness of its contained object. It can be collected at any time if there are no strong references to it.</p>
<p>WeakRef currently uses delegation to pass method calls through to the contained object. This encourages a pattern where a WeakRef is passed to methods that expect to have a reference to the underlying object, making it appear to be that object.</p>
<p>Unfortunately, this is <em>never</em> a good idea. Because the object can be collected at any time, you may get a nil reference from <strong>getobj</strong> <em>arbitrarily</em> in code that tries to call methods against the given WeakRef. That means using WeakRef as a delegate will always result in unreliable code, and errors may happen for inexplicable reasons.</p>
<p>I believe Ruby 2.0 should eliminate WeakRef's delegation features and make it a simple reference holder. There's no safe way to use a weak reference except to grab a reference to the object, check that it is alive (non-nil) and then proceed with the use of the object, as follows:</p>
<p>obj = weakref.<strong>getobj</strong><br>
raise AppropriateError unless obj<br>
obj.do_something<br>
obj.do_something_else</p>
<p>Along with eliminating delegation, I would recommend simply making the get method #get, since the uglier #<strong>getobj</strong> is only named that way because it is not delegated.</p> Ruby master - Feature #6154 (Closed): Eliminate extending WaitReadable/Writable at runtimehttps://bugs.ruby-lang.org/issues/61542012-03-16T13:02:03Zheadius (Charles Nutter)headius@headius.com
<p>The nonblocking IO operations started extending WaitReadable or WaitWritable into the Errno::EAGAIN instance some time during the 1.9 series. This has a rather high cost, since a singleton class must be created and the global method cache must be flushed.</p>
<p>The attached patch instead creates two new classes of the following form, and raises them rather than raising a singleton EAGAIN:</p>
<p>class IO::EAGAINReadable < Errno::EAGAIN<br>
include WaitReadable<br>
end</p>
<p>class IO::EAGAINWritable < Errno::EAGAIN<br>
include WaitWritable<br>
end</p>
<p>The performance of repeatedly doing unsuccessful nonblocking reads improves by about 20%:</p>
<p>BEFORE:</p>
<p>system ~/projects/ruby $ ./ruby2.0.0 -rbenchmark -rsocket -e "sock = TCPSocket.new('localhost', 22); 10.times { puts Benchmark.measure { 100_000.times { begin; sock.read_nonblock(10); rescue IO::WaitReadable; end } } }"<br>
1.210000 0.110000 1.320000 ( 1.328921)<br>
1.220000 0.120000 1.340000 ( 1.326136)<br>
1.220000 0.110000 1.330000 ( 1.334026)<br>
1.230000 0.110000 1.340000 ( 1.349927)<br>
1.310000 0.130000 1.440000 ( 1.426608)<br>
1.210000 0.110000 1.320000 ( 1.333530)<br>
1.220000 0.120000 1.340000 ( 1.330352)<br>
1.230000 0.110000 1.340000 ( 1.350455)<br>
1.220000 0.120000 1.340000 ( 1.327550)<br>
1.220000 0.110000 1.330000 ( 1.337785)</p>
<p>AFTER:</p>
<p>system ~/projects/ruby $ ./ruby2.0.0 -rbenchmark -rsocket -e "sock = TCPSocket.new('localhost', 22); 10.times { puts Benchmark.measure { 100_000.times { begin; sock.read_nonblock(10); rescue IO::WaitReadable; end } } }"<br>
0.980000 0.110000 1.090000 ( 1.092166)<br>
1.010000 0.120000 1.130000 ( 1.129877)<br>
1.090000 0.120000 1.210000 ( 1.202066)<br>
0.960000 0.110000 1.070000 ( 1.076274)<br>
0.970000 0.100000 1.070000 ( 1.078000)<br>
0.970000 0.110000 1.080000 ( 1.078156)<br>
0.970000 0.110000 1.080000 ( 1.078005)<br>
0.970000 0.110000 1.080000 ( 1.078266)<br>
0.980000 0.110000 1.090000 ( 1.093039)<br>
1.000000 0.110000 1.110000 ( 1.112519)</p>
<p>This benchmark does not show the hidden cost of constantly invalidating the global method cache.</p>
<p>I also modified a similar case in OpenSSL, where it previously created an SSLError and extended WaitReadable into it.</p> Ruby master - Bug #6030 (Closed): Thread-local "leak" in rb_exec_recursive*https://bugs.ruby-lang.org/issues/60302012-02-16T03:41:52Zheadius (Charles Nutter)headius@headius.com
<p>I believe there may be a "leak" in the rb_exec_recursive* functions in thread.c.</p>
<p>We have ported these methods for recursion detection in JRuby, and as in MRI they use a map inside a thread-local to track method name + object references. However, none of these method ever clean up the thread local when the recursive walk is complete.</p>
<p>The thread-local data is initialized in thread.c, recursive_list_access, around line 3819 (in 1.9.3 branch):</p>
<pre><code>if (NIL_P(hash) || TYPE(hash) != T_HASH) {
hash = rb_hash_new();
OBJ_UNTRUST(hash);
rb_thread_local_aset(rb_thread_current(), recursive_key, hash);
</code></pre>
<p>As far as I can tell, this thread-local is never cleared, holding a hash reference for as long as the thread is alive.</p> Ruby master - Bug #4301 (Third Party's Issue): Off-by-one line number in Psych parse errorhttps://bugs.ruby-lang.org/issues/43012011-01-21T09:34:58Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
For the following yaml:</p>
<h1>based on "SGML/XML character entity reference" at <a href="http://www.bitjungle.com/isoent/" class="external">http://www.bitjungle.com/isoent/</a>
</h1>
<h1></h1>
<hr>
<p>#DOUBLE LOW-9 QUOTATION MARK<br>
#requires fontenc:T1<br>
ldquor: ,,</p>
<p>Psych produces the following error:</p>
<p>/Users/headius/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/psych.rb:148:in `parse': couldn't parse YAML at line 5 column 9 (Psych::SyntaxError)</p>
<p>The error is at line 6, not line 5.<br>
=end</p> Ruby master - Feature #4288 (Closed): Allow invoking arbitrary method names with foo."something" ...https://bugs.ruby-lang.org/issues/42882011-01-18T14:50:04Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
This is one Groovy feature I tend to like.</p>
<p>For non-standard or arbitrary method names, it would be nice to have a way to invoke them directly that doesn't require the parser to be made more complex nor require an intermediate "send" call. Groovy does this by allowing the following form:</p>
<p>foo."asdf"()</p>
<p>This syntax would make it easier to integrate with other languages that have different naming rules. For example, =!@+= is a valid operator in Scala. With this syntax, you could invoke it as</p>
<p>foo."=!@+=" bar</p>
<p>The alternative in JRuby is that we have to map such names as eq_bang_at_plus_eq, which is certainly not as elegant, or force people to use send (and force them to use :"" anyway, since :=!@+= is not a valid symbol).</p>
<p>It's left up for debate whether string interpolation should be allowed in this syntax.<br>
=end</p> Ruby master - Feature #4265 (Rejected): Provide a core method Kernel#ruby for invoking a new Ruby...https://bugs.ruby-lang.org/issues/42652011-01-12T00:54:47Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Many libraries re-launch Ruby to perform additional tasks. Sometimes this is done for process isolation, sometimes it's to change startup command-line for a subset of work, and sometimes it's just the easiest way to launch a clean Ruby environment. In almost every case, the command line is built using "ruby" explicitly as the command to be executed. This only works if the Ruby version/implementation you want to run is installed as "ruby" and exists on PATH. If, for example, you are running JRuby, you most likely want the subprocess to launch a new JRuby instance. I propose that we add a standard way to launch a new Ruby instance to Kernel.</p>
<p>Kernel#ruby would launch, in an implementation-specific way, a new instance of the <em>current</em> Ruby. On normal C Ruby, this would simply use the full path to the executing "ruby" binary (using appropriate name, like "/usr/bin/ruby1.9" if necessary) and launch it directly, given the specified parameters. On implementations like JRuby, which can launch many instances in the same JVM, the Kernel#ruby method could simple launch a new instance of JRuby. In both cases, it would avoid problems with expecting libraries to build their own command line (and usually getting it wrong).</p>
<p>The method would take, at minimum, a set of command-line parameters to the new instance. These could be in the form of an array of strings, a set of string parameters, or a single string parameter (I don't know which is best). There may be a need for non-command-line options to the #ruby method, so the array or single string might be best (for example, an option to specify whether to attempt to keep the launch in-process, using some form of MVM)</p>
<p>The method would launch Ruby with the given command-line parameters in some implementation-specific way.</p>
<p>The method could either return streams and pid, like p/open/3/4, or simply return the subprocess's output. There could also be a separate #ruby_open to do popen-like stream-driving.</p>
<p>The method should be backported to 1.8.7, ideally, so that libraries could freely start to use it as soon as possible.</p>
<p>Example usages:</p>
<p>ruby "-e 'puts 1'"<br>
in, out, err = ruby "-EUTF-8", mvm:true<br>
=end</p> Ruby master - Feature #4264 (Feedback): General type coercion protocol for Rubyhttps://bugs.ruby-lang.org/issues/42642011-01-12T00:38:10Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Justification: Ruby objects variously define to_ary, to_int and others for either explicit (before call) or implicit (during call) coercion. However, the set of methods is limited, and adding more names just clutters namespaces and isn't feasible long-term anyway. This proposal will hopefully start a discussion toward adding a general type-coercion protocol via a #to or #to_any method. To blunt the naming discussion a bit, I will refer to it as #to_x.</p>
<p>Description: The #to_x method will be a "supermethod" of sorts that can be used to coerce a given object to an arbitrary type. Where currently there are specific methods for coercing to specific types (to_ary, to_str), and other more general methods intended not for coercion but for explicitly re-structuring an object's data (to_a, to_s), there's no one protocol for doing general coercion. #to_x would fill the roles of the coercion methods, accepting a target class and responding appropriately.</p>
<p>The response will depend on whether the target object can be coerced to the given type. The result for success should obviously be an instance of the target type. The result for failure could either be "soft": returning nil, or "hard": raising an error. There could also be an optional boolean flag that specifies hard or soft.</p>
<p>Existing coercion methods could (but need not be) implemented in terms of #to_x</p>
<p>def to_ary<br>
to_x(Array)<br>
end</p>
<p>def to_str<br>
to_x(String)<br>
end</p>
<p>Prior art: JRuby supports coercing Ruby objects to arbitrary Java types in this way. Currently only a set of hard-coded target types are supported for various core Ruby classes, but this is intended to eventually be part of the invocation protocol when calling Java. In other words, if the object being passed is not the exact type of the target parameter, JRuby will invoke to_java(target_param_type) to do the coercion. Performance implications in this are obvious...so there may need to be discussions about modifying this protocol to make it easier to optimize.<br>
=end</p> Ruby master - Bug #4255 (Rejected): When on a case-insensitive filesystem, "loaded features" sear...https://bugs.ruby-lang.org/issues/42552011-01-10T16:45:24Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
This should not re-load the same library when running on a system with case-insensitive filenames:</p>
<p>require 'blah'<br>
require 'BLAH'</p>
<p>The resource in question has been loaded, and should not be reloaded again, even if it is attempted with a differing case.</p>
<p>The above case seems contrived, but we have run into many real-world cases of this problem:</p>
<ul>
<li>win32 APIs return paths with the drive name capitalized. Many libraries that manipulate load paths use lower-case drive names. (C: versus c:)</li>
<li>User code that adds load path entries at command-line or in script can easily differ in case</li>
<li>Typos in path names that introduce case differences will quietly work but cause double-requires. These may or may not be detected.</li>
</ul>
<p>A specific case in JRuby caused a series of files to be loaded twice, ultimately resulting in an alias chain looping back on itself (!!!). This is an extremely difficult case to debug, but very easy to reproduce.</p>
<p>I propose that Ruby 1.9 (and ideally Ruby 1.8, since I consider this a bug and not a feature) should treat "loaded features" $" entries sourced from a case-insensitive filesystem with a case-insensitive comparison.<br>
=end</p> Ruby 1.8 - Bug #4181 (Closed): Backport Ruby 1.9 singleton.rb, since 1.8's is not thread-safehttps://bugs.ruby-lang.org/issues/41812010-12-22T00:04:37Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Ruby 1.9 modified singleton.rb by eliminating much of the lazy init logic, using a real mutex instead of Thread.critical, and eliminating the redefinition of "instance" on first call. None of these changes have been backported into a 1.8 release, which means all 1.8 releases have a broken singleton.rb.</p>
<p>The following script breaks under any version of 1.8:</p>
<code>
require 'singleton'
$jruby = RUBY_PLATFORM =~ /java/
require 'jruby/synchronized' if $jruby
<p>loop do<br>
$inits = []<br>
$inits.extend JRuby::Synchronized if $jruby<br>
classes = []<br>
1000.times do<br>
classes << Class.new do<br>
include Singleton<br>
end<br>
end</p>
<pre><code>(0..10).map do
Thread.new do
classes.each do |cls|
cls.instance
end
end
end.map(&:join)
puts "loop completed"
</code></pre>
<p>end<br>
</p></code>
<p>Results:</p>
<p>~/projects/jruby ➔ ruby -v singleton_killer.rb<br>
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-darwin10.4.0]<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
singleton_killer.rb:18: undefined method <code>instance' for #<Class:0x1001896a0> (NoMethodError) from singleton_killer.rb:1:in </code>join'<br>
from singleton_killer.rb:1:in <code>to_proc' from singleton_killer.rb:21:in </code>map'<br>
from singleton_killer.rb:21<br>
from singleton_killer.rb:5:in `loop'<br>
from singleton_killer.rb:5</p>
<p>~/projects/jruby ➔ ruby -v singleton_killer.rb<br>
ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]<br>
loop completed<br>
loop completed<br>
loop completed<br>
loop completed<br>
singleton_killer.rb:18: undefined method <code>instance' for #<Class:0x100348c70> (NoMethodError) from singleton_killer.rb:1:in </code>join'<br>
from singleton_killer.rb:1:in <code>to_proc' from singleton_killer.rb:21:in </code>map'<br>
from singleton_killer.rb:21<br>
from singleton_killer.rb:5:in `loop'<br>
from singleton_killer.rb:5</p>
<p>This can lead to lazy failures in any library that uses singleton.rb. See also this commit to Nokogiri, where they had to stop using Singleton because of this issue:</p>
<p><a href="https://github.com/tenderlove/nokogiri/commit/5eb036e39ea85a8e12eebee11bc5086b0e4ce6e3" class="external">https://github.com/tenderlove/nokogiri/commit/5eb036e39ea85a8e12eebee11bc5086b0e4ce6e3</a><br>
=end</p> Ruby master - Bug #2121 (Closed): mathn/rational destroys Fixnum#/, Fixnum#quo and Bignum#/, Bign...https://bugs.ruby-lang.org/issues/21212009-09-19T14:07:09Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
I've known this for a while, but only now realized this is actually a terrible bug.</p>
<p>The mathn library replaces Fixnum#/ and Bignum#/ causing them to return a different value. When the result of a division is not an integral value, the default versions will return 0. I can think of many algorithms that would use this expectation, and most other languages will not upconvert integral numeric types to floating-point or fractional types without explicit consent by the programmer.</p>
<p>When requiring 'mathn', Fixnum#/ and Bignum#/ are replaced with versions that return a fractional value ('quo') causing a core math operator to return not just a different type, but <em>a different value</em>.</p>
<p>No core library should be allowed to modify the return value of core numeric operators, or else those operators are worthless; you can't rely on them to return a specific value <em>ever</em> since someone else could require 'mathn' or 'rational'.</p>
<p>Note also that 'rational' destroys Fixnum#quo and Bignum#quo. This is also a bug that should be fixed, though it is less dangerous because they're not commonly-used operators.</p>
<p>The following code should not fail; Fixnum#/ should never return a value of a different magnitude based on which libraries are loaded:</p>
<p>{{{<br>
require 'test/unit'</p>
<p>class TestFixnumMath < Test::Unit::TestCase</p>
<a name="0-to-ensure-it-runs-first-for-illustration-purposes"></a>
<h1 >0 to ensure it runs first, for illustration purposes<a href="#0-to-ensure-it-runs-first-for-illustration-purposes" class="wiki-anchor">¶</a></h1>
<p>def test_0_without_mathn<br>
assert_equal 0, 1/3<br>
end</p>
<p>def test_with_mathn<br>
require 'mathn'<br>
assert_equal 0, 1/3<br>
end<br>
end<br>
}}}</p>
<p>But it does fail:</p>
<p>{{{<br>
~/projects/jruby ➔ ruby test_fixnum_math.rb<br>
Loaded suite test_fixnum_math<br>
Started<br>
.F<br>
Finished in 0.016003 seconds.</p>
<ol>
<li>Failure:<br>
test_with_mathn(TestFixnumMath) [test_fixnum_math.rb:11]:<br>
<0> expected but was<br>
<1/3>.</li>
</ol>
<p>2 tests, 2 assertions, 1 failures, 0 errors<br>
}}}<br>
=end</p> Ruby master - Bug #1152 (Closed): profiler.rb is not concurrent-execution threadsafehttps://bugs.ruby-lang.org/issues/11522009-02-13T07:54:57Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
The library profiler.rb uses class variables to store data without wrapping them in mutexes. On current C Ruby, this may only rarely cause a problem, but as more and more code is allowed to run in parallel it's going to lead to issues. If I or other JRuby community members have a chance, we'll try to make it thread-safe, but I wanted to file this issue to ensure it's out there.</p>
<p>The library is also largely unchanged in 1.9.1.</p>
<p>See <a href="http://jira.codehaus.org/browse/JRUBY-2133" class="external">http://jira.codehaus.org/browse/JRUBY-2133</a> for the (admittedly sparse) JRuby issue.<br>
=end</p> Backport186 - Backport #1151 (Closed): Aliased methods change super logic when retrieved with Obj...https://bugs.ruby-lang.org/issues/11512009-02-13T01:36:58Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
This is a peculiar case I don't believe I've reported before. It seems that "method" can change the super behavior of an alias:</p>
<a name="Test-weird-likely-a-bug-where-method-will-repurpose-where-super-goes-to"></a>
<h1 >Test weird likely-a-bug where method() will repurpose where super goes to<a href="#Test-weird-likely-a-bug-where-method-will-repurpose-where-super-goes-to" class="wiki-anchor">¶</a></h1>
<p>class Foo222<br>
def a; 'a'; end<br>
def b; 'b'; end<br>
end</p>
<p>class Bar222 < Foo222<br>
def a; super; end<br>
alias b a<br>
end</p>
<p>puts('a' == Bar222.new.b) # true<br>
puts('a' == Bar222.new.method(:b).call) # false</p>
<p>Ruby 1.9 behaves as you would expect, calling the "a" super method in both cases. We changed our behavior in JRuby 1.1.2 to match Ruby 1.8.6, but I still believe this is a bug. The JRuby bug report is here: <a href="http://jira.codehaus.org/browse/JRUBY-1192" class="external">http://jira.codehaus.org/browse/JRUBY-1192</a> and I reported it to ruby-core here: <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/11600" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/11600</a> and a patch was proposed here: <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/11603" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/11603</a>. Since it behaves as I expect in 1.9, I assume the 1.8.x behavior is incorrect.<br>
=end</p> Ruby 1.8 - Bug #1034 (Closed): Ruby 1.8 evaluates block argument out of order from other argument...https://bugs.ruby-lang.org/issues/10342009-01-22T03:46:48Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
In Ruby 1.9 and all alternative impls I tested, block arguments are evaluated in the order they're encountered in a call. For example:</p>
<p>a.foo(b, c, &d)</p>
<p>The expected order would be left to right, a, then b, then c, then d. But Ruby 1.8 evaluates in the order d, a, b, c. In a case like the following (somewhat contrived) this would have odd side effects:</p>
<p>x = 0<br>
(x += 1; a).foo(x += 1, x += 1, &(x += 1, d))</p>
<p>In 1.8, the two non-block arguments would be 3, 4, while on Ruby 1.9 and other impls it would be 2, 3 as you'd expect.<br>
=end</p> Ruby master - Bug #967 (Closed): Results of const_missing are being cached incorrectlyhttps://bugs.ruby-lang.org/issues/9672009-01-02T15:05:45Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Ruby 1.9 sped up constant lookup by adding an inline cache, cleared by a global serial number incremented when any constant is set. This works well for normal constants, but unfortunately 1.9 is also caching the result of const_missing. For example, the following output:</p>
<p>◆ ruby1.9 -e "def Object.const_missing(sym); Time.now; end; 5.times { p XXX; sleep 1 }"<br>
2009-01-01 23:59:33 -0600<br>
2009-01-01 23:59:33 -0600<br>
2009-01-01 23:59:33 -0600<br>
2009-01-01 23:59:33 -0600<br>
2009-01-01 23:59:33 -0600</p>
<p>The result of const_missing is being cached at the lookup site for XXX. Here's what the output should look like, with incrementing seconds:</p>
<p>◆ ruby -e "def Object.const_missing(sym); Time.now; end; 5.times { p XXX; sleep 1 }"<br>
Fri Jan 02 00:02:09 -0600 2009<br>
Fri Jan 02 00:02:10 -0600 2009<br>
Fri Jan 02 00:02:11 -0600 2009<br>
Fri Jan 02 00:02:12 -0600 2009<br>
Fri Jan 02 00:02:13 -0600 2009</p>
<p>Note that this also affects colon2 and colon3 constant lookup:</p>
<p>◆ ruby1.9 -e "def Object.const_missing(sym); cls = Class.new; cls.class_eval 'Foo = Time.now'; cls; end; 5.times { p XXX::Foo; sleep 1 }"<br>
2009-01-02 00:02:44 -0600<br>
2009-01-02 00:02:44 -0600<br>
2009-01-02 00:02:44 -0600<br>
2009-01-02 00:02:44 -0600<br>
2009-01-02 00:02:44 -0600</p>
<p>◆ ruby1.9 -e "def Object.const_missing(sym); Time.now; end; 5.times { p ::XXX; sleep 1 }"<br>
2009-01-02 00:03:06 -0600<br>
2009-01-02 00:03:06 -0600<br>
2009-01-02 00:03:06 -0600<br>
2009-01-02 00:03:06 -0600<br>
2009-01-02 00:03:06 -0600</p>
<p>The result of const_missing should never be cached, since doing so would break code that expects const_missing to keep firing (some DSLs for example).</p>
<p>This was run against the just-released Ruby 1.9 RC.<br>
=end</p> Ruby master - Bug #921 (Closed): autoload is not thread-safehttps://bugs.ruby-lang.org/issues/9212008-12-23T02:19:07Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Currently autoload is not safe to use in a multi-threaded application. To put it more bluntly, it's broken.</p>
<p>The current logic for autoload is as follows:</p>
<ol>
<li>A special object is inserted into the target constant table, used as a marker for autoloading</li>
<li>When that constant is looked up, the marker is found and triggers autoloading</li>
<li>The marker is first removed, so the constant now appears to be undefined if retrieved concurrently</li>
<li>The associated autoload resource is required, and presumably redefines the constant in question</li>
<li>The constant lookup, upon completion of autoload, looks up the constant again and either returns its new value or proceeds with normal constant resolution</li>
</ol>
<p>The problem arises when two or more threads try to access the constant. Because autoload is stateful and unsynchronized, the second thread may encounter the constant table in any number of states:</p>
<ol>
<li>It may see the autoload has not yet fired, if the first thread has encountered the marker but not yet removed it. It would then proceed along the same autoload path, requiring the same file a second time.</li>
<li>It may not find an autoload marker, and assume the constant does not exist.</li>
<li>It may see the eventual constant the autoload was intended to define.</li>
</ol>
<p>Of these combinations, (3) is obviously the desired behavior. (1) can only happen on native-threaded implementations that do not have a global interpreter lock, since it requires concurrency during autoload's internal logic. (2) can happen on any implementation, since while the required file is processing the original autoload constant appears to be undefined.</p>
<p>I have only come up with two solutions:</p>
<ul>
<li>When the autoload marker is encountered, it is replaced (under lock) with an "autoload in progress" marker. All subsequent threads will then see this marker and wait for the autoloading process to complete. the mechanics of this are a little tricky, but it would guarantee concurrent autoloads would only load the target file once and would always return the intended value to concurrent readers.</li>
<li>A single autoload mutex, forcing all autoloads to happen in serial.</li>
</ul>
<p>There is a potential for deadlock in the first solution, unfortunately, since two threads autoloading two constants with circular autoloaded constant dependencies would ultimately deadlock, each waiting for the other to complete. Because of this, a single autoload mutex for all autoloads may be the only safe solution.<br>
=end</p> Ruby master - Feature #905 (Closed): Add String.new(fixnum) to preallocate large bufferhttps://bugs.ruby-lang.org/issues/9052008-12-19T08:54:14Zheadius (Charles Nutter)headius@headius.com
<p>=begin<br>
Because Strings are used in ruby as arbitrary byte buffers, and because the cost of growing a String increases as it gets larger (especially when it starts small), String.new should support a form that takes a fixnum and ensures the backing store will have at least that much room. This is analogous to Array.new(fixnum) which does the same thing.</p>
<p>The simple implementation of this would just add a Fixnum check to the String.new method, and the result would be an empty string with that size buffer. This would allow heavy string-appending algorithms and libraries (like ERb) to avoid doing so many memory copies while they run.<br>
=end</p>