Bug #9580: Refinements regression in IRB

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 2.2.0dev
• Tags: maybenotbug, refine

[davidbalbert (David Albert)]

The problem: Top level refinements do not work in IRB. They worked in 2.0.0-p451, but don't work in 2.1.0, 2.1.1, or today's trunk.

Details:

Here some code in a file:

#refine.rb

module A
  refine String do
    def asdf
      :asdf
    end
  end
end

using A

p "foo".asdf

In all versions, of Ruby between 2.0.0-p451 and 2.2.0dev, running this file, prints :asdf. This is the expected behavior. Ruby 2.0.0 also prints the "Refinements are experimental" warning, as expected:

# Ruby 2.0.0-p451
$ ruby refine.rb 
refine.rb:2: warning: Refinements are experimental, and the behavior may change in future versions of Ruby!
:asdf

# Ruby 2.1.0, 2.1.1, and 2.2.0dev
$ ruby refine.rb 
:asdf

In Ruby 2.0.0-p451, the same code also works in IRB, also as expected:

irb(main):001:0> "#{RUBY_VERSION}-#{RUBY_PATCHLEVEL}"
=> "2.0.0-451"
irb(main):002:0> module A
irb(main):003:1>   refine String do
irb(main):004:2*     def asdf
irb(main):005:3>       :asdf
irb(main):006:3>     end
irb(main):007:2>   end
irb(main):008:1> end
(irb):3: warning: Refinements are experimental, and the behavior may change in future versions of Ruby!
=> #<refinement:String@A>
irb(main):009:0> using A
=> main
irb(main):010:0> "foo".asdf
=> :asdf

However, in all newer versions of Ruby (2.1.0, 2.1.1, and 2.2.0dev), this code raises a NoMethodError in IRB:

irb(main):001:0> RUBY_VERSION
=> "2.1.0"
irb(main):002:0> module A
irb(main):003:1>   refine String do
irb(main):004:2*     def asdf
irb(main):005:3>       :asdf
irb(main):006:3>     end
irb(main):007:2>   end
irb(main):008:1> end
=> #<refinement:String@A>
irb(main):009:0> using A
=> main
irb(main):010:0> "foo".asdf
NoMethodError: undefined method `asdf' for "foo":String
	from (irb):10
	from out/bin/irb:11:in `<main>'


irb(main):001:0> RUBY_VERSION
=> "2.1.1"
irb(main):002:0> module A
irb(main):003:1>   refine String do
irb(main):004:2*     def asdf
irb(main):005:3>       :asdf
irb(main):006:3>     end
irb(main):007:2>   end
irb(main):008:1> end
=> #<refinement:String@A>
irb(main):009:0> using A
=> main
irb(main):010:0> "foo".asdf
NoMethodError: undefined method `asdf' for "foo":String
	from (irb):10
	from bin/irb:11:in `<main>'


irb(main):001:0> RUBY_VERSION
=> "2.2.0"
irb(main):002:0> module A
irb(main):003:1>   refine String do
irb(main):004:2*     def asdf
irb(main):005:3>       :asdf
irb(main):006:3>     end
irb(main):007:2>   end
irb(main):008:1> end
=> #<refinement:String@A>
irb(main):009:0> using A
=> main
irb(main):010:0> "foo".asdf
NoMethodError: undefined method `asdf' for "foo":String
	from (irb):10
	from bin/irb:11:in `<main>'

This seems like a bug because the code behaves differently in IRB than how it behaves in the file. If it's the intended behavior, it's frustrating because it makes it harder to prototype code that uses refinements in the REPL.

This issue is not specific to IRB. I get the same behavior in Pry (works in 2.0.0, doesn't work in newer Ruby versions). This makes me think the issue is not inside the IRB source, but rather has something to do with using's behavior in the Binding objects that IRB and Pry are probably using. I haven't looked at the Pry or IRB source in quite a long time, so this paragraph is mostly speculation.

Please let me know if there's any more info I can provide to make this easier to fix

[shugo (Shugo Maeda)]

David Albert wrote:

This seems like a bug because the code behaves differently in IRB than how it behaves in the file. If it's the intended behavior, it's frustrating because it makes it harder to prototype code that uses refinements in the REPL.

This change was intentionally introduced in r42396.
The following code illustrates why using doesn't work on IRB.

module Foo
  refine Object do
    def foo
      puts "foo"
    end
  end
end

eval("using Foo; foo") # Foo is activated only in the given string
eval("foo")       # => NoMethodError

I understand it's not convenient for prototyping.
How about to add a new option for using to activate refinements globally?

eval("using Foo, global: true")
eval("foo") #=> foo

This got me puzzled for a couple minutes. Is there a good reason why irb shouldn't have 'using' working the same way normal env works?

[davidbalbert (David Albert)]

I'm embarrassed that I missed these responses to my bug report. I had forgotten that I had created this issue. Sorry!

Shugo: using Foo, global: true would fix the problem, though it feels a bit awkward to me. I guess this is because an IRB session feels like it should be one continuous lexical scope, even though I realize that this isn't how things work in practice.

Perhaps we could add an instance method to Binding that would allow refinements to stay active across multiple calls to eval for that binding instance? It might look something like this:

module Foo
  refine Object do
    def foo
      puts "foo"
    end
  end
end

b = binding

# Normal behavior:
b.eval("using Foo")
b.eval("foo") #=> NoMethodError

# Special behavior, useful for building REPLs:
b.retain_refinements!
b.eval("using Foo")
b.eval("foo") #=> foo

Binding#retain_refinements!, could be made to apply only to refinements activated with main.using (top_using in eval.c) rather than Module#using (mod_using in eval.c). Maybe retain_toplevel_refinements! would be a better name for the method. I'm not sure.

If this was implemented, IRB could call Binding#retain_refinements! on the binding used for its session in lib/irb/workspace.rb and IRB would behave as I originally expected.

At this point, I realize that I'm talking about a design change, not a bug fix, which I know is a bigger deal. Which of the proposed solutions is better (using(mod, global: true) vs Binding#retain_refinements!) depends on who you think should be responsible for knowing about the behavior. If you think it should be the user of the REPL, then the first solution makes sense. If you think it should be the author of the REPL, then the second solution makes sense.

I think this is something that the user of the REPL shouldn't have to know about because it's a pretty small edge case, and it has surprising results for a casual Ruby user who hasn't thought about how IRB is implemented. It was even confusing for me, and at the time that I created this issue, I had some experience reading through the source of both IRB and Pry. That said, I do see the merits of the other solution and hope that one of them gets accepted, even if it isn't the one I prefer.

Thanks!
David

[kernigh (George Koehler)]

• Backport deleted (1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: UNKNOWN)

In ruby 2.4.0dev, top-level refinements still don't work in IRB. Now I get a RuntimeError when using them:

$ cat big.rb
module Big
  refine(Numeric) do
    def hundred
      self * 100
    end
  end
end
$ ruby -v
ruby 2.4.0dev (2016-05-19 trunk 55066) [x86_64-openbsd5.9]
$ irb -I. -rbig
irb(main):001:0> using Big
RuntimeError: main.using is permitted only at toplevel
        from (irb):1:in `using'
        from (irb):1
        from /home/kernigh/prefix/bin/irb:11:in `<main>'

This is a problem with the Binding object that IRB uses to evaluate my code. If I create my own Binding object in a place that allows using refinements, then it works:

irb(main):002:0> b = module Dummy; binding; end
=> #<Binding:0x001779c844dc80>
irb(main):003:0> b.eval('using Big')
=> Dummy
irb(main):004:0> b.eval('5.hundred')
=> 500

I looked at IRB's code to check how it creates the binding. I noticed that IRB has an option --context-mode or IRB.conf[:CONTEXT_MODE]. The default is --context-mode=3 where refinements don't work. If I change it to 0, 1, or 2, then refinements work:

$ irb -I. -rbig --context-mode=1
irb(main):001:0> using Big
=> main
irb(main):002:0> 5.hundred
=> 500

When --context-mode=0 then the backtraces from exceptions are too long. (Type fail at IRB prompt to see the backtrace.) So I prefer to use --context-mode=1 because it has working refinements and short backtraces. I now put IRB.conf[:CONTEXT_MODE] = 1 in my ~/.irbrc for working refinements.

The bug here is that IRB doesn't allow using refinements by default. We should change the default context mode to allow using refinements.

[marcandre (Marc-Andre Lafortune)]

I ran into this today.

We really should change the default to something that works for refinements. I note that:

irb --single-irb # => works (shared context for nested sessions)
irb --context 0 # => works
irb --context 1 # => works
irb --context 2 # => works
irb --context 3 # (default) => does not work

I get the same stacktraces in all modes in Ruby 2.7.

I am proposing a new default context in https://github.com/ruby/irb/pull/138 but any other context mode seems better than the current one.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

The change of the default irb --context-mode from 3 to 4 has fixed this issue (though it caused #17623).