Bug #16983


RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval

Added by pocke (Masataka Kuwabara) almost 4 years ago. Updated over 2 years ago.

Target version:
ruby -v:
ruby 2.8.0dev (2020-06-23T13:58:26Z master dc351ff984) [x86_64-linux]



RubyVM::AST.of(method) returns a meaningless node if the method is defined in eval.
For example:

p 'blah'

eval <<~RUBY, binding, __FILE__, __LINE__ + 1
  def foo

method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")

I expect the node of foo method, or nil. But it returns a STR node.

It becomes a big problem when AST.of receives arbitrary methods.
Because we can't distinguish a method is defined in eval or not.
It means we can't believe the returned value of AST.of if the method may receive a method defined in eval.

For example:

def do_something_for_each_method_ast(klass)
  klass.instance_methods(false).each do |m|
    ast = RubyVM::AbstractSyntaxTree.of(klass.instance_method(m))
    next unless ast

    do_something ast

class A
  eval <<~RUBY, binding, __FILE__, __LINE__ + 1
    def foo

do_something_for_each_method_ast A

In the example, I expect the do_something method receives only node for a method definition,
but it may pass a wrong node if any method is defined in eval.

Cause (I guess)

I guess the cause is misleading node number.
In and out of an eval block uses different sequences of node number.
So if I specify __FILE__ to eval, the actual file and code in eval may have the same node number.

For example

p 'blah' # Node number for 'blah' is 1, file name is "test.rb"

eval <<~RUBY, binding, __FILE__, __LINE__ + 1
  def foo # Node number for `def` is also 1, file name is also "test.rb"

method = method(:foo)
# It finds a node from node number 1 by reading "test.rb", so it get the str node.
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")

Updated by jeremyevans0 (Jeremy Evans) over 3 years ago

I'm not sure if this is a bug, but it does seem like a fundamental and significant limitation with the design of RubyVM::AbstractSyntaxTree.of. RubyVM::AbstractSyntaxTree.of reparses the file the method is defined in and cannot handle any cases where eval or similar are used. You'll get a node completely different from what you would expect. Here's another example:

eval, binding, __FILE__, 14
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)

  def foo


(VCALL@1:16-1:23 :binding)

Because it reparses the file, you'll also get the wrong result if the file is modified:

def bar

File.write(__FILE__,'def bar', "def foo\nbar"))

method = method(:bar)
pp RubyVM::AbstractSyntaxTree.of(method)


 tbl: []
    pre_num: 0
    pre_init: nil
    opt: nil
    first_post: nil
    post_num: 0
    post_init: nil
    rest: nil
    kw: nil
    kwrest: nil
    block: nil)
 body: (VCALL@2:0-2:3 :bar))

And if the interpreter can no longer access the file (chroot, file deletion, permission change, or other file system access limiting), you get an error.

I can't think of a way to fix this without all iseq methods holding a reference to the string used to parse them, and having RubyVM::AbstractSyntaxTree.of work off that string. I'm not sure how much extra memory use that would cause, or if such an approach is considered acceptable.

Updated by mame (Yusuke Endoh) almost 3 years ago

  • Status changed from Open to Assigned
  • Assignee set to ko1 (Koichi Sasada)

This ticket was discussed on dev-meeting. A method (or proc) created in an eval context should be marked and AST.of should raise an exception against a marked method. @ko1 (Koichi Sasada) said that he will do.

Updated by jeremyevans0 (Jeremy Evans) almost 3 years ago

Because I know how busy @ko1 (Koichi Sasada) is, I thought I'd save him some work and tried to implement this myself:

Actions #4

Updated by jeremyevans (Jeremy Evans) over 2 years ago

  • Status changed from Assigned to Closed

Applied in changeset git|64ac984129a7a4645efe5ac57c168ef880b479b2.

Make RubyVM::AbstractSyntaxTree.of raise for method/proc created in eval

This changes Thread::Location::Backtrace#absolute_path to return
nil for methods/procs defined in eval. If the realpath of an iseq
is nil, that indicates it was defined in eval, in which case you
cannot use RubyVM::AbstractSyntaxTree.of.

Fixes [Bug #16983]

Co-authored-by: Koichi Sasada


Also available in: Atom PDF