Thoughts around Frame Introspection and Debuggers

Goals

  • Make it possible to write a faster, better (introspecting) debugger.
  • Expose the VM a little more so folks can see what's going on such as to perform different kinds of code analysis or code improvement

So what's wrong with debug.rb?

  • It's slow.
  • And sometimes it just doesn't work...

and why is that?

  • Code has to have line-event trace instructions compiled in which means arranging to do so in advance. This also makes the code bulkier, uglier, (and potentially slower)
  • In order be able to evaluate expressions in the context of the debugged program, the debugger needs to save extra information (bindings) on calls and this is an expensive operation. A better approach would be to allow some sort of introspection to get binding (and Proc) information

Approach(es)

  • Add more Ruby introspection routines into Ruby 1.9
    • Advantages:
    • it is needed :-)
    • it standardizes what is desirable across all Ruby 1.9 implementations
    • Disadvantages:
    • the API must not be too implementation specific.
    • it is harder to get consensus across implementers. In particular, one might define a minimal kind of Frame class extended off of the Thread class, e.g. Thread::Frame with things like a method to return a binding and another to access the previous frame on the call stack.
  • Expose more of the C API, allowing people to get at more of the VM internals
    • Advantages:
    • it's needed :-) Note that even if Ruby 1.9 introspection routines are added, this generally goes with corresponding C access routines
    • More implementation-specific routines can be allowed
    • Disadvantage
    • This ties ko1 and the VM design down more as people start depending on the API. Alternatively it may require programmers using the API to change their code to match with VM changes.
  • Levels of Publicness:
    1. Ruby 1.9 methods, variables, classes. This is the most public form.
    2. Installed C headers off of <ruby.h> which contains C macros, extern's, typedefs
    3. Opaque C typedefs: this allows C methods to be declared, but hides internals of a struct
    4. No headers, but extern'd (rather than static) functions, variables in YARV which allow C extensions to duplicate the internals as needed. This is the least public form. Invariably some changes to use code may be required as the VM and not-generally-public API changes.

Proposed plan.

  • Get as much in the generic API agreed upon.
  • Make the smallest number of changes to the VM core that will allow the API to happen.
  • Implement the API (and possibly more experimental ideas) as a gem (with a C extension) outside of the Ruby 1.9 core. By doing this the release cycles of the experimental extension are not tied to Ruby. It is contemplated that the experimental extensions will have a faster release cycle, and not be depended on as critically. So this code might have more bugs.

Miscellaneous: debugger for Ruby 1.9, tracing, dtrace

See http://wiki.rubyonrails.org/gsoc/2009/ideas/yarv-debugger

Sorry there's a little duplication between that and this. But see the wiki specifically for some detail on these important aspects:

  • Instruction trap replacement.
  • using DBGp as a common protocol for interfacing with front ends. A couple of patches have been proposed to extend set_trace_func to allow an event bit mask for finer-grained control on the calling side for which kind of events to call. See http://www.ruby-forum.com/attachment/3511/trace-mask.patch

And finally: