Feature #7895

Exception#backtrace_locations to go with Thread#backtrace_locations and Kernel#caller_locations

Added by Charles Nutter about 1 year ago. Updated 7 months ago.

[ruby-core:52596]
Status:Open
Priority:Normal
Assignee:-
Category:-
Target version:Ruby 2.1.0

Description

Thread#backtracelocations and Kernel#callerlocations were added in Ruby 2.0.0, but no equivalent method was added to Exception to get backtrace locations. The String format of Exception#backtrace elements makes it difficult to do any inspection of the actual files and lines it reports, so having backtrace_locations would be very useful.

In #7435 ko1 pointed out that Exception defines setbacktrace, which takes an array (presumably of Strings in the same format as Exception#backtrace) and that this would need to be addressed in order to support backtracelocations. I propose that if you setbacktrace, you are already breaking the ability to get structured Location elements, so setbacktrace should cause backtrace_locations to return nil or an empty array (probably an empty array, so it can always be expected to be non-nil).

We could consider also adding setbacktracelocations as a way to set Location elements into the exception.

An example script where backtrace_locations would be very useful is here (avoiding the need to regexp match to get file and line number): https://gist.github.com/headius/4999244


Related issues

Related to ruby-trunk - Feature #8960: Add Exception#backtrace_locations Feedback 09/27/2013

History

#1 Updated by Koichi Sasada about 1 year ago

(2013/02/21 6:02), headius (Charles Nutter) wrote:

I propose that if you setbacktrace, you are already breaking the ability to get structured Location elements, so setbacktrace should cause backtrace_locations to return nil or an empty array (probably an empty array, so it can always be expected to be non-nil).

We could consider also adding setbacktracelocations as a way to set Location elements into the exception.

I want to agree with you. But I feel there are worries.

(a) Exception#set_backtrace are used by frameworks

Maybe Exception#setbacktrace are used by frameworks such as Rails. So
that your proposal "if Exception#set
backtrace was called, disable
Exception#backtracelocations" seems bad idea because 99% of people
works on Rails (1) and 99% (1) of people can't use
Exception#backtrace
locations.

*1: joking.

(b) We can't use Exception#backtrace_locations in library

If you write a library using Exception#backtracelocations, this library
can't use with this library with programs using Exception#set
backtrace.
As I pointed out by (a), we can't mix with frameworks.


Other options:

(1) Make Exception#set_backtrace parse each backtrace lines.
Problem:
(1-p1) Parsing is difficult. Path format is OS dependent.
(1-p2) How to parse not well-formed line?

(2) Deprecate setbacktrace and replace it with setbacktrace_locations
Problem:
(2-p1) Compatibility!!!

(1) with heuristics parsing technique is pragmatic?

--
// SASADA Koichi at atdot dot net

#2 Updated by Charles Nutter about 1 year ago

On Thu, Feb 21, 2013 at 8:36 AM, SASADA Koichi ko1@atdot.net wrote:

I want to agree with you. But I feel there are worries.

How about #backtrace/#setbacktrace and
#backtrace
locations/#setbacktracelocations are completely
separate representations of the backtrace? That preserves backward
compatibility while providing a path forward to the more structured
backtrace form. I think it's reasonable to expect that the
unstructured backtrace form doesn't update or break the structured
form.

  • Charlie

#3 Updated by Koichi Sasada about 1 year ago

(2013/02/21 8:50), Charles Oliver Nutter wrote:

On Thu, Feb 21, 2013 at 8:36 AM, SASADA Koichi ko1@atdot.net wrote:

I want to agree with you. But I feel there are worries.

How about #backtrace/#setbacktrace and
#backtrace
locations/#setbacktracelocations are completely
separate representations of the backtrace? That preserves backward
compatibility while providing a path forward to the more structured
backtrace form. I think it's reasonable to expect that the
unstructured backtrace form doesn't update or break the structured
form.

I think it is reasonable.

(3) Separate Exception#bactrace and Exception#backtrace_locations
Problems:
(3-p1) which should output as an error backtrace when interpreter dies.
(3-p2) may introduce confusion?

Other comments and ideas are welcome.

--
// SASADA Koichi at atdot dot net

#4 Updated by Charles Nutter about 1 year ago

On Thu, Feb 21, 2013 at 5:47 PM, SASADA Koichi ko1@atdot.net wrote:

I think it is reasonable.

(3) Separate Exception#bactrace and Exception#backtrace_locations
Problems:
(3-p1) which should output as an error backtrace when interpreter dies.

Good question! I was wondering about this too. Does #backtrace remain
the canonical stack trace source, or do we move to
#backtrace_locations? Perhaps there's a heuristic necessary here to
support forward migration. To be honest, the ability to replace the
stack trace has always bothered me a bit, since it means a program
could potentially hide where an error occurs. I feel like if the
exception bubbles all the way out, you should get a proper trace. I'm
probably missing the use case where you want to provide a customized
trace, though.

(3-p2) may introduce confusion?

I think this is unavoidable, to some extent. We would like to migrate
people toward using the structured backtrace, but it is not
backward-compatible with the unstructured backtrace. So...we make a
decision: prioritize the structured version or prioritize the
unstructured version? I tend toward the ideal future and would
emphasize the structured version as much as possible...meaning that
interpreter death uses backtracelocations, unless setbacktrace has
been used to provide the old-style trace. As I say above, there's
probably a reasonable heuristic possible here based on what we want to
encourage people to use.

Thinking through this a bit...

  • Base all backtrace output on the backtrace_locations content unless otherwise indicated.
  • If you specify a backtrace at construction time, inspect whether it is string content (old style) or Location content (new style).
  • Any modification of the string-based #backtrace overrides
    #backtracelocations for printing purposes. If #backtrace has not been
    set, either via constructor or via #set
    backtrace, then we generate it
    based on #backtrace_locations.

    Basically, #backtracelocations is the primary source of backtrace
    data, unless the user provides an old-style #backtrace or uses
    #set
    backtrace to do something custom.

    Of course #setbacktracelocations should enforce that the array
    provided is actually Location objects.

  • Charlie

#5 Updated by Charles Nutter about 1 year ago

What's next here? I'm about to implement caller_locations in JRuby, so I'm close to being able to prototype this feature request too.

#6 Updated by Koichi Sasada about 1 year ago

(2013/03/09 17:08), headius (Charles Nutter) wrote:

What's next here? I'm about to implement caller_locations in JRuby, so I'm close to being able to prototype this feature request too.

Now, only two attendees (you and I).
I want to get other opinions, especially users of #set_backtrace().

--
// SASADA Koichi at atdot dot net

#7 Updated by Charles Nutter about 1 year ago

For now I have implemented it in JRuby.

Here's the commit: https://github.com/jruby/jruby/commit/281ead709e1e855b552a42e86e48f8f91ca8ebb5

It works simply enough; Exception#backtrace_locations always reflects the original exception trace and (at least for now) it can't be modified.

irb(main):004:0> def foo; raise; end
=> nil
irb(main):005:0> def bar; 1.times { foo }; end
=> nil
irb(main):006:0> ex = begin; bar; rescue; $!; end
=> RuntimeError
irb(main):007:0> ex.backtrace_locations.each {|loc| puts "method: #{loc.label}, line: #{loc.lineno}"}
method: foo, line: 4
method: bar, line: 5
method: times, line: 271
method: bar, line: 5
method: evaluate, line: 6

#8 Updated by Daniel DeLeo about 1 year ago

Hi, I work on Opscode's Chef, a server configuration framework. I have an interest in both parts of this topic. In Chef, we use set_backtrace but we also parse backtrace lines to find relevant code in our error messages.

When we use set_backtrace, it is because we are catching an exception and then re-raising as either the same exception class or as a different exception class. Our goal here is to add context to exceptions that would otherwise be very difficult for the end user to understand.

We somewhat recently rewrote our error messaging to be more user friendly. To do this, we need to process backtraces and filter out framework code (leaving only user code) and then use regular expressions to find the relevant line numbers. Having structured backtrace data would be much better than what we're doing now.

For our use case, we need a way to set backtracelocations from one exception to another. We currently only use setbacktrace immediately after creating an exception, so it would be fine if we could pass the needed data into the constructor, as long as there is a reasonable way to make the method signatures in current ruby compatible via monkey patching or #respond_to?

#9 Updated by Charles Nutter about 1 year ago

kallistec (Daniel DeLeo) wrote:

Hi, I work on Opscode's Chef, a server configuration framework. I have an interest in both parts of this topic. In Chef, we use set_backtrace but we also parse backtrace lines to find relevant code in our error messages.

When we use set_backtrace, it is because we are catching an exception and then re-raising as either the same exception class or as a different exception class. Our goal here is to add context to exceptions that would otherwise be very difficult for the end user to understand.

So, as long as you'd be happy with Thread::Backtrace::Location's elements, it should just be a matter of creating a new Array of Location objects and passing it to an hypothetical setbacktracelocations (or hey, let's get crazy and call it backtrace_locations= perhaps?)

Some specifications I proposed before:

  • If an exception has had no modifications applied, #backtrace and #backtrace_locations will be the same content (in different form, obviously).
  • If an exception has had #setbacktracelocations (#backtracelocations=) called, #backtrace and #backtracelocations will reflect the new content.
  • If an exception has had #setbacktrace called, #backtrace will reflect the new content but #backtracelocations will not (due to the impossibility of guaranteeing #set_backtrace lines will parse into Location objects).

So basically #backtrace_locations is the future path for dealing with exception data and #backtrace (probably deprecated in future) only piggy-backs off it.

Seem reasonable?

#10 Updated by Anonymous about 1 year ago

On Sat, Mar 16, 2013 at 02:12:03AM +0900, headius (Charles Nutter) wrote:

Issue #7895 has been updated by headius (Charles Nutter).

kallistec (Daniel DeLeo) wrote:

Hi, I work on Opscode's Chef, a server configuration framework. I have an interest in both parts of this topic. In Chef, we use set_backtrace but we also parse backtrace lines to find relevant code in our error messages.

When we use set_backtrace, it is because we are catching an exception and then re-raising as either the same exception class or as a different exception class. Our goal here is to add context to exceptions that would otherwise be very difficult for the end user to understand.

Rails uses set_backtrace, and is basically in the same boat.

So, as long as you'd be happy with Thread::Backtrace::Location's elements, it should just be a matter of creating a new Array of Location objects and passing it to an hypothetical setbacktracelocations (or hey, let's get crazy and call it backtrace_locations= perhaps?)

Some specifications I proposed before:

  • If an exception has had no modifications applied, #backtrace and #backtrace_locations will be the same content (in different form, obviously).
  • If an exception has had #setbacktracelocations (#backtracelocations=) called, #backtrace and #backtracelocations will reflect the new content.
  • If an exception has had #setbacktrace called, #backtrace will reflect the new content but #backtracelocations will not (due to the impossibility of guaranteeing #set_backtrace lines will parse into Location objects).

So basically #backtrace_locations is the future path for dealing with exception data and #backtrace (probably deprecated in future) only piggy-backs off it.

Seem reasonable?

This is something we could work with. :-)

--
Aaron Patterson
http://tenderlovemaking.com/

#11 Updated by Charles Nutter about 1 year ago

What's the next step here? Do we need a patch to proceed? It seems like this feature is universally liked.

#12 Updated by Thomas Enebo about 1 year ago

Has anyone talked about .backtrace returning a subclass of String (or singleton) which duck-types to the same methods as Location? It is arguably hacky and would not work for === but it would still allow them to be used in a compatible way. Or maybe .backtracelocations provides this special string. If setbacktrace is used it will just be dressed up with this methods it .backtrace (or .backtrace_locations) is called.

#13 Updated by Charles Nutter about 1 year ago

I think that was tossed around on IRC or somewhere. It would certainly work, but it feels a little too hacky to me.

You'd also never be able to rely on the String objects from #backtrace actually being Location-like, since set_backtrace would clear them. I still feel like this is the best plan:

  • backtrace_locations is the master copy of backtrace data
  • setbacktracelocations can be used to modify it (optional...I have no strong opinion)
  • backtrace generates based on backtracelocations unless explicitly set with setbacktrace
  • setting backtrace to nil reverts it to using backtrace_locations (maybe?)

#14 Updated by Charles Nutter 12 months ago

I should mention that Exception#backtrace_locations is implemented in JRuby on master (1.7.4) in ruby 2.0 mode. I had not intended to add it before MRI, but it's there. We may remove it before release, but this would give folks a change to play with it on a standard build (jruby master).

#15 Updated by Koichi Sasada 12 months ago

(2013/04/17 2:54), headius (Charles Nutter) wrote:

What's the next step here? Do we need a patch to proceed? It seems like this feature is universally liked.

I don't have any objection about it now.

Patches are welcome! :)

Make another issue for CRuby/trunk?

--

// SASADA Koichi at atdot dot net

#16 Updated by Charles Nutter 7 months ago

I have filed https://bugs.ruby-lang.org/issues/8960 to get this feature added to MRI. It is already available in JRuby.

#17 Updated by Charles Nutter 7 months ago

  • Target version set to Ruby 2.1.0

Also available in: Atom PDF