Feature #21501: Include native filenames in backtraces as sources for native methods

• Status: Open
• Assignee: -
• Target version: -

[ivoanjo (Ivo Anjo)]

Consider this example:

require 'bigdecimal'

BigDecimal.singleton_class.prepend(
  Module.new do
    def save_rounding_mode
      super
    end
  end
)

[:example].each do
  BigDecimal.save_rounding_mode do
    puts caller
    sleep 1
  end
end

which Ruby will print as (from 3.4, since https://bugs.ruby-lang.org/issues/19117):

native-filenames-example.rb:6:in 'BigDecimal.save_rounding_mode'
native-filenames-example.rb:6:in 'save_rounding_mode'
native-filenames-example.rb:12:in 'block in <main>'
native-filenames-example.rb:11:in 'Array#each'
native-filenames-example.rb:11:in '<main>'

Having the class names helps, but I think this behavior can still be confusing. Without looking at the code, and understanding that Array#each and BigDecimal.save_rounding_mode are written in native code, it can be confusing to see them be assigned to native-filenames-example.rb in the backtrace.

I would like to propose that, whenever possible (see notes below), Ruby shows the name of the native filename where a method was defined. This would result on the backtrace becoming:

/rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode'
native-filenames-example.rb:6:in 'save_rounding_mode'
native-filenames-example.rb:12:in 'block in <main>'
/rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4:0:in 'Array#each'
native-filenames-example.rb:11:in '<main>'

or, alternatively, this mechanism could be applied only to extensions, not to libruby/the ruby binary (it's not particularly hard to detect and exclude it):

/rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode'
native-filenames-example.rb:6:in 'save_rounding_mode'
native-filenames-example.rb:12:in 'block in <main>'
native-filenames-example.rb:11:in 'Array#each' # Don't touch array ;)
native-filenames-example.rb:11:in '<main>'

This would make it even easier to understand what's coming from where, and I believe it's a great complement to showing the class names.

Furthermore, displaying the actual native libraries matches really well with the semantics we get for regular Ruby files: if I were to look on my machine, I would see /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so and /rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4. And if I moved them, or overrode them, and loaded them from a different path, I would see the difference. (I say this because in the past I've thought about displaying the actual source .c files and lines, but that a) requires debug information; and b) the files often no longer exist in the filesystem; c) is much harder to implement cross-os).

---

The extra cool thing is that this is really, really easy to implement!

I actually created a micro gem to show this off as well as added this as a feature to the Datadog Ruby profiler. Other than checking for headers and whatnot, fetching the info can be implemented in 41 lines of C: https://github.com/ivoanjo/native-filenames/blob/main/ext/native_filenames_extension/native_filenames_extension.c#L45-L86 and works in Linux, macOS and Windows!

The way it works is, we can use the function pointer passed to rb_define_method, looking it put using dladdr (or some of the other variants shown in the code). Then, given just the function pointer, we can get a path back. It's as simple as that -- it can be added transparently as part of storing the cfunc.

And if for some reason the path is not available, we could fall back to the current behavior.

I'd be very happy to contribute a PR to make this change, but I decided to start with a proposal first as I realize not everyone may be as excited about this idea as I am ;)