Feature #6012: Proc#source_location also return the column

• Status: Closed
• Assignee: nobu (Nobuyoshi Nakada)
• Target version: -

[rogerdpack (Roger Pack)]

As originally suggested in http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/42418

Suggestion/feature request:
have #source_location also return the beginning column where it was defined.
["test.rb", 8, 33]

Thanks!
-roger-

[rogerdpack (Roger Pack)]

oops make that a feature request, but I'm unable to edit them myself.
Cheers!
-r

[nahi (Hiroshi Nakamura)]

• Tracker changed from Bug to Feature

[ko1 (Koichi Sasada)]

• Category set to core
• Assignee set to nobu (Nobuyoshi Nakada)
• Target version set to 2.0.0

[trans (Thomas Sawyer)]

Would this effect Method#source_location too?

I'm not sure I am really digging this idea. First of all it means I have to go back and fix some code. Secondly it means I have to always worry about the additional piece of data even though most of the time it doesn't matter. And if the return can vary between 2 or 3 elements that's another thing to worry with.

On the other hand I can understand that it could be useful information in some cases.

In times like this that I think "Embrace the Object".

proc.source_location #=> #<SourceLocation @file="foo.rb" @line=12 @column=14>

And then a few different methods could provide that information in various useful forms.

proc.source_location.to_a #=> ["foo.rb", 12, 14]
proc.source_location.to_s #=> "foo.rb:12"
proc.source_location.values_at(:file, :line) #=> ["foo.rb", 12]

Or what have you.

[trans (Thomas Sawyer)]

BTW & OT: When is any one going to explain how we format code examples as monospace text on this site?

[drbrain (Eric Hodel)]

On Feb 26, 2012, at 6:33 AM, Thomas Sawyer wrote:

BTW & OT: When is any one going to explain how we format code examples as monospace text on this site?

Click the RD button and use RD formatting (two spaces).

Here's a bash alias to help, which works for rdoc too.

alias rdindent='pr -l1 -o2'

[trans (Thomas Sawyer)]

Thanks Eric! I ((never)) noticed that ((%RD%)) "button" before (hardly looks like a button).

Why did it put:

=begin
=end

In the textarea when I clicked on it? ... maybe I'll find out by submitting this...

=begin
What's with the =begin =end?

Testing 1 2 3...

Try ((em)) (({code})) ((|ls|)) ((%var%)).
=end

Sorry for the noise.

[trans (Thomas Sawyer)]

Well, that failed miserably. LOL :-)

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Assigned

[yhara (Yutaka HARA)]

• Target version changed from 2.0.0 to 2.6

[naruse (Yui NARUSE)]

• Target version deleted (2.6)

[mame (Yusuke Endoh)]

Now the abstract syntax tree has column information, so we can implement this issue. We even add the last point of method.

# test.rb
◆def foo # ◆: line 2, column 0
end★     # ★: line 3, column 3

p method(:foo).source_location #=> ["test.rb", 2, 0, 3, 3]

[ioquatix (Samuel Williams)]

If changing this API is too complicated due to backwards compatibility, why not introduce new more general API:

Method#source -> Source.new(path, line_number, line_count, code, ...)

Usage:

method.source.code
method.source.path
method.source.location -> [2, 0, 3, 3]

Maybe including byte offset and length would also be useful (for seek).

[ioquatix (Samuel Williams)]

I also wish there was some meaningful implementation of proc.source.hash that was reasonably consistent across invocations of Ruby. Even if it was just best effort.

[ioquatix (Samuel Williams)]

I was playing around with this idea trying to make an implementation of class Source.

Is the source file cached in Ruby? Or should we use File.read to load it into memory?

It seems inefficient for large files, to find line/column offset. It would be nice to have absolute offset to seek to.

Maybe it's possible for source_location to append one more thing - the actual source code - if possible. This would be useful for situations like eval, where you might define something for a path that doesn't actually exist, but the source code is still available.

[duerst (Martin Dürst)]

ioquatix (Samuel Williams) wrote:

I also wish there was some meaningful implementation of proc.source.hash that was reasonably consistent across invocations of Ruby. Even if it was just best effort.

Please make that a separate feature if you are serious about it.

[Eregon (Benoit Daloze)]

• Related to Feature #17930: Add column information into error backtrace added

[Eregon (Benoit Daloze)]

It seems good to revisit this, the workarounds are pretty messy and CRuby-specific, e.g. https://github.com/rails/rails/pull/53055/files

I think @mame (Yusuke Endoh) 's suggestion in https://bugs.ruby-lang.org/issues/6012#note-12 is fine although I think it would be convenient to also expose the byte offsets (start and end):
.source_location #=> [path, start_line, start_column, start_offset, end_line, end_column, end_offset]

Then it would be really easy to "get the source code of a Proc".
And of course we should do the same for Method/UnboundMethod (#8751).

[Eregon (Benoit Daloze)]

I also really like @ioquatix (Samuel Williams) 's suggestion in https://bugs.ruby-lang.org/issues/6012#note-13 and it is a lot more flexible and more efficient too (since computing e.g. column information if unused is not cheap).
For instance method.source.code is great because it completely hides the details how to get the source code and slice it.
TruffleRuby currently keeps the source code in memory and so could provide this automatically without needing to reread the file from disk.
CRuby keeps it but only if RubyVM.keep_script_lines = true, so then could use that if available and automatically fallback to read the file from disk (great, because we should avoid users/gems referring to RubyVM in their code).

One question is where would we place/how would we name this class?
We could reuse Thread::Backtrace::Location as it's quite similar and already has path, lineno.
But it's not really related to a backtrace here.
Still it seems quite a good fit, and I don't have much idea where to place it otherwise (top-level Source seems way too prone for conflicts).

I think in term of the interface we should have:

• start_line
• start_column
• start_offset
• end_line
• end_column
• end_offset
• code: gets the source of the Proc/Method/UnboundMethod
• EDIT: source to get the source code of that file (without needing CRuby-specific RubyVM::InstructionSequence#script_lines). Can be nil in some cases.

Related and similar ideas in #18231.

[Eregon (Benoit Daloze)]

• Related to Feature #18231: `RubyVM.keep_script_lines` added

[Eregon (Benoit Daloze)]

One idea for this new SourceLocation/CodeLocation class would be under the new Ruby module, see https://bugs.ruby-lang.org/issues/20884#note-3

[bkuhlmann (Brooke Kuhlmann)]

💡 Please see Feature 21005 as an evolution of this discussion based on feedback I've had with Kevin Newton and Benoit Daloze. Thanks.

[Eregon (Benoit Daloze)]

• Related to Feature #21005: Update the source location method to include line start/stop and column start/stop details added

[nobu (Nobuyoshi Nakada)]

I'm not sure about attributes and Binding#source_location.

https://github.com/ruby/ruby/pull/12539

[mame (Yusuke Endoh)]

This was discussed at the October 2024 dev meeting and @matz (Yukihiro Matsumoto) approved source_location to return [path, start_line, start_column, end_line, end_column].

https://github.com/ruby/dev-meeting-log/blob/28e0f45f334b31e60c7e07d25b6b6cb3655b2c75/2024/DevMeeting-2024-10-03.md?plain=1#L176

Matz was negative neither on adding a new class like Ruby::CodeLocation for this purpose, or adding offset information.

We briefly re-asked matz about this at today's dev meeting, and he had not changed his mind.

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed

Applied in changeset git|073c4e1cc712064e626914fa4a5a8061f903a637.

---

[Feature #6012] Extend source_location for end position and columns

[Eregon (Benoit Daloze)]

Great to see this was accepted and even already implemented! (thanks @nobu (Nobuyoshi Nakada)!)

Byte offsets would be mostly for performance ("character" columns in multibyte files can be expensive to compute) and convenience (e.g. read the code at the given source_location from the file).
They are not strictly necessary.

[Dan0042 (Daniel DeLorme)]

mame (Yusuke Endoh) wrote in #note-25:

Matz was negative neither on adding a new class like Ruby::CodeLocation for this purpose

Any idea why? Because it seems like the "obviously correct" design so I'm curious why it was decided against.

[Eregon (Benoit Daloze)]

• Related to Bug #21784: Proc#source_location start column seems strange for -> {} added

[Eregon (Benoit Daloze)]

• Related to Bug #21783: {Method,UnboundMethod,Proc}#source_location returns columns in bytes and not in characters added

[matz (Yukihiro Matsumoto)]

Although last minute, we will cancel this feature in 4.0 because of design ambiguities such as whether to return column positions in bytes or characters as in #21783.

Matz.

[mame (Yusuke Endoh)]

• Status changed from Closed to Open

I record the summary of the discussion regarding Matz's decision to revert this for now. The decision was based on a combination of several reasons:

• Unclear use cases
  • The description of #6012 lacked a use case.
  • The use case presented in #21005 was not convincing.
• Difficulty in changing Binding#source_location (#6012#note-24, #21005#note-19)
  • Binding#source_location still returns [file, lineno].
  • Implementation issues: It was challenging to preserve the location information of Kernel#binding calls within the ISeq.
  • Compatibility issues: eval(code, nil, *binding.source_location) is used in several gems; adding column information would break them.
• Related issues:
  • We may need to adjust Proc#source_location's position (#21784).
  • We need to discuss column units (bytes vs characters) (#21783).
• Timing: We can still revert without compatibility issues at this point.

[Eregon (Benoit Daloze)]

mame (Yusuke Endoh) wrote in #note-32:

I record the summary of the discussion regarding Matz's decision to revert this for now. The decision was based on a combination of several reasons:

Thank you for documenting that.
It feels to me like the decision was taken while we were in the middle of discussing https://bugs.ruby-lang.org/issues/21783,
where we seem to now have reached an agreement, but the revert decision (and revert PR merged) was taken without any time for me to even reply there.

• Unclear use cases

The fact many issues have been filed about this seems to show this is clearly a wanted feature.
It has always been useful to find where a method/block is defined.
https://rubygems.org/gems/method_source has 700 millions downloads, what other proof do we need?

• Difficulty in changing Binding#source_location (#6012#note-24, #21005#note-19)

I wasn't aware of these, but I think this is much less needed than on {Proc,Method,UnboundMethod}.
Unless I'm missing something, a Binding doesn't always have a properly-defined source location/definition, e.g. for b = binding.
It has a start line if we consider that to be at the line of the binding call, but not really an end line/column.
IOW it's not a "scope" unlike a method or block, and seems much less useful.

• Related issues:
  • We may need to adjust Proc#source_location's position (#21784).
  • We need to discuss column units (bytes vs characters) (#21783).

I'm fixing both of these in https://github.com/ruby/ruby/pull/15580.
Both issues seem to have rather clear agreements at this point.

• Timing: We can still revert without compatibility issues at this point.

The feature has been implemented since 9th January, almost a year ago.
It's not been in a proper release yet of course but it has been in multiple previews.
I'm not sure if other people have tried it already, but I'm pretty confident a bunch of people are gonna be disappointed if this is removed last minute and it needs another year before it's in a Ruby release.
Do we really want to encourage heuristics like https://bugs.ruby-lang.org/issues/20999#note-1 vs a proper a solution by providing the necessary information in {Method,UnboundMethod,Proc}#source_location?

[vo.x (Vit Ondruch)]

Just FTR, not sure how the revert is supposed to look like, but at least Pry was adjusted for the new behavior:

https://github.com/pry/pry/pull/2357

It does not benefit from columns, but I suspect the revert would somehow influenced also the end of method location.

[vo.x (Vit Ondruch)]

And RSpec were also adjusted to the new behavior AFAICT:

https://github.com/rspec/rspec/pull/282/commits/1c20fa80772ca7a1ed0512056ce7cd6a94f8e68d

[Eregon (Benoit Daloze)]

Right, I thought about that too but forgot to write it.
Several gems have already adapted to this feature and their functionality and/or tests might rely on having the extra information, so it's definitely not great to remove this last minute.
If https://github.com/ruby/prism/pull/3808 had been merged already we'd make the entire method unusable in Prism, which certainly sounds bad.

I think we should do https://github.com/ruby/ruby/pull/15580

[matz (Yukihiro Matsumoto)]

I didn't mean to cancel nor reject this proposal. It would probably be merged in 4.1 (or later). But since any design mistake would remain virtually forever in Ruby, and I am the one to be blamed.
We had some concerns, e.g. #21783 or #21784 and it was only days before the release.

Matz.

[Eregon (Benoit Daloze)]

• Status changed from Open to Closed

Applied in changeset git|d82fc3360d7cfa7e1e1a4dddb668b4c38808538a.

---

Reapply "[Feature #6012] Extend source_location for end position

• This reverts commit 065c48cdf11a1c4cece84db44ed8624d294f8fd5.
• This functionality is very valuable and has already taken 14 years
  to agree on the API.
• Let's just document it's byte columns (in the next commit).
• See https://bugs.ruby-lang.org/issues/21783#note-9

[Eregon (Benoit Daloze)]

• Target version set to 4.1

[mame (Yusuke Endoh)]

• Status changed from Closed to Open
• Target version deleted (4.1)

@Eregon (Benoit Daloze) Why was this re-merged?

The first two issues raised in #note-32 (re-listed below) have not yet been resolved.

mame (Yusuke Endoh) wrote in #note-32:

• Unclear use cases
  • The description of #6012 lacked a use case.
  • The use case presented in #21005 was not convincing.
• Difficulty in changing Binding#source_location (#6012#note-24, #21005#note-19)
  • Binding#source_location still returns [file, lineno].
  • Implementation issues: It was challenging to preserve the location information of Kernel#binding calls within the ISeq.
  • Compatibility issues: eval(code, nil, *binding.source_location) is used in several gems; adding column information would break them.

In addition, as pointed out by @vo.x (Vit Ondruch):

https://github.com/pry/pry/pull/2357
https://github.com/rspec/rspec/pull/282

These show that changing the return value of source_location from 2 elements to 5 elements has caused actual incompatibility issues.
While the maintainers kindly addressed this ahead of time for the Ruby 4.0 preview, this suggests that we must be more cautious regarding this change.

Given the above, I believe this should be reverted for now.

[Eregon (Benoit Daloze)]

• Status changed from Open to Closed

@mame (Yusuke Endoh) Matz explicitly approved it: https://github.com/ruby/ruby/pull/15580#issuecomment-3691058342
And he also made it clear it was just too late in the release cycle to address #21783 and #21784, see https://bugs.ruby-lang.org/issues/6012#note-37.

I also don't think it's worth discussing all over again why this is useful.

EDIT:
We have plenty of evidence it is useful:

https://bugs.ruby-lang.org/issues/6012#note-33
It has always been useful to find where a method/block is defined.
https://rubygems.org/gems/method_source has 700 millions downloads, what other proof do we need?

Re Proc#source_location, there is e.g. https://github.com/joker1007/proc_to_ast/issues/5 (https://rubygems.org/gems/proc_to_ast, 71 million downloads), https://bugs.ruby-lang.org/issues/21783#note-21, etc.

[Eregon (Benoit Daloze)]

Re Binding#source_location I don't think it's very useful to have given Kernel#binding seems to just return the location at which it was called, but not of the surrounding method/block:

$ ruby -e 'def foo
  p binding.source_location
end
foo'
["-e", 2]

So given the lower value and the implementation issues let's just not have Binding#source_location, same as since a year ago (https://bugs.ruby-lang.org/issues/6012#note-26).

[mame (Yusuke Endoh)]

Eregon (Benoit Daloze) wrote in #note-41:

@mame (Yusuke Endoh) Matz explicitly approved it: https://github.com/ruby/ruby/pull/15580#issuecomment-3691058342

Ah, I missed that the PR had already been approved. Sorry about that.

However, we should consider the fact that code like source_location.last actually exists in the wild, as RSpec fixed in advance. It seems this idiom is used in quite a few gems.

1980-01-01 /srv/gems/minitest-junit-2.1.0/lib/minitest/junit.rb:        testcase['line'] = result.source_location.last
1980-01-02 /srv/gems/ci-queue-0.81.0/lib/minitest/queue/junit_reporter.rb:          lineno = tests.first.source_location.last
1980-01-02 /srv/gems/ci-queue-0.81.0/lib/minitest/queue/test_data.rb:        @test.source_location.last
1980-01-02 /srv/gems/command_deck-0.3.3/TAGS:      def lineno() @binding.source_location.last endlineno11,0
1980-01-02 /srv/gems/ed-precompiled_bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
1980-01-02 /srv/gems/ed-precompiled_bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
1980-01-02 /srv/gems/ed-precompiled_bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 4, exc.bindings.first.source_location.last
1980-01-02 /srv/gems/ed-precompiled_bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
1980-01-02 /srv/gems/ed-precompiled_bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
1980-01-02 /srv/gems/grape_rails_logger-1.2.0/lib/grape_rails_logger/subscriber.rb:      line = endpoint.source.source_location.last
1980-01-02 /srv/gems/itsi-0.2.20/gems/server/lib/itsi/server/typed_handlers/source_parser.rb:          proc_line = source_location.last - 1
1980-01-02 /srv/gems/itsi-server-0.2.20/lib/itsi/server/typed_handlers/source_parser.rb:          proc_line = source_location.last - 1
1980-01-02 /srv/gems/mutant-0.14.1/lib/mutant/matcher/method.rb:          source_location.last
1980-01-02 /srv/gems/openhab-scripting-5.44.0/lib/openhab/dsl/rules/name_inference.rb:            "#{file}:#{block.source_location.last}".tr(".", "_")
1980-01-02 /srv/gems/pry-0.16.0/lib/pry/wrapped_module/candidate.rb:          last_method_source_location.last
1980-01-02 /srv/gems/rspec-expectations-3.13.5/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
1980-01-02 /srv/gems/scnr-introspector-0.3.2/lib/scnr/introspector/data_flow/sink.rb:            lineno   = @method_source_location.last
2011-09-03 /srv/gems/minecraft-0.3.3/lib/minecraft/extensions.rb:        src_b, src_e = get_comment_range(meth.source_location.last)
2012-10-25 /srv/gems/rubymirrors-0.0.3/lib/ruby/reflection/method_mirror.rb:        source_location.last - 1
2012-11-15 /srv/gems/zeus-edge-0.12.1/lib/zeus/m/test_method.rb:          start_line = method.source_location.last
2014-04-06 /srv/gems/zeus-justinf-0.13.5/lib/zeus/m/test_method.rb:          start_line = method.source_location.last
2015-02-02 /srv/gems/billygoat-0.0.7.2/lib/billygoat/documentation.rb:      @line_number ||= self.method.source_location.last
2015-04-01 /srv/gems/turnip-dry_run-0.1.0/lib/turnip/dry_run.rb:        step_method.source_location.last
2015-04-18 /srv/gems/fare-0.1.8/lib/fare/configuration_dsl.rb:          raise "No run list specified for stack on line #{@source_location.last} of #{@source_location.first}"
2015-04-18 /srv/gems/fare-0.1.8/lib/fare/configuration_dsl.rb:          raise "Stack without topics for stack on line #{@source_location.last} of #{@source_location.first}"
2015-11-12 /srv/gems/git_compound-0.2.2/lib/git_compound/logger/debug/task.rb:        "in line `#{@block.source_location.last}`, " \
2015-12-08 /srv/gems/minirails-0.2.0/lib/minirails.rb:            c.instance_method(m).source_location.last
2016-05-06 /srv/gems/matest-1.7.4/lib/matest/example_block.rb:    lineno = block.source_location.last
2016-05-22 /srv/gems/transform_tree-0.3.1/lib/transform_tree/node.rb:    "#{File.basename closure.source_location.first}:#{closure.source_location.last}"
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      line = @method.source_location.last
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      MethodSpecs::SourceLocation.method(:redefined).source_location.last.should == 13
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      MethodSpecs::SourceLocation.new.method(:aka).source_location.last.should == 17
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @lambda.source_location.last
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @proc_new.source_location.last
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @proc.source_location.last
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_lambda.source_location.last.should == 42
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_proc_new.source_location.last.should == 47
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_proc.source_location.last.should == 37
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_lambda.source_location.last.should == 23
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_proc_new.source_location.last.should == 30
2016-05-23 /srv/gems/tauplatform-1.0.3/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_proc.source_location.last.should == 16
2016-07-13 /srv/gems/rblineprof-0.3.7/test/test_lineprof.rb:    line = profile[__FILE__][m.source_location.last]
2018-05-07 /srv/gems/attestify-0.1.1/lib/attestify/test_list.rb:          return result if source_location.last > line
2019-02-11 /srv/gems/mutest-0.0.10/lib/mutest/matcher/method.rb:          source_location.last
2019-03-11 /srv/gems/pomodorokun-0.1.0/bundle/ruby/2.4.0/gems/rspec-expectations-3.8.2/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2019-07-10 /srv/gems/bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 4, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/bindex-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/skiptrace-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/skiptrace-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 14, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/skiptrace-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 4, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/skiptrace-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
2019-07-10 /srv/gems/skiptrace-0.8.1/test/skiptrace/exception_test.rb:      assert_equal 6, exc.bindings.first.source_location.last
2020-01-03 /srv/gems/puppet-function-updater-0.0.5/lib/pfu/parser.rb:        return [funcname, opts, block.source_location.last]
2020-11-28 /srv/gems/spud-0.2.10/lib/spud/block_param_info.rb:      line = File.read(@filename).split("\n")[@block.source_location.last - 1]
2021-01-13 /srv/gems/seeing_is_believing-4.0.1/spec/event_stream_spec.rb:        return raises_exception.source_location.last
2021-03-04 /srv/gems/dialekt-0.1.0/lib/dialekt/util/call_adapter.rb:        "#{File.basename(@callable.source_location.first)}:#{@callable.source_location.last}"
2021-09-17 /srv/gems/loupe-0.1.5/lib/loupe/executor.rb:            line_numbers.include?(klass.instance_method(method_name).source_location.last.to_s)
2021-11-23 /srv/gems/glimmer-dsl-specification-0.0.5/lib/glimmer/specification/element/fact.rb:          source_code = PutsDebuggerer::SourceFile.new(@block.source_location.first).source(1, @block.source_location.last)
2022-03-18 /srv/gems/andyw8-seeing_is_believing-4.2.1/spec/event_stream_spec.rb:        return raises_exception.source_location.last
2022-05-02 /srv/gems/boothby-0.1.1/lib/boothby/seeds.rb:      methods.sort_by { |method| seed_class.method(method).source_location.last }
2022-05-02 /srv/gems/boothby-0.1.1/lib/boothby/seeds.rb:    methods.sort_by { |method| seed_class.method(method).source_location.last }
2022-11-24 /srv/gems/opal-rspec-1.0.0/rspec-expectations/upstream/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2022-12-28 /srv/gems/pork-2.1.0/lib/pork/extra/show_source.rb:      lowers = test.source_location.last
2023-01-06 /srv/gems/takuya-lvm-snapshot-0.1.0/bundle/ruby/2.7.0/gems/rspec-expectations-3.12.1/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2023-03-04 /srv/gems/blinka-reporter-0.8.0/lib/blinka_reporter/minitest_adapter.rb:      @line ||= source_location.last
2023-09-05 /srv/gems/web-console-4.2.1/lib/web_console/source_location.rb:      def lineno() @binding.source_location.last end
2023-12-18 /srv/gems/ali-cli-0.0.2/lib/ali/template.rb:    Rouge.highlight "\n#{"".ljust(indent, " ")}# #{task_path}:#{source_location.last}\n#{source}", lexer, formatter
2023-12-18 /srv/gems/builder_apm-0.5.14/lib/builder_apm/controllers/instrumenter.rb:          line_number = method_info.source_location.last
2023-12-18 /srv/gems/constrain-0.10.0/lib/constrain.rb:      when Proc; "Proc@#{expr.source_location.first}:#{expr.source_location.last}"
2024-02-11 /srv/gems/prawn-manual_builder-0.4.0/lib/prawn/manual_builder/chapter.rb:          block_source_line = example.source_location.last
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      line = @method.source_location.last
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      MethodSpecs::SourceLocation.method(:redefined).source_location.last.should == 13
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/method/source_location_spec.rb:      MethodSpecs::SourceLocation.new.method(:aka).source_location.last.should == 17
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @lambda.source_location.last
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @proc_new.source_location.last
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      line = @proc.source_location.last
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_lambda.source_location.last.should == 42
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_proc_new.source_location.last.should == 47
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_detached_proc.source_location.last.should == 37
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_lambda.source_location.last.should == 23
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_proc_new.source_location.last.should == 30
2024-02-15 /srv/gems/rhodes-7.6.0/spec/framework_spec/app/spec/core/proc/source_location_spec.rb:      ProcSpecs::SourceLocation.my_multiline_proc.source_location.last.should == 16
2024-07-10 /srv/gems/curlybars-1.12.0/lib/curlybars/rendering_support.rb:        line_number = source_location ? helper.source_location.last : "n/a"
2024-09-11 /srv/gems/zeus-0.17.0/lib/zeus/m/test_method.rb:          start_line = method.source_location.last
2024-09-16 /srv/gems/minitest-distributed-0.2.11/lib/minitest/distributed/reporters/junitxml_reporter.rb:            lineno = test.source_location.last
2025-01-07 /srv/gems/lazy_graph-0.2.0/lib/lazy_graph/node/derived_rules.rb:            derived.source_location.last + offset
2025-01-07 /srv/gems/lazy_graph-0.2.0/lib/lazy_graph/node/derived_rules.rb:            proc_line = source_location.last - 1
2025-08-08 /srv/gems/openstudio-common-measures-0.12.3/lib/measures/envelope_and_internal_load_breakdown/measure.rb:      method_hash[section.to_s] = OsLib_ReportingHeatGainLoss.method(section).source_location.last
2025-08-08 /srv/gems/openstudio-common-measures-0.12.3/lib/measures/example_report/measure.rb:      method_hash[section.to_s] = OsLib_Reporting_example.method(section).source_location.last

@matz (Yukihiro Matsumoto), are you sure this is okay?

[Eregon (Benoit Daloze)]

mame (Yusuke Endoh) wrote in #note-43:

However, we should consider the fact that code like source_location.last actually exists in the wild, as RSpec fixed in advance. It seems this idiom is used in quite a few gems.

This change has been in all 3.5 and 4.0 preview builds and seems to have caused very little incompatibility.
It will have another full year before the 4.1 release, so I think this is no problem.

BTW, if we remove tests & specs from that list (since they wouldn't break any code) we have:

$ cat matches.txt | grep -Fv '_spec.rb' | grep -Fv '_test.rb' | grep -Fv '/test_'
1980-01-01 /srv/gems/minitest-junit-2.1.0/lib/minitest/junit.rb:        testcase['line'] = result.source_location.last
1980-01-02 /srv/gems/ci-queue-0.81.0/lib/minitest/queue/junit_reporter.rb:          lineno = tests.first.source_location.last
1980-01-02 /srv/gems/command_deck-0.3.3/TAGS:      def lineno() @binding.source_location.last endlineno11,0
1980-01-02 /srv/gems/grape_rails_logger-1.2.0/lib/grape_rails_logger/subscriber.rb:      line = endpoint.source.source_location.last
1980-01-02 /srv/gems/itsi-0.2.20/gems/server/lib/itsi/server/typed_handlers/source_parser.rb:          proc_line = source_location.last - 1
1980-01-02 /srv/gems/itsi-server-0.2.20/lib/itsi/server/typed_handlers/source_parser.rb:          proc_line = source_location.last - 1
1980-01-02 /srv/gems/mutant-0.14.1/lib/mutant/matcher/method.rb:          source_location.last
1980-01-02 /srv/gems/openhab-scripting-5.44.0/lib/openhab/dsl/rules/name_inference.rb:            "#{file}:#{block.source_location.last}".tr(".", "_")
1980-01-02 /srv/gems/pry-0.16.0/lib/pry/wrapped_module/candidate.rb:          last_method_source_location.last
1980-01-02 /srv/gems/rspec-expectations-3.13.5/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
1980-01-02 /srv/gems/scnr-introspector-0.3.2/lib/scnr/introspector/data_flow/sink.rb:            lineno   = @method_source_location.last
2011-09-03 /srv/gems/minecraft-0.3.3/lib/minecraft/extensions.rb:        src_b, src_e = get_comment_range(meth.source_location.last)
2012-10-25 /srv/gems/rubymirrors-0.0.3/lib/ruby/reflection/method_mirror.rb:        source_location.last - 1
2015-02-02 /srv/gems/billygoat-0.0.7.2/lib/billygoat/documentation.rb:      @line_number ||= self.method.source_location.last
2015-04-01 /srv/gems/turnip-dry_run-0.1.0/lib/turnip/dry_run.rb:        step_method.source_location.last
2015-04-18 /srv/gems/fare-0.1.8/lib/fare/configuration_dsl.rb:          raise "No run list specified for stack on line #{@source_location.last} of #{@source_location.first}"
2015-04-18 /srv/gems/fare-0.1.8/lib/fare/configuration_dsl.rb:          raise "Stack without topics for stack on line #{@source_location.last} of #{@source_location.first}"
2015-11-12 /srv/gems/git_compound-0.2.2/lib/git_compound/logger/debug/task.rb:        "in line `#{@block.source_location.last}`, " \
2015-12-08 /srv/gems/minirails-0.2.0/lib/minirails.rb:            c.instance_method(m).source_location.last
2016-05-06 /srv/gems/matest-1.7.4/lib/matest/example_block.rb:    lineno = block.source_location.last
2016-05-22 /srv/gems/transform_tree-0.3.1/lib/transform_tree/node.rb:    "#{File.basename closure.source_location.first}:#{closure.source_location.last}"
2019-02-11 /srv/gems/mutest-0.0.10/lib/mutest/matcher/method.rb:          source_location.last
2019-03-11 /srv/gems/pomodorokun-0.1.0/bundle/ruby/2.4.0/gems/rspec-expectations-3.8.2/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2020-01-03 /srv/gems/puppet-function-updater-0.0.5/lib/pfu/parser.rb:        return [funcname, opts, block.source_location.last]
2020-11-28 /srv/gems/spud-0.2.10/lib/spud/block_param_info.rb:      line = File.read(@filename).split("\n")[@block.source_location.last - 1]
2021-03-04 /srv/gems/dialekt-0.1.0/lib/dialekt/util/call_adapter.rb:        "#{File.basename(@callable.source_location.first)}:#{@callable.source_location.last}"
2021-09-17 /srv/gems/loupe-0.1.5/lib/loupe/executor.rb:            line_numbers.include?(klass.instance_method(method_name).source_location.last.to_s)
2021-11-23 /srv/gems/glimmer-dsl-specification-0.0.5/lib/glimmer/specification/element/fact.rb:          source_code = PutsDebuggerer::SourceFile.new(@block.source_location.first).source(1, @block.source_location.last)
2022-05-02 /srv/gems/boothby-0.1.1/lib/boothby/seeds.rb:      methods.sort_by { |method| seed_class.method(method).source_location.last }
2022-05-02 /srv/gems/boothby-0.1.1/lib/boothby/seeds.rb:    methods.sort_by { |method| seed_class.method(method).source_location.last }
2022-11-24 /srv/gems/opal-rspec-1.0.0/rspec-expectations/upstream/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2022-12-28 /srv/gems/pork-2.1.0/lib/pork/extra/show_source.rb:      lowers = test.source_location.last
2023-01-06 /srv/gems/takuya-lvm-snapshot-0.1.0/bundle/ruby/2.7.0/gems/rspec-expectations-3.12.1/lib/rspec/expectations/block_snippet_extractor.rb:        source_location.last
2023-03-04 /srv/gems/blinka-reporter-0.8.0/lib/blinka_reporter/minitest_adapter.rb:      @line ||= source_location.last
2023-09-05 /srv/gems/web-console-4.2.1/lib/web_console/source_location.rb:      def lineno() @binding.source_location.last end
2023-12-18 /srv/gems/ali-cli-0.0.2/lib/ali/template.rb:    Rouge.highlight "\n#{"".ljust(indent, " ")}# #{task_path}:#{source_location.last}\n#{source}", lexer, formatter
2023-12-18 /srv/gems/builder_apm-0.5.14/lib/builder_apm/controllers/instrumenter.rb:          line_number = method_info.source_location.last
2023-12-18 /srv/gems/constrain-0.10.0/lib/constrain.rb:      when Proc; "Proc@#{expr.source_location.first}:#{expr.source_location.last}"
2024-02-11 /srv/gems/prawn-manual_builder-0.4.0/lib/prawn/manual_builder/chapter.rb:          block_source_line = example.source_location.last
2024-07-10 /srv/gems/curlybars-1.12.0/lib/curlybars/rendering_support.rb:        line_number = source_location ? helper.source_location.last : "n/a"
2024-09-16 /srv/gems/minitest-distributed-0.2.11/lib/minitest/distributed/reporters/junitxml_reporter.rb:            lineno = test.source_location.last
2025-01-07 /srv/gems/lazy_graph-0.2.0/lib/lazy_graph/node/derived_rules.rb:            derived.source_location.last + offset
2025-01-07 /srv/gems/lazy_graph-0.2.0/lib/lazy_graph/node/derived_rules.rb:            proc_line = source_location.last - 1
2025-08-08 /srv/gems/openstudio-common-measures-0.12.3/lib/measures/envelope_and_internal_load_breakdown/measure.rb:      method_hash[section.to_s] = OsLib_ReportingHeatGainLoss.method(section).source_location.last
2025-08-08 /srv/gems/openstudio-common-measures-0.12.3/lib/measures/example_report/measure.rb:      method_hash[section.to_s] = OsLib_Reporting_example.method(section).source_location.last

In any case it's a tiny fraction of e.g. the change in 3.4 for backtraces from `foo' to 'Module#foo'.

In https://bugs.ruby-lang.org/issues/21777#note-7 you write:

Is this incompatibility really ok? I feel it is disproportionately incompatible for unclear use cases.

Why don't you consider the gems that need this functionality method_source (700 million downloads) and proc_to_ast (71 million downloads) as valid use cases?
Clearly people use them.
Currently method_source which defines Method#source works around by parsing from the start line and adding an extra line one by one until valid:
https://github.com/banister/method_source/blob/06f21c66380c64ff05c8031c0208eef240da0e83/lib/method_source/code_helpers.rb#L95-L98
This is obviously hacky (but there was no better way before this), and wouldn't work with multiple def on the same line.

proc_to_ast currently uses the parser gem to find the block at the given line but fails if there are multiple blocks at that line.
This functionality will fix that: https://github.com/joker1007/proc_to_ast/issues/5

This is already plenty of use cases I think, but additionally we also discussed:

• It would enabling fixing the issue with power_assert with 2 assert on the same line: https://bugs.ruby-lang.org/issues/21783#note-21. It might also help power_assert to support multi-line blocks, which is a pretty big limitation.
• It enables to implement Prism.node_for (https://bugs.ruby-lang.org/issues/21005#note-5) and similar functionality in a portable-across-Ruby-implementations way, so it's both useful directly and indirectly. #21795 would also solve that, but not necessarily the other use cases above, which don't all need the AST (e.g. Method#source, which e.g. can be useful to just show the source of a method for documentation, debugging, etc).

[mame (Yusuke Endoh)]

Eregon (Benoit Daloze) wrote in #note-44:

mame (Yusuke Endoh) wrote in #note-43:
Why don't you consider the gems that need this functionality method_source (700 million downloads) and proc_to_ast (71 million downloads) as valid use cases?

First, I suspect that what many users of method_source gem actually need is the AST. The proposal #21005 itself demonstrates this.

I recognize the use case of retrieving the AST subtree corresponding to some kinds of Ruby objects. However, source_location is not an appropriate key to look up the AST subtree corresponding to a Ruby object. At least for CRuby, node_id should be used, though I don't say you should implement node_id in TruffleRuby.

Therefore, the valid use case for source_location itself is not clear yet, IMO.

[mame (Yusuke Endoh)]

Note that there is also an idiom like foo(*obj.source_location), not just source_location.last. By checking the result of "gem-codesearch", we see not only @binding.source_location but also @proc.source_location. I am not sure if these are actually affected, though.
I guess there are probably other affected code patterns.

$ gem-codesearch -f rb "\\(.*\\*.*\\.source_location[^\\[]"
1980-01-02 /srv/gems/alchemy_cms-8.0.2/vendor/bundle/ruby/3.4.0/gems/web-console-4.2.1/lib/web_console/evaluator.rb:        "=> #{@binding.eval(input, *@binding.source_location).inspect}\n"
1980-01-02 /srv/gems/alchemy_i18n-5.0.1/vendor/bundle/ruby/4.0.0/gems/alchemy_cms-8.0.2/vendor/bundle/ruby/3.4.0/gems/web-console-4.2.1/lib/web_console/evaluator.rb:        "=> #{@binding.eval(input, *@binding.source_location).inspect}\n"
2017-06-09 /srv/gems/aristotle-0.3.0/lib/aristotle/presenter.rb:      code = find_code(*proc.source_location) || 'no code found'
2025-12-10 /srv/gems/cucumber-10.2.0/lib/cucumber/glue/hook.rb:        @location = Cucumber::Core::Test::Location.from_source_location(*@proc.source_location)
2025-12-10 /srv/gems/cucumber-10.2.0/lib/cucumber/glue/registry_and_more.rb:      location = Cucumber::Core::Test::Location.from_source_location(*proc.source_location)
2025-12-10 /srv/gems/cucumber-10.2.0/lib/cucumber/glue/step_definition.rb:        @location ||= Cucumber::Core::Test::Location.from_source_location(*@proc.source_location)
2010-04-06 /srv/gems/dde-0.2.11/spec/spec_helper.rb:    it description_from(caller[0]), &block # it description_from(*block.source_location), &block
2010-01-16 /srv/gems/git_hub-0.3.0/spec/spec_helper.rb:    it description_from(*block.source_location), &block
2017-04-17 /srv/gems/ikra-0.0.2/lib/sourcify/lib/sourcify/method/parser.rb:        @source_code = SourceCode.new(*_meth.source_location)
2017-04-17 /srv/gems/ikra-0.0.2/lib/sourcify/lib/sourcify/proc/parser.rb:        @arity, @source_code = _proc.arity, SourceCode.new(*_proc.source_location(false))
1980-01-02 /srv/gems/irb-kit-2.0.0/lib/irb/kit/handlers/method_editor.rb:          editor.call(*object.method(name).source_location)
2017-12-11 /srv/gems/lab42_core-0.5.1/lib/lab42/core/kernel.rb:    dir = File.expand_path File.join( '..', blk.(), '*.rb'), blk.source_location.first
2011-08-02 /srv/gems/live_ast-1.0.2/lib/live_ast/base.rb:        Linker.find_method_ast(obj.owner, obj.name, *obj.source_location)
2011-08-02 /srv/gems/live_ast-1.0.2/lib/live_ast/linker.rb:            ast = find_ast(*obj.source_location) or raise ASTNotFoundError
2016-10-31 /srv/gems/mobiusloop-0.1.5/lib/cucumber/filters/prepare_world.rb:          empty_hook_location = Cucumber::Core::Ast::Location.from_source_location(*empty_hook.source_location)
2016-10-31 /srv/gems/mobiusloop-0.1.5/lib/cucumber/rb_support/rb_hook.rb:        @location = Cucumber::Core::Ast::Location.from_source_location(*@proc.source_location)
2016-10-31 /srv/gems/mobiusloop-0.1.5/lib/cucumber/rb_support/rb_language.rb:      location = Cucumber::Core::Ast::Location.from_source_location(*proc.source_location)
2016-10-31 /srv/gems/mobiusloop-0.1.5/lib/cucumber/rb_support/rb_step_definition.rb:        @location ||= Cucumber::Core::Ast::Location.from_source_location(*@proc.source_location)
1980-01-02 /srv/gems/mvz-live_ast-2.3.0/lib/live_ast/base.rb:        Linker.find_method_ast(obj.owner, obj.name, *obj.source_location)
1980-01-02 /srv/gems/mvz-live_ast-2.3.0/lib/live_ast/linker.rb:            ast = find_ast(*obj.source_location) or raise ASTNotFoundError
2025-12-02 /srv/gems/openproject-primer_view_components-0.78.1/app/lib/primer/experimental_render_helpers.rb:        return yield(*args, **kwargs) if block&.source_location.nil?
2025-12-03 /srv/gems/primer_view_components-0.48.0/app/lib/primer/experimental_render_helpers.rb:        return yield(*args, **kwargs) if block&.source_location.nil?
2024-12-30 /srv/gems/r_spec-clone-1.7.2/lib/r_spec/clone/dsl.rb:        Logger.source(*block.source_location)
2024-12-30 /srv/gems/r_spec-clone-1.7.2/lib/r_spec/clone/dsl.rb:        Logger.source(*block.source_location)
2010-09-06 /srv/gems/remote_logger-0.0.5/spec/spec_helper.rb:    it description_from(*block.source_location), &block
2017-02-23 /srv/gems/respectable-0.3.3/lib/respectable.rb:        instance_eval(<<-IT, *block.source_location)
2017-02-23 /srv/gems/respectable-0.3.3/lib/respectable.rb:            eval(block.source, binding, *block.source_location)
2025-10-19 /srv/gems/scrapbook-0.4.0/vendor/ruby/2.7.0/gems/irb-1.4.1/lib/irb/workspace.rb:        @binding = eval("IRB.conf[:__MAIN__].instance_eval('binding', __FILE__, __LINE__)", @binding, *@binding.source_location)
1980-01-02 /srv/gems/showcase-rails-0.5.0/app/models/showcase/sample.rb:    source = extract_source_block_via_matched_indentation_from(*block.source_location)
2011-05-02 /srv/gems/sourcify-0.5.0/lib/sourcify/proc/parser.rb:        @arity, @source_code = _proc.arity, SourceCode.new(*_proc.source_location(false))
2018-11-12 /srv/gems/suture-1.1.2/lib/suture/error/verification_failed.rb:        "Proc # (in: `#{describe_source_location(*comparator.source_location)}`)"
2018-11-12 /srv/gems/suture-1.1.2/lib/suture/error/verification_failed.rb:        "#{comparator.inspect}.new, # (in: `#{describe_source_location(*comparator.method(:call).source_location)}`)"
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:      <dd><%= display_source_locaction(*Rack::Attack.blocklisted_response.source_location) %></dd>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:      <dd><%= display_source_locaction(*Rack::Attack.throttled_response.source_location) %></dd>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:                <%= display_source_locaction(*throttle.limit.source_location)  %>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:                <%= display_source_locaction(*throttle.period.source_location)  %>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:              <%= display_source_locaction(*throttle.block.source_location) %>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:          <td><%= display_source_locaction(*safelist.block.source_location) %></td>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:            <td><%= display_source_locaction(*blocklists.block.source_location) %></td>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:              <td><%= display_source_locaction(*track.filter.block.source_location)  rescue 'No Source Location' %></td>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rack_attack/index.html.erb:            <td><%= display_source_locaction(*listener.instance_variable_get(:@delegate).source_location) rescue 'Unknown'   %></td>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rails/index.html.erb:          <td colspan="4"><code><%= display_source_locaction(*k.source_location) %></code></td>
2018-02-12 /srv/gems/tuttle-0.0.8/app/views/tuttle/rails/index.html.erb:              <td colspan="4"><code><%= display_source_locaction(*k.source_location) %></code></td>
2023-09-05 /srv/gems/web-console-4.2.1/lib/web_console/evaluator.rb:        "=> #{@binding.eval(input, *@binding.source_location).inspect}\n"
2011-11-11 /srv/gems/win-0.3.27/spec/spec_helper.rb:    it description_from(caller[0]), &block # it description_from(*block.source_location), &block
2011-11-11 /srv/gems/win_gui-0.2.21/spec/spec_helper.rb:    it description_from(*block.source_location), &block

[Eregon (Benoit Daloze)]

mame (Yusuke Endoh) wrote in #note-45:

First, I suspect that what many users of method_source gem actually need is the AST.

Many yes but not all, maybe even only a minority (see below).
The fact there is Method#source indicates that some usages only want the code, and don't need to parse it.
If it was all, there likely wouldn't be Method#source, but another method to return the (e.g. parser gem) AST directly then.

A quick search shows some examples which do not use an AST:
https://github.com/search?q=language%3ARuby+%2F%28method%7Cmeth%29%5C.source%5Cb%2F&type=code

# spree/spree
    obj_method.source.squish[/(expect *({|do) *)(.*?)( *(}|end).(not_)*to)/, 3]
# evilmartians/terraforming-rails, perfect use case for end_line in source_location
        # Uses `method_source` to get the source code and the number of lines
        meth.source_location.last..(meth.source_location.last + meth.source.lines.size)
# pry/pry-git
        final_code = before_code << meth.source.lines.to_a << after_code
# Darkmux/cyberspy, seems pry related, makes sense to be able to show the source of a method/block in a REPL
        new(meth.source, start_line, meth.source_type)
# rambler-digital-solutions/rusby, the preprocessor acts on the code String, not an AST
        Preprocessor.apply(orig_method.source),
# tdg5/tco_method
      code = <<-CODE
        #{receiver_class} #{receiver.name}
          #{existing_method.source}
        end
      CODE
# garmoshka-mo/pry-moves, perfect use case for end_line in source_location
        end: (source[1] + method.source.count("\n") - 1)
# SyborgStudios/overrides_tracker
    def self.outdented_method_body(method)
      body = method.source
      indent = body.match(/^\W+/).to_s
# rpanachi/core_ext, perfect use case for end_line in source_location
              end_line = method.source.count("\n") + start_line - 1
# annkissam/scryglass
            method_lines = Hexes.capture_io { puts method.source }.split("\n")
# callstacking/callstacking-rails, seems like it adds extra code to instrument, no AST, no dependency on parser
            method_source = method(__method__).super_method.source if settings.analyze_source?
# jeromedalbert/rubocop-obsession
                corrector.replace(method, method.source.sub(method.method_name.to_s, 'perform'))
# geni/sweatshop, perfect use case for end_line in source_location
        end_line = method.source.count("\n") + start_line + 1
# Auralcat/my-dotfiles
# Display source code quickly in IRB.
def srd(method)
   method.source.display
end

and that's only the examples from the first page (of a very restricted search).
In fact a majority of the matches in that first page (14/20) seem to not use an AST, so I'm not even sure the majority of uses cases need the AST.

Finding the end_line of a method seems a recurrent need, this is now provided with the extended source_location.

---

However, source_location is not an appropriate key to look up the AST subtree corresponding to a Ruby object.

I believe it is though, with the knowledge of what kind of node we are looking for.
For example in def foo; bar; end, bar in the AST is covered exactly by both a StatementsNode and a CallNode.
If we are using Thread::Backtrace::Location#ast we'd want the location of the call to bar, so we know we want the CallNode, not the StatementsNode and there is no ambiguity.
I believe the same holds for all 5 methods proposed in #21795.
My intuition there is all nodes listed in https://bugs.ruby-lang.org/issues/21795#note-2 cannot have the exact same source_location (e.g. we cannot have code with the same starting and ending position that is two of DefNode, LambdaNode, ForNode, Call*Node, Index*Node, YieldNode).
Do you have a counter-example where this wouldn't hold?

In fact if #21795 is accepted I do plan to implement it in TruffleRuby based on byte offsets in the source, which are equivalent to line+column start/end.

[mame (Yusuke Endoh)]

A quick search shows some examples which do not use an AST:

A "use case" is the ultimate goal that the examples aim to achieve via Method#source.

By the way, I tried the method_source gem, and I noticed that it includes heredoc content that lies outside the method definition block.

require "method_source"

def foo; <<END; end
FOO
END

p method(:foo).source          #=> "def foo; <<END; end\nFOO\nEND\n"
p method(:foo).source_location #=> ["/tmp/t.rb", 3, 0, 3, 19]

On the other hand, the current source_location (as shown above) only includes the range of def ... end.

I am not arguing that source_location needs to be fixed. My point is that the "better" return value depends entirely on the final use case.

If the goal of Method#source is to return a parsable code fragment (I don't know if that is the official goal), then its current behavior might be expected.
However, since @Eregon (Benoit Daloze) seems to want to use source_location to identify AST subtrees, source_location should not be fine-tuned for one specific use case.

I don't think there is a single, perfect "correct" source_location that everyone can agree on. It will likely vary depending on the ultimate use case.
Instead, I believe users should extract the AST and decide for themselves what location information to obtain.

Do you have a counter-example where this wouldn't hold?

I don't have one at present. However, such a case could arise through future extensions.
For example, @ko1 (Koichi Sasada) might add a new TracePoint type in the future, and depending on the type, it might be necessary to distinguish between StatementsNode and CallNode.

More importantly, CRuby can implement Prism.node_for using node_id without the risk of future ambiguity.
There is no reason to implement such a heuristic hack that tries to select overlapping nodes based on Ruby object type.

[Eregon (Benoit Daloze)]

mame (Yusuke Endoh) wrote in #note-48:

A "use case" is the ultimate goal that the examples aim to achieve via Method#source.

The above snippets show examples of source transformation, instrumentation of code by adding extra code in the method body, displaying the source code of a method to the user, and other use cases which would need more time to find out.
In any case I don't think it's our place to judge whether each usage is "valid" or "correct" or not, it's definitely useful functionality since it's used so much.
For example, some gems might transform the source code of a method as a string, you might argue it's less safe but it's also much simpler than parsing + unparsing and it obviously works well enough for them.

By the way, I tried the method_source gem, and I noticed that it includes heredoc content that lies outside the method definition block.

I think that's just a side effect of method_source currently adding lines until it's valid Ruby source code.
In practice I think there are very few heredocs like this for methods, so it probably does not matter for 99.99% cases.

Note that the current method_source approach is much worse than using the new source_location, because it's incorrect for e.g.

def method
end; any code

e.g.

def foo
end; def bar
end

where it would return the entire code for the source of foo.

This is much more common than heredocs spanning across end.
For example this is common in tests:

def some_test
  some assertions
end unless /mswin|mingw/ =~ RUBY_PLATFORM

The gem currently return the entire snippet, but with source_location it would be correct and stop at end

My point is that the "better" return value depends entirely on the final use case.

Yes, that's fair but that edge case is very rare.
For the final use case of getting an AST, Prism.node_for(Method) or Method#ast is more direct and works even for that edge case.
For the final use case of displaying the method source code it seems nicer to include the heredoc, and OTOH that is somewhat complicated and unintuitive to do from the AST.

Do you have a counter-example where this wouldn't hold?

I don't have one at present. However, such a case could arise through future extensions.
For example, @ko1 (Koichi Sasada) might add a new TracePoint type in the future, and depending on the type, it might be necessary to distinguish between StatementsNode and CallNode.

#21795 is specifically about call/return TracePoint, so that's not an issue.
I don't think anyone would ever want to get a StatementsNode from some #ast method, that's "invisible" in user code (IOW, it doesn't "consume" characters in the source to exist).
So I think for every node that "consumes" characters in the source to exist there is no problem, it's uniquely identified by start/end line/column.

More importantly, CRuby can implement Prism.node_for using node_id without the risk of future ambiguity.
There is no reason to implement such a heuristic hack that tries to select overlapping nodes based on Ruby object type.

IMO using node_id is a hack, and my implementation of Prism.node_for in https://github.com/ruby/prism/pull/3808 is AFAIK fully correct and portable (does not depend on CRuby internals, can easily be supported by all Ruby implementations through the extended source_location from this issue).
There is no "heuristic hack", it's the fully correct and deterministic decision in this context.
Nobody expects the StatementsNode instead of the CallNode for Prism.node_for(Proc).

That implementation also works even with --parser=parse.y, while it wouldn't if it were using node_id.
This is a good illustration that start/end line/column are more universal and well-defined, unlike node_id which is dependent on the parser and order-of-iteration.

I do agree with you that for final use case of getting a Prism AST, #21795 would be more direct and safer (e.g. can check hash of the file).
And there it's totally fine to use node_id internally on CRuby, as it remains an implementation detail not exposed to the user.
So I'd be happy if you help me get that issue approved.

But there are other use cases for source_location to include end line and start/end column as shown in my previous comment, and the current implementation meets them well, except for very rare edge cases where as you say, there is no correct or wrong way.