Bug #16797: Array#flatten not checking `respond_to? :to_ary`

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN

[UlyssesZhan (Ulysses Zhan)]

def (a = Object.new).method_missing(...)
  Object.new
end
[a].flatten # TypeError

It should check whether an object responds to to_ary before trying to convert it into an array.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Rejected

Read the error message.

-:4:in `flatten': can't convert Object to Array (Object#to_ary gives Object) (TypeError)

[Dan0042 (Daniel DeLorme)]

Actually what happens here is pretty interesting. This is all done via rb_check_array_type which attempts to convert an object by using to_ary.

If your object defines its own respond_to? method, it is used to determine if to_ary exists and should be called.

a = Object.new
def a.method_missing(...); Object.new; end
def a.respond_to?(...); super; end
[a].flatten #=> [#<Object:0x000055ca978ce058>]

b = Object.new
def b.to_ary; bug!; end
def b.respond_to?(...); false; end
[b].flatten #=> [#<Object:0x000055ca9789eab0>]

Otherwise if the object only has the default respond_to? method, its tries to call method_missing. NoMethodError is rescued and taken to mean that the object does not respond to to_ary, and the object is not flattened.
Without a NoMethodError it means the object effectively responds to to_ary, and the return type must be Array.

a = Object.new
def a.method_missing(...); 42; end
[a].flatten #=> TypeError (can't convert Object to Array (Object#to_ary gives Integer))

def a.method_missing(...); [[42]]; end
[a].flatten #=> [42]

def a.method_missing(...); super; end #NoMethodError triggered via super
[a].flatten #=> [#<Object:0x00005603341f3b18>]

def a.method_missing(...); 42.foobar; end #NoMethodError is actually because of foobar
[a].flatten #=> [#<Object:0x0000560334024300>]

[UlyssesZhan (Ulysses Zhan)]

nobu (Nobuyoshi Nakada) wrote in #note-1:

Read the error message.

-:4:in `flatten': can't convert Object to Array (Object#to_ary gives Object) (TypeError)

Read what I wrote:

It should check whether an object responds to to_ary before trying to convert it into an array.

I know it used to_ary method and got an unexpected result, but in fact calling a.to_ary itself is unexpected because a.respond_to? :to_ary is false.

Detecting NoMethodError is a bit silly.