Feature #11537: Introduce "Safe navigation operator"

• Status: Closed
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -

[hsbt (Hiroshi SHIBATA)]

I sometimes write following code with rails application:

u = User.find(id)
if u && u.profile && u.profile.thumbnails && u.profiles.thumbnails.large
  ...

or

# Use ActiveSupport
if u.try!(:profile).try!(:thumbnails).try!(:large)
 ...

I hope to write shortly above code. Groovy has above operator named "Safe navigation operator" with "?." syntax.
Ruby can't use "?." operator.

Can we use ".?" syntax. like this:

u = User.find(id)
u.?profile.?thumbnails.?large

Matz. How do you think about this?

[akr (Akira Tanaka)]

• Related to Feature #8191: Short-hand syntax for duck-typing added

[akr (Akira Tanaka)]

• Related to Feature #8237: Logical method chaining via inferred receiver added

[akr (Akira Tanaka)]

• Related to Feature #11034: Nil Conditional added

[rosenfeld (Rodrigo Rosenfeld Rosas)]

+1. Besides Groovy, CoffeeScript also allows that and I've used this feature a lot in my past days with Groovy and still use it all the time with CoffeeScript and have been missing this feature in Ruby for a long time.

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)

[phluid61 (Matthew Kerwin)]

As per the discussion a few years ago in #8191, I'm +1 for .?foo syntax.

I still have a lingering question about the following; does this print "Hello" twice?

a = Object.new
def a.b
  puts "Hello"
end
a.?b.?c

a && a.b && a.b.c    # this does
a.try!(:b).try!(:c)  # this doesn't (?)

I think it should NOT call a.b twice.

[akr (Akira Tanaka)]

• Related to Feature #1122: request for: Object#try added

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Matthew, it shouldn't be implemented as a && a.b && a.b.c, but something like (temp1 = a) && (temp2 = temp1.b) && temp2.c

[matz (Yukihiro Matsumoto)]

I like the idea. My remaining concern is ".?" is too similar to "?." which is chosen by other languages.
We cannot use "?." since it conflicts with a method call with a "*?" predicate method.

Matz.

[sawa (Tsuyoshi Sawada)]

The && and try are different. I am considering the && version.

Since we already have:

a &&= b

which means

a = a && b

By analogy from the above, and given that we want

a && a.b

what about:

a.&&b

or more shortly:

a.&b

[matz (Yukihiro Matsumoto)]

In several languages (Groovy Swift etc.), use ?. but we cannot use it in Ruby, because foo? is a valid method name.
Thus .? is a reasonable alternative for Ruby, I think.

Accepted.

Matz.

[matz (Yukihiro Matsumoto)]

Oh, I made mistake. We will introduce .? (typo fixed already).

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Great news! Thanks! Is this going to be released on next minor or on Ruby 3 only?

[marcandre (Marc-Andre Lafortune)]

Great!

Just to be clear, this will check only for nil, right?

nil.?foo # => nil
false.?foo # => NoMethodError

[nobu (Nobuyoshi Nakada)]

Marc-Andre Lafortune wrote:

Just to be clear, this will check only for nil, right?

I think so, and my implementation too.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset r52214.

---

Safe navigation operator

• compile.c (iseq_peephole_optimize): peephole optimization for
  branchnil jumps.
• compile.c (iseq_compile_each): generate save navigation operator
  code.
• insns.def (branchnil): new opcode to pop the tos and branch if
  it is nil.
• parse.y (NEW_QCALL, call_op, parser_yylex): parse token '.?'.
  [Feature #11537]

[treznick (Tom Reznick)]

Hi,

I think we may have found some unexpected behavior with the .? operator.

If I call the following:

s = Struct.new(:x)
o = s.new()
o.x #=> nil
o.x.nil? #=> true
o.x.?nil? #=> nil
o.x.kind_of?(NilClass) #=> true
o.x.?kind_of?(NilClass) #=> nil
o.x.methods.include?(:nil?) #=> true

While it's arguably a bit peculiar to try to check that nil is nil, in a nil-safe way, .?kind_of?(NilClass) could reasonably return true.

Also this is clearly a bit of a contrived example, it does highlight some of the more unexpected behaviors of the .? operator.

Any chance of clarification?

All best!

Tom Reznick
Software Engineer
Continuity
@threznick

Nobuyoshi Nakada wrote:

Applied in changeset r52214.

---

Safe navigation operator

• compile.c (iseq_peephole_optimize): peephole optimization for
  branchnil jumps.
• compile.c (iseq_compile_each): generate save navigation operator
  code.
• insns.def (branchnil): new opcode to pop the tos and branch if
  it is nil.
• parse.y (NEW_QCALL, call_op, parser_yylex): parse token '.?'.
  [Feature #11537]

[asterite (Ary Borenszweig)]

I know this is already decided and the commit is out there, but since you are adding new syntax and a new feature to the language, I suggest you reconsider https://bugs.ruby-lang.org/issues/9076

With that change, instead of adding special syntax for safe nil traversing, you get generalized syntax for the implicit block argument. Instead of this:

obj.?bar(x, y)

You do:

obj.try &.bar(x, y)

The try method is simply:

class Object
  # But obviously implemented in C for performance reasons
  def try
    yield self unless self.is_a?(NilClass)
  end
end

As I mention in the original issue, foo &.bar simply gets translated by the parser to something like foo { |x| x.bar } (where x doesn't conflict with any other identifier in the method). So it's just a change in the parser, no need to change compile.c, insns.def, etc (although I can understand that nil checking might be optimized with a VM instruction).

My main worry is code like this:

obj.?empty?

That looks confusing to the eye. I associate "?" next to method names to "query" methods, and now when reading an expression "?" can have several meanings (in addition to the ternary operator).

We already have this syntax in Crystal and it's working really well.

[jeremyevans0 (Jeremy Evans)]

Tom Reznick wrote:

Hi,

I think we may have found some unexpected behavior with the .? operator.

If I call the following:

s = Struct.new(:x)
o = s.new()
o.x #=> nil
o.x.nil? #=> true
o.x.?nil? #=> nil
o.x.kind_of?(NilClass) #=> true
o.x.?kind_of?(NilClass) #=> nil
o.x.methods.include?(:nil?) #=> true

While it's arguably a bit peculiar to try to check that nil is nil, in a nil-safe way, .?kind_of?(NilClass) could reasonably return true.

I think it's completely expected that nil.?kind_of?(NilClass) returns nil and not true. The whole point of .? is to return nil without calling the method if the receiver is nil. I'm not sure if .? is a good idea syntax-wise, but if you are going to have it, it shouldn't have special cases for specific methods.

[phluid61 (Matthew Kerwin)]

On 23/10/2015 2:46 AM, [e-mail] wrote:

Issue #11537 has been updated by Jeremy Evans.

Tom Reznick wrote:

Hi,

I think we may have found some unexpected behavior with the .?
operator.

If I call the following:

s = Struct.new(:x)
o = s.new()
o.x #=> nil
o.x.nil? #=> true
o.x.?nil? #=> nil
o.x.kind_of?(NilClass) #=> true
o.x.?kind_of?(NilClass) #=> nil
o.x.methods.include?(:nil?) #=> true

While it's arguably a bit peculiar to try to check that nil is nil,
in a nil-safe way, .?kind_of?(NilClass) could reasonably return true.

I think it's completely expected that nil.?kind_of?(NilClass) returns
nil and not true. The whole point of .? is to return nil without
calling the method if the receiver is nil. I'm not sure if .? is a
good idea syntax-wise, but if you are going to have it, it shouldn't have
special cases for specific methods.

I agree, or put another way: if you're testing for nil in two ways, .? has
higher priority. That makes it a programmer issue, not a ruby one.

[treznick (Tom Reznick)]

Thanks for the thoughtful replies guys! That definitely helps clarify the .? operator
Matthew Kerwin wrote:

On 23/10/2015 2:46 AM, [e-mail] wrote:

Issue #11537 has been updated by Jeremy Evans.

Tom Reznick wrote:

Hi,

I think we may have found some unexpected behavior with the .?
operator.

If I call the following:

s = Struct.new(:x)
o = s.new()
o.x #=> nil
o.x.nil? #=> true
o.x.?nil? #=> nil
o.x.kind_of?(NilClass) #=> true
o.x.?kind_of?(NilClass) #=> nil
o.x.methods.include?(:nil?) #=> true

While it's arguably a bit peculiar to try to check that nil is nil,
in a nil-safe way, .?kind_of?(NilClass) could reasonably return true.

I think it's completely expected that nil.?kind_of?(NilClass) returns
nil and not true. The whole point of .? is to return nil without
calling the method if the receiver is nil. I'm not sure if .? is a
good idea syntax-wise, but if you are going to have it, it shouldn't have
special cases for specific methods.

I agree, or put another way: if you're testing for nil in two ways, .? has
higher priority. That makes it a programmer issue, not a ruby one.

[trans (Thomas Sawyer)]

Yukihiro Matsumoto wrote:

I like the idea. My remaining concern is ".?" is too similar to "?." which is chosen by other languages.
We cannot use "?." since it conflicts with a method call with a "*?" predicate method.

Maybe there is a better syntax by requiring a space:

u ? .profile ? .thumbnails ? .large

In this way is more an extension of the ternary operator -- the initial dot on the method signals the difference. This should also allow:

u ? .profile ? .thumbnails ? .large : default

[wycats (Yehuda Katz)]

Yukihiro Matsumoto wrote:

I like the idea. My remaining concern is ".?" is too similar to "?." which is chosen by other languages.
We cannot use "?." since it conflicts with a method call with a "*?" predicate method.

Matz.

I agree with this concern. I also agree with the idea of trying to make it more like a traditional infix operator by encouraging spaces.

What about:

u && .profile && .thumbnails && .large

As shorthand for:

u && u.profile && u.profile.thumbnails && u.profile.thumbnails.large

It reads nicely to me and its meaning is clear. It also doesn't conflict with Ruby's method predicates (? suffix).

Matz, what do you think?

[nobu (Nobuyoshi Nakada)]

What does sole .profile do?
If it requires && just before it, a space should not be between them.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Yehuda Katz wrote:

As shorthand for:

u && u.profile && u.profile.thumbnails && u.profile.thumbnails.large

Actually, it would be a shorthand for something like:

u && (tmp1 = u.profile) && (tmp2 = tmp1.thumbnails) && tmp2.large

But I still prefer the syntax implemented in trunk as it's similar to other languages and will be more familiar for those used to the operator in Groovy and CoffeeScript, for example.

[DerKobe (Philip Claren)]

Thomas Sawyer wrote:

In this way is more an extension of the ternary operator -- the initial dot on the method signals the difference. This should also allow:

u ? .profile ? .thumbnails ? .large : default

Although I agree that .? is not that intuitive to read, the extended ternary operator would have a problem: boolean false passes for the safe navigator (because it's a valid value) but not for the ternary operator.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

On the other hand, the problem with using "u?.profile" is that "u?" is a valid method, but if Ruby required an space for using the new operator, we could get it working very similarly to Groovy and CoffeeScript:

u ?.profile ?.thumbnails ?.large || default

[mame (Yusuke Endoh)]

Rodrigo Rosenfeld Rosas wrote:

u ?.profile ?.thumbnails ?.large || default

Consider a method +.

str.?+"foo" causes no conflict, but str ?.+"foo" will be parsed as str(?. + "foo").

--
Yusuke Endoh [e-mail]

[rosenfeld (Rodrigo Rosenfeld Rosas)]

I believe you are referring to implementation details and in this case I can't really opinionate, but with regards to syntax, since "?." is not a valid code, it should be interpreted as "?.+" and "foo" would be parsed as its argument.

But if this is too complicate to implement, I'm also fine with using ".?".

[mame (Yusuke Endoh)]

"?." is a valid code. Try: p ?. + "foo"

--
Yusuke Endoh [e-mail]

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Thanks for pointing me that. Everyday learning a new hidden feature from Ruby ;)

[mame (Yusuke Endoh)]

Rodrigo Rosenfeld Rosas wrote:

Everyday learning a new hidden feature from Ruby ;)

Me too. Ruby has infinite possibilities.

--
Yusuke Endoh [e-mail]

[matz (Yukihiro Matsumoto)]

I don't think u && .profile && .thumbnails && .large is acceptable for some reasons:

• it's longer
• unclear

than proposed .?. Instead, &. or .& can be alternatives. But my concern for the idea is && does handle false values (false and nil), where proposed behavior only accepts nil.

Matz.

The same discussion happens to be on TypeScript and ES6 worlds.
Using .. instead of ?. or .? because it's way more clear when you are using the ternary ? : operator on the same line.

If it's not a conflict in Ruby syntax perhaps worths looking at

https://esdiscuss.org/topic/existential-operator-null-propagation-operator#content-65
https://github.com/Microsoft/TypeScript/issues/16#issuecomment-152275052

Posting the markdown info from there:

This would be amazing operator!! Especially for ES6/ES7/TypeScript - and why not Ruby ?

var error = a.b.c.d; //this would fail with error if a, b or c are null or undefined.
var current = a && a.b && a.b.c && a.b.c.d; // the current messy way to handle this
var currentBrackets = a && a['b'] && a['b']['c'] && a['b']['c']['d']; //the current messy way to handle this
var typeScript = a?.b?.c?.d; // The typescript way of handling the above mess with no errors
var typeScriptBrackets = a?['b']?['c']?['d']; //The typescript of handling the above mess with no errors

However I propose a more clear one - as not to confuse ? from the a ? b : c statements with a?.b statements:

var doubleDots = a..b..c..d; //this would be ideal to understand that you assume that if any of a, b, c is null or undefined the result will be null or undefined.
var doubleDotsWithBrackets = a..['b']..['c']..['d'];

For the bracket notation, I recommend two dots instead of a single one as it's consistent with the others when non brackets are used. Hence only the property name is static or dynamic via brackets.

Two dots, means if its null or undefined stop processing further and assume the result of expression is null or undefined. (as d would be null or undefined).

Two dots make it more clear, more visible and more space-wise so you understand what's going on.

This is not messing with numbers too - as is not the same case e.g.

1..toString(); // works returning '1'
var x = {};
x.1 = {y: 'test' }; //fails currently
x[1] = {y: 'test' }; //works currently
var current = x[1].y; //works
var missing= x[2].y; //throws exception
var assume= x && x[2] && x[2].y; // works but very messy

About numbers two options: Your call which one can be adopted, but I recommend first one for compatibility with existing rules!

1. Should fail as it does now (x.1.y == runtime error)

var err = x..1..y; // should fail as well, since 1 is not a good property name, nor a number to call a method, since it's after x object.

2. Should work since it understands that is not a number calling a property from Number.prototype

var err = x..1..y; // should work as well, resulting 'test' in this case
var err = x..2..y; // should work as well, resulting undefined in this case

With dynamic names:

var correct1 = x..[1]..y; //would work returning 'test'
var correct2 = x..[2]..y; //would work returning undefined;

What do you think folks?

P.S. foo?.bar and foo?['bar'] syntax would work too.

However the using both current ? : operator and ?. might be very confusing on the same line.

e.g. using ?. and ?['prop']

var a = { x: { y: 1 } };
var b = condition ? a?.x.?y : a?.y?.z;
var c = condition ? a?['x']?['y'] : a?['y']?['z'];

as opposed to double dots .. and ..['prop']

var a = { x: { y: 1 } };
var b = condition ? a..x..y : a..y..z;
var c = condition ? a..['x']..['y'] : a..['y']..['z'];

Which one does look more clear to you?

[phluid61 (Matthew Kerwin)]

a..b already means something in ruby

I'm happy with Matz's acceptance of .? especially in light of all the
discussions that lead up to it.

[trans (Thomas Sawyer)]

Laurentiu Macovei wrote:

The same discussion happens to be on TypeScript and ES6 worlds.
Using .. instead of ?. or .? because it's way more clear when you are using the ternary ? : operator on the same line.

If it's not a conflict in Ruby syntax perhaps worths looking at

.. wouldn't work. But that reminds me. Was ! ever considered?

u!profile!thumbnails!large

[trans (Thomas Sawyer)]

Philip Claren wrote:

Thomas Sawyer wrote:

In this way is more an extension of the ternary operator -- the initial dot on the method signals the difference. This should also allow:

u ? .profile ? .thumbnails ? .large : default

Although I agree that .? is not that intuitive to read, the extended ternary operator would have a problem: boolean false passes for the safe navigator (because it's a valid value) but not for the ternary operator.

Is allowing false necessary/useful? On the other hand, if it is, then might a nil-only ternary operator be useful too (regardless of this issue)?

[nobu (Nobuyoshi Nakada)]

Thomas Sawyer wrote:

.. wouldn't work. But that reminds me. Was ! ever considered?

u!profile!thumbnails!large

No, it's a unary operator.

[matz (Yukihiro Matsumoto)]

Thomas, that reminds me of old UUCP addresses (grin).
But ,as Nobu pointed out, it can be parsed as u(!profile(!thumbnails(!large))) unfortunately.

Matz.

[enebo (Thomas Enebo)]

How about ''? We can pay homage to Windows file delimeter?

a\b\c

I just scanned lexer and I cannot think of a reason off the top of my head why not...what does \ mean? who knows...what for .? mean? I don't know that either. This suggestion is twice as efficient though since it only uses one character.

-Tom

[normalperson (Eric Wong)]

[e-mail] wrote:

How about ''? We can pay homage to Windows file delimeter?

a\b\c

I just scanned lexer and I cannot think of a reason off the top of my head why not...what does \ mean? who knows...what for .? mean? I don't know that either. This suggestion is twice as efficient though since it only uses one character.

It's already used for continuing long lines.

[matz (Yukihiro Matsumoto)]

I think about this for a while, and thinking of introducing &. instead of .?, because:

• .? is similar to ?. in Swift and other languages, but is different anyway.
• Since ? is a valid suffix of method names in Ruby, we already see a lot of question marks in our programs.
• u&.profile reminds us as short form of u && u.profile.

But behavior of &. should be kept, i.e. it should skip nil but recognize false.

Matz.

[enebo (Thomas Enebo)]

Eric Wong wrote:

[e-mail] wrote:

How about ''? We can pay homage to Windows file delimeter?

a\b\c

I just scanned lexer and I cannot think of a reason off the top of my head why not...what does \ mean? who knows...what for .? mean? I don't know that either. This suggestion is twice as efficient though since it only uses one character.

It's already used for continuing long lines.

well it is but that is not really an issue since it only will acknowledge '' if right before end of line (lexer just says spaceSeen and loops back up to top of lexer for its next token). What is really weird to me is if it isn't at the end of the line it returns as the token '\'. I see nowhere in the grammar where ruby acknowledges this as a valid token. Any other literal escaping happens within lexing. Did I just find a vestigial organ or am I missing something simple?

In any case it looks like Matz might have picked something other than .?, which was main reason I came up with \ as an idea (I feel reversing order from other languages will confuse them more than it will help -- so pick something different altogether).

-Tom

[nobu (Nobuyoshi Nakada)]

Binary operators implicitly continue the next line, without a backslash.

e.g.,

foo +
1

is same as foo + 1.

If \ were become a binary operator, there is an ambiguity.

foo \
1

is foo(1) (current interpretation) or foo\1?

[shugo (Shugo Maeda)]

• Status changed from Closed to Open

Yukihiro Matsumoto wrote:

I think about this for a while, and thinking of introducing &. instead of .?, because:

It sounds nice because &. is more friendly to syntax highlighting of my favorite editor,
but le me play devil's advocate here.

• .? is similar to ?. in Swift and other languages, but is different anyway.

This also applies to ->, but -> is accepted now, so it's not a strong reason.

• Since ? is a valid suffix of method names in Ruby, we already see a lot of question marks in our programs.

& also has other roles such as a binary operator and the prefix for block arguments.
The current syntax allows x.&y as a valid expression, whose behavior is the same as x & y,
so it may be confusing.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Yukihiro Matsumoto wrote:

But behavior of &. should be kept, i.e. it should skip nil but recognize false.

I'm not sure I understood exactly what you meant by this.

Did you mean .? and &. would be implemented in the same way?

For example: false.?inexistent will raise. Should false&.inexisting raise or return false?

I'd like to see it always raising no matter you decide for ".?" or "&.". I don't see any reasons why one would like to call a method conditionally in the false object. If this happens it's most likely to be a bug from the method returning false.

[nobu (Nobuyoshi Nakada)]

Rodrigo Rosenfeld Rosas wrote:

Did you mean .? and &. would be implemented in the same way?

Yes.

For example: false.?inexistent will raise. Should false&.inexisting raise or return false?

It will raise a NoMethodError.

[ko1 (Koichi Sasada)]

I have weak objection because foo&.bar seems check nil and false.

Maybe this is because I read this expression as "foo and foo.bar", which expression checks also false.

[enebo (Thomas Enebo)]

Nobuyoshi Nakada wrote:

Binary operators implicitly continue the next line, without a backslash.

e.g.,

foo +
1

is same as foo + 1.

If \ were become a binary operator, there is an ambiguity.

foo \
1

is foo(1) (current interpretation) or foo\1?

I am willing to withdraw this proposal and it is potentially ambiguous (although still able to be properly parsed) but I can count the number of times I have seen a rubyist use a line continuation with zero fingers :) So I do not think they confusion would be very large.

-Tom

[[e-mail] (Uwe Kubosch)]

Yukihiro Matsumoto wrote:

I think about this for a while, and thinking of introducing &. instead of .?, because:

• .? is similar to ?. in Swift and other languages, but is different anyway.
• Since ? is a valid suffix of method names in Ruby, we already see a lot of question marks in our programs.
• u&.profile reminds us as short form of u && u.profile.

But behavior of &. should be kept, i.e. it should skip nil but recognize false.

I must say I am very happy with this change. "&." is much easier to read than ".?" .

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset r52895.

---

node.c: NODE_QCALL

• node.c (dump_node): dump NODE_QCALL. [Feature #11537]
  http://twitter.com/watson1978/status/673042429931446272