Feature #13820: Add a nil coalescing operator

• Status: Open
• Assignee: -
• Target version: -

[williamn (William Newbery)]

It would be nice if Ruby had an operator that only considered nil as false, like the null coalescing operators or "Logical Defined-Or operator" (Perl) found in some other languages. Ive seen things like // and //=m ?? and ??=, or ?: used for this.

This would work like || and ||= for short circuiting etc. except that only nil is considered a false condition.

While Ruby considers only "false" and "nil" as false, with everything else true ("", [], {}, etc.) I still find occasionally people trip up when using logical or, || and ||= when the value may be false.

a = 0     || 55 # = 0 Ruby already considers 0, "", etc. as true (oter languages do differ a lot here)
a = 0     ?? 55 # = 0 So no change here
a = nil   || 55 # = 55, nil is false so right side is evaulated.
a = nil   ?? 55 # = 55, again no change
a = false || 55 # = 55, however false is false for logical or
a = false ?? 55 # = false, but its still a non-nil value

For example when doing things like:

def lazy
  @lazy ||= compute_this
end


def fetch(id, **opts)
  host  = opts[:host] || default_host
  https = opts[:https] || true
  port  = opts[:port] || (https ? 443 : 80)
  ...

Normally the intention is to use a default value or compute an action if no value is provided, which if the value may be false then requires special handling, or sometimes is missed and results in a bug.

[shevegen (Robert A. Heiler)]

I am not sure that using a special-purpose operator would make a lot of sense.

I myself use nil primarily as means to indicate a default, "non-set" value. The
moment it is set to a boolean, be it false or true, is for me an indication
that it has been "upgraded" (or set by a user on the commandline etc...)

I do also tend to explicitely query for .nil? on some objects.

By the way, did you actually propose an actual syntax? The two '?'?

I do not think that ?? has any realistic chance for implementation due to
? already being used in ruby - in method definitions or ternary
operator for example. People may wonder why there are so many ? coming out
of nowhere. (For the record, I also consider || to be not pretty ... I
strangely end up using a more verbose but explicit way to set or ensure
defaults in ruby code. I would never write a line such as "
port = opts[:port] || (https ? 443 : 80)" simply because it takes my
brain too long to process what is going on there; my code always ends
up being so simple that I do not have to think about it much at all).

[phluid61 (Matthew Kerwin)]

In perl I find $x // $y useful vs $x || $y because sometimes you want to accept "" and 0 as values.

In ruby the only 'defined' falsey value is false, so I'm not sure how useful this feature is here.

If you're pulling options from a hash, for example, it's probably better to use a signal like h.fetch 'x', y to show that you accept falsey values from the hash, and/or x.nil? ? y : x to show that you explicitly only don't want nil

[williamn (William Newbery)]

shevegen (Robert A. Heiler) wrote:

By the way, did you actually propose an actual syntax? The two '?'?

Not really personally set on any given syntax, just ?? and // are familiar to me from other programming. Although actually for ?? specifically, I guess the fact Ruby uses it in both methods and ternary causes a conflict rather than just one or the other (x.nil?? "was nil" : "not nil"). I wouldn't know if the parser can figure that out or not.

But more the concept that any specific syntax.

I myself use nil primarily as means to indicate a default, "non-set" value. The
moment it is set to a boolean, be it false or true, is for me an indication
that it has been "upgraded" (or set by a user on the commandline etc...)

Hmm, maybe I didn't explain clearly. That is pretty much the pattern I come across repeatedly in Ruby code, and it fails for the false value because false is not "upgraded" when people do "x || my_default".

"a truthy value" || foo("something else") # The operator also short circuits so the method `foo` will never even get called
false || foo("something else") # Left side is falsy, so evaluate the right side, but is often not the intent
nil || foo("something else") # Nil is also falsy, so evaluate the right side

"a truthy value" ?? foo("something else") # String is true and not nil, so nothing changes here
false ?? foo("something else") # This changes. Left side is not nil, so the right side is never evaluated
nil ?? foo("something else") # Like with `||`, left side is nil so evaluate the right side

# so this "does the right thing", as far as my maybe not great example goes
https = opts[:https] ?? true

phluid61 (Matthew Kerwin) wrote:

In perl I find $x // $y useful vs $x || $y because sometimes you want to accept "" and 0 as values.
But not false?

While hash.fetch is nice, I still see || used a lot, in places that maybe wont convert so nice. Also it wont short circuit, in the event the default is not trivial (e.g. with say active record stuff, its easy to have something that goes to the DB without really thinking about it).

opts[:foo] || @foo_config || App.config.foo # Occasionally I see 3 or more chained together
hash[:foo] ||= fetch_foo # fetch doesn't assign the value like this does, and it wont short circuit `fetch_foo`
@lazy ||= calc_lazy # Sometimes used with non-hashes

But yes your right, they can all be done other ways, and maybe the better answer is to discourage || in the first place, but I struggled finding ones as tidy to suggest instead.

opts.fetch(:foo, !@foo_config.nil? @foo_config : App.config.foo)
hash[:foo] = fetch_foo if !hash.has_key?(:foo)

@lazy = calc_lazy if @lazy.nil?
@lazy # needed because get nil if the if condition is false

if @lazy.nil?
  @lazy = calc_lazy
end
@lazy

[phluid61 (Matthew Kerwin)]

williamn (William Newbery) wrote:

phluid61 (Matthew Kerwin) wrote:

In perl I find $x // $y useful vs $x || $y because sometimes you want to accept "" and 0 as values.

But not false?

Not in perl ;)

While hash.fetch is nice, I still see || used a lot, in places that maybe wont convert so nice. Also it wont short circuit, in the event the default is not trivial (e.g. with say active record stuff, its easy to have something that goes to the DB without really thinking about it).

Yes, short-circuit is handy. It's why I was a proponent of &.. Maybe it's okay to add // even if it's only used sometimes.

[nobu (Nobuyoshi Nakada)]

williamn (William Newbery) wrote:

shevegen (Robert A. Heiler) wrote:

By the way, did you actually propose an actual syntax? The two '?'?

Not really personally set on any given syntax, just ?? and // are familiar to me from other programming. Although actually for ?? specifically, I guess the fact Ruby uses it in both methods and ternary causes a conflict rather than just one or the other (x.nil?? "was nil" : "not nil"). I wouldn't know if the parser can figure that out or not.

?? is a string literal, and // is a regexp literal.

def fetch(id, **opts)
  host  = opts[:host] || default_host
  https = opts[:https] || true
  port  = opts[:port] || (https ? 443 : 80)

Why not keyword arguments?

[swrobel (Stefan Wrobel)]

• Subject changed from Add a nill coalescing operator to Add a nil coalescing operator

[bsarrazin (Ben Sarrazin)]

Kotlin has this feature, Swift has this feature, many other languages have this feature.
Ruby needs this feature :D

"a truthy value" || foo("something else") # The operator also short circuits so the method `foo` will never even get called
false || foo("something else") # Left side is falsy, so evaluate the right side, but is often not the intent
nil || foo("something else") # Nil is also falsy, so evaluate the right side

"a truthy value" ?? foo("something else") # String is true and not nil, so nothing changes here
false ?? foo("something else") # This changes. Left side is not nil, so the right side is never evaluated
nil ?? foo("something else") # Like with `||`, left side is nil so evaluate the right side

# so this "does the right thing", as far as my maybe not great example goes
https = opts[:https] ?? true

This is exactly why Ruby needs this feature. The lack of type safety, combined with the fact that many tools are taking "false" string as input is compelling argument to add this to the language.

[shyouhei (Shyouhei Urabe)]

bsarrazin (Ben Sarrazin) wrote in #note-7:

Kotlin has this feature, Swift has this feature, many other languages have this feature.

No. Kotlin does not have this feature (distinguish false and null). It is a really bad idea for you to refer Kotlin to have something like that. Kotlin is a statically typed language, and its || operator does not take nullable values. No confusion over false versus null must happen. That is why they need ?: operator; they need something similar to || which also work for nullables.

So if you want a ruby operator because false and nil are confusing, that's a totally different story than Kotlin's.

PS. I'm not against the feature itself. I'm just telling that other languages have their own design that do not immediately apply here.

[sawa (Tsuyoshi Sawada)]

Your proposal to distinguish nil from false is ad hoc, and is not a real solution for your use case, which is to add a value to a hash only when it does not yet have a corresponding key.

Following your way of doing it, you would still not be able to distinguish opts1 that explicitly has a key-value pair :foo => nil and opts2 that lacks such pair.

opts1 = {foo: nil}
opts2 = {}

opts1[:foo] # => nil
opts2[:foo] # => nil

Particularly, applying your code to update opts1 would overwrite the explicit nil value (?? stands for your proposed feature):

opts1[:foo] ??= "foo"

opts1[:foo] # => "foo"

Ruby is aware of such use case, and has already prepared a real solution: the Hash#key? method. The correct way of doing it is:

opts1[:foo] = "foo" unless opts1.key?(:foo)

Another way of doing it is:

opts1 = {foo: "foo"}.merge(opts1)

[akim (Akim Demaille)]

I second this proposal.

Sure, it is not needed, we can always write things in a different way. As a matter of fact, with such an argument, hardly any improvement should be accepted.

In practice when people use ||, they do mean ?? (whatever its spelling). It just so happens that most of the time, it does what you want, because you happen to not be dealing with Booleans. But the semantics you mean to express is not about "truthness", but about "nilness". And occasionally you get bitten because false does exist, and behaves differently.

Wrt syntax, ?? chains much more naturally than conditionals.

Many of the new languages make the difference, and support this explicitly.

[akim (Akim Demaille)]

shyouhei (Shyouhei Urabe) wrote in #note-8:

bsarrazin (Ben Sarrazin) wrote in #note-7:

Kotlin has this feature, Swift has this feature, many other languages have this feature.

No. Kotlin does not have this feature (distinguish false and null).

I disagree. Kotlin does have the feature which is the topic of this page: a null coalescing operator (https://kotlinlang.org/docs/reference/null-safety.html#elvis-operator). And you are rightfully stating it:

That is why they need ?: operator

which exactly corresponds to the proposal about ??.

[lamont (Lamont Granquist)]

This would cut down on a lot of bugs dealing with lazy initialization of values where false is valid and needs to be remembered:

# hammers on the expensive determination method if it returns false
def should_dosomething?
  @should_dosomething ||= expensively_determine_if_should_do_something
end

# correctly remembers either false or true
def should_dosomething?
  @should_dosomething ??= expensively_determine_if_should_do_something
end

This happens more than you might expect because people learn to use ||= in that situation and wind up accidentally writing the buggy code when they're faced with this problem of remembering an expensive thing that returns booleans.

The ??= operator should be introduced into the language and really the idiom should switch to consistently use it all the time to avoid this bug.

[lamont (Lamont Granquist)]

Here's example code which works around the lack of a ??= operator in the wild:

https://github.com/chef/chef/blob/4d3b847aee1b917bb139862c623e9633d180fb31/lib/chef/chef_fs/file_system/chef_server/data_bag_dir.rb#L43-L48

          def exists?
            if @exists.nil?
              @exists = parent.children.any? { |child| child.name == name }
            end
            @exists
          end

[p8 (Petrik de Heus)]

This is also related to the Hash#fetch_set proposal

cache.fetch_set(key) { calculation }

https://bugs.ruby-lang.org/issues/17342

[TylerRick (Tyler Rick)]

Ruby already has the safe navigation operator ?., added back in Ruby 2.3, which was definitely a step in the right direction with regards to first-class nil-aware operations.

Now we just need a null coalescing operator like most other popular languages have and our support for null-aware operators will be complete! I would jump for joy. Please add this.

Adding this operator would make it elegantly concise and trivially simple to just write what you mean the first time: "if LHS is nil , then use (??) / initialize using (??=) this default value" — instead of being tempted to use the almost-correct but subtly different-meaning ("if LHS is falsy , then...") alternative that we've all ended up using instead, something || default. And then having to "upgrade"/fix the logic to something more complicated/verbose when you inevitably realize that false values were inadvertently also using the logic path you intended only for nil values.

As akim (Akim Demaille) summed up perfectly in #note-10:

In practice when people use ||, they [really] mean ??

It should be easier to do the right thing here, and not so easy (by virtue of it being the only obvious/easy solution provided by the syntax) to shoot yourself in the foot by inadvertently allowing false values to use the same logic path that you only wanted to be used for nil...

If we had the syntax to allow it, it would be a lot easier / more common to allow false values as distinct values from nil — and allow them (without any extra effort) to arrive all the way inside your method/etc. instead of getting mistakenly lumped together with nil values (and accidentally triggering the "default value" path). As Michael Harrington put it... :)

Ruby, as a community, unfairly discriminates against false [...]
false is a perfectly non-nil object, and you're excluding it out of convenience!

---

I would prefer spelling it ?? since that seems the most consistent with the existing ?. operator, and seems like the most popular spelling (used by C#, JavaScript, PHP, PowerShell, and Swift).

(?: as in Kotlin would be a fun spelling, but IMHO it look too much like the ternary operator ? :, which of course is different since it checks for truthy values rather than nullish values, so that similarity could be surprising.)

Shoot, even JavaScript has had this operator since ECMAScript 2020. Are we really going to let JavaScript and PHP be more developer-friendly than Ruby on this point??? Say it ain't so.

[genezys (Vincent Robert)]

Here are my 2 cents regarding this subject, if it can help moving forward with this feature.

Wkipedia has a great article explaining and listing null coalescing operators in other languages: https://en.wikipedia.org/wiki/Null_coalescing_operator
Here is a summary of all operators ordered by usage:

• ??: ATS, C#, JavaScript, PHP, PowerShell, Swift
• No operator: Python, Ruby, Rust, SQL
• ?:: CFML, Kotlin, Objective-C
• !: Freemarker, Haskell
• :-: Bourne-like shells
• |?: F#
• //: Perl
• %||%: R
• If: VB.NET

All languages that do not provide an operator offer a way to perform null coalescing.
In Ruby, you could add a coalesce method with a special implementation on NilClass:

class Object
  def coalesce(value = nil)
    self
  end
end

class NilClass
  def coalesce(value = nil)
    block_given? ? yield : value
  end
end

> "a".coalesce("b") # => "a"
> "a".coalesce { "b" } # => "a"
> nil.coalesce("b") # => "b"
> nil.coalesce { "b" } # => "b"
> false.coalesce(true) # => false
> nil.coalesce(true) # => true

Keep in mind that this does not address the issue of the null coalescing assignment operator (aka ??=)