Bug #9251: ! operator has lower precedence than = in an assignment expression

• Status: Third Party's Issue
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -
• ruby -v: ruby 2.0.0p353 (2013-11-22 revision 43784) [i686-linux]
• Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN

[rits (First Last)]

irb(main):001:0> !a = 1
(irb):1: warning: found = in conditional, should be ==
=> false
irb(main):002:0> a
=> 1

! is supposed to have higher precedence than =, so !a = 1 should be (!a) = 1, and thus an error, not !(a = 1)

[shyouhei (Shyouhei Urabe)]

When you want to use it in while loop your supposition is not that concrete.

while ! line = gets do
....
end

[rits (First Last)]

shyouhei (Shyouhei Urabe) wrote:

When you want to use it in while loop your supposition is not that concrete.

not clear what you're saying

while ! line = gets do
....
end

your example merely illustrates the current behavior, it does not address its correctness

First of all, it doesn't matter where the expression appears, the rules of precedence don't vary (at least I would hope not), so there is no reason to introduce context into the discussion, is there?

The precedence rules are pretty clearly spelled out in "the ruby programming language" book, the ! operator has higher precedence than the = operator, therefore "!a = b" is "(!a) = b" not "!(a = b)"

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Feedback
• Priority changed from Normal to 3

Then do you want it be a syntax error?
Assignment to a negated expression makes no sense.
Or can you elaborate its semantics?

[rits (First Last)]

nobu (Nobuyoshi Nakada) wrote:

Then do you want it be a syntax error?
Assignment to a negated expression makes no sense.
Or can you elaborate its semantics?

It seems self evident to me that the specified rules of precedence should hold. In this case it would be a syntax error.

[shyouhei (Shyouhei Urabe)]

• Category set to doc

So this is a doc issue. The book is wrong here. We have never changed operator preceedence for decades.

[shyouhei (Shyouhei Urabe)]

I would also like to say that Ruby is here for you to have your better life, not to force you live in a inconvenient specification.

[rits (First Last)]

shyouhei (Shyouhei Urabe) wrote:

So this is a doc issue. The book is wrong here.

How can you be sure? Matz hasn't weighed in.

Obviously = can't have globally (with respect to all other operators) higher precedence than !, so the argument is for an exception that allows ! to normally have the highest precedence but be lower than only =.

I don't think such exceptions help anyone. It's more helpful to have clear sensible logical rules.

[rits (First Last)]

shyouhei (Shyouhei Urabe) wrote:

I would also like to say that Ruby is here for you to have your better life, not to force you live in a inconvenient specification.

Ruby is a programming language not some loose colloquial jargon. Don't know about you, but my life does not get better when a language I am using does not follow its specification.

[shyouhei (Shyouhei Urabe)]

rits (First Last) wrote:

shyouhei (Shyouhei Urabe) wrote:

So this is a doc issue. The book is wrong here.

How can you be sure? Matz hasn't weighed in.

I am sure becasue I know it was matz himself who wrote this particular syntax long before the book.

https://github.com/ruby/ruby/blame/trunk/parse.y#L854

[mame (Yusuke Endoh)]

I don't have the book, but I guess it is not wrong.

Perhaps the book explains the precedence rules of the expression. But the left-hand side of = is not an expression. So there is nothing wrong in the book and implementation.

--
Yusuke Endoh [e-mail]

[zzak (zzak _)]

@shyouhei (Shyouhei Urabe) if this is our doc bug can you assign it to me with conclusion?

[rits (First Last)]

mame (Yusuke Endoh) wrote:

I don't have the book, but I guess it is not wrong.

Perhaps the book explains the precedence rules of the expression. But the left-hand side of = is not an expression. So there is nothing wrong in the book and implementation.

The book has a table of all operators grouped and sorted by precedence. ! has higher precedence than =. The rules of precedence are precisely for determining which operator binds tighter or has precedence in an expression. "!a = true" is an expression and according to ruby's rules of precedence should mean "(!a) = true" and thus an error. The current implementation violates the rules of precedence from the book.

[mame (Yusuke Endoh)]

"!a = true" is an expression as a whole, but the left of = is not an expression, so the rule does not apply here.
The left side is what we call "lhs" in Ruby. Roughly speaking, we can write only a "mutable" subset of the expression in "lhs", such as variable, field access, and array access.

--
Yusuke Endoh [e-mail]

[rits (First Last)]

mame (Yusuke Endoh) wrote:

"!a = true" is an expression as a whole, but the left of = is not an expression, so the rule does not apply here.
The left side is what we call "lhs" in Ruby. Roughly speaking, we can write only a "mutable" subset of the expression in "lhs", such as variable, field access, and array access.

--
Yusuke Endoh [e-mail]

your assertion that rules of precedence don't apply to lvalues although true in practice, is not in the book (at least I have not noticed it), this exception to the general rule, means that the precedence table is wrong/incomplete, since the rules vary by context

a = b && c means a = (b && c)

but

a && b = c means a && (b = c)

and

a && b = c && d means a && (b = (c && d))

so in the same expression, = has both a higher and lower precedence than &&, depending on which side you look

this variation/exception/special casing, ostensibly for convenience, does not in practice help anyone, it's better to have a few clear logical rules, than a multitude of magical exceptions

[rits (First Last)]

this exception leads to bizarre, visually difficult to parse and likely unintended legal expressions:

a * -b = -c / d means a * -(b = (-c / d))

how's that better than if the precedence rules were universal so the above would mean

(a * -b) = (-c / d) and thus an error

in cases like these == is likely intended, so a syntax error is much better than a legal but nonsensical expression

[nobu (Nobuyoshi Nakada)]

(13/12/17 9:28), rits (First Last) wrote:

so in the same expression, = has both a higher and lower precedence than &&, depending on which side you look

Operators have not only precedence, but also associativity.
Assignment operator hash right associativity.

[nobu (Nobuyoshi Nakada)]

(13/12/17 9:28), rits (First Last) wrote:

so in the same expression, = has both a higher and lower precedence than &&, depending on which side you look

Operators have not only precedence, but also associativity.
Assignment operator has right associativity.

[rits (First Last)]

Operators have not only precedence, but also associativity. Assignment operator has right associativity.

Associativity is not involved here: a && b = c && d

The associativity of an operator specifies the order of evaluation when the same operator (or operators with the same precedence) appear sequentially in an expression

= and && are different operators with different precedence

As I said, in the above expression, = shows both higher (on the left) and lower (on the right) precedence than && at the same time.

You did not address my last post, it's pretty clear there that this lvalue exception leads to nothing but trouble.

[mame (Yusuke Endoh)]

I think you are confused because you know lvalue. In C, lvalue is indeed an expression. But in Ruby, "lhs" (not lvalue) is NOT an expression. This is a matter of taste of the language design.

As the thought, the book may be slightly unkind if it lists "=" in the precedence table without notes. (Again, I don't have the book.)

Anyway, the current behavior is useful and actually used for some people. It must not change to an inconvenient behavior just because of a philosophical reason.

--
Yusuke Endoh [e-mail]

[rits (First Last)]

mame (Yusuke Endoh) wrote:

I think you are confused because you know lvalue. In C, lvalue is indeed an expression. But in Ruby, "lhs" (not lvalue) is NOT an expression. This is a matter of taste of the language design.

As the thought, the book may be slightly unkind if it lists "=" in the precedence table without notes. (Again, I don't have the book.)

Anyway, the current behavior is useful and actually used for some people. It must not change to an inconvenient behavior just because of a philosophical reason.

--
Yusuke Endoh [e-mail]

Note that I have not asserted that lfs is or should be an expression, so your focus on this is a straw man, the issue is whether it's necessary or desirable to have different precedence rules for the lfs of = than for rhs.

I submit it is neither necessary nor desirable. a * -b = -c / d should mean (a * -b) = (-c / d) per the spec, and produce an error. That's the path of logic, clarity, simplicity and least surprise. For it to mean a * -(b = (-c / d)) is beyond useless, it's dangerous, will conceal bugs.

[duerst (Martin Dürst)]

• File parse.y parse.y added
• Status changed from Feedback to Third Party's Issue
• Assignee set to pragdave (Dave Thomas)

rits (First Last) wrote:

Note that I have not asserted that lfs is or should be an expression, so your focus on this is a straw man, the issue is whether it's necessary or desirable to have different precedence rules for the lfs of = than for rhs.

There are no different precedence rules.

I submit it is neither necessary nor desirable. a * -b = -c / d should mean (a * -b) = (-c / d) per the spec, and produce an error. That's the path of logic, clarity, simplicity and least surprise. For it to mean a * -(b = (-c / d)) is beyond useless, it's dangerous, will conceal bugs.

The spec isn't in the book. A draft of an "official" spec can be found at http://www.ipa.go.jp/files/000011432.pdf. Other spec-like grammars can be found e.g. at http://web.njit.edu/all_topics/Prog_Lang_Docs/html/ruby/yacc.html or http://www.cse.buffalo.edu/~regan/cse305/RubyBNF.pdf.

The actual spec is given by the implementation, in particular the file parse.y, and by yacc/bison (see e.g. http://www.gnu.org/software/bison/manual/bison.html), which is used to produce executable code from the grammar in parse.y, or in other words by the LALR parser generator algorithm.

Parsing isn't necessarily an easy subject, and the Ruby grammar isn't exactly the simplest of grammars. I have written a strongly simplified grammar, which is attached. This can be run on any system where bison is available; a simple one-liner is:

bison -v parse.y && gcc parse.tab.c && ./a.out
(use ./a.exe on Cygwin or similar)
The simplified grammar doesn't allow any spaces, and will produce an error on the first newline. But expressions such as a*!b=!c/d; can easily be tried, and the workings of the parser followed ('!' is interpreted as unary minus). Also, a look at the file parse.output (produced by the -v option to bison) will show lots of details about the grammar.

Studying this simple grammar with some examples shows that LALR is capable of figuring out that the restriction on LHS is more 'important' than the priority of operations with higher priority than '='.

The Ruby grammar is in many ways more fluid than most other programming language grammars. The fact that semicolons and parentheses around argument lists are mostly optional is a good example. You seem to have found a corner of the Ruby grammar that can lead to confusion and isn't documented very well. For the moment, I have assigned this issue to Dave, the author of the book you cite, and marked it as "third party" (because the book isn't part of what's tracked here), but I guess there are also ways to improve the documentation inside the Ruby source code.

[rits (First Last)]

Dave Thomas is the author of "programming ruby" not the "the ruby programming language" to which I referred.

[rits (First Last)]

rits (First Last) wrote:

Dave Thomas is the author of "programming ruby" not the "the ruby programming language" to which I referred.

matz is the author of "the ruby programming language", so if he confirms the current behavior as correct, he might be inclined to add a clarification there (if not present already), so you should probably assign this to him, not Dave

[rits (First Last)]

Studying this simple grammar with some examples shows that LALR is capable of figuring out that the restriction on LHS is more 'important' than the priority of operations with higher priority than '='.

You say it almost with admiration as if this is a great feature, but as I pointed out it's worse than worthless.

being able to write "while ! a = b" is hardly a desirable feature, it's a poor way to communicate intent (this "feature" isn't documented and not intuitive, as it goes against the precedence table that is well documented, so many/most would be surprised by it) Close to 100% would write "while !(a = b)" or at least "while not a = b" (not is documented having lower precedence than =)

and there clear benefits in a * -b = -c / d producing an error, rather than silently evaluating an almost certainly unintended expression.

[duerst (Martin Dürst)]

• Assignee changed from pragdave (Dave Thomas) to matz (Yukihiro Matsumoto)

Sorry about the confusion with the books, but the "pickaxe" has exactly the same issue, too.

As for "almost with admiration", that might apply to LALR, but not to the syntax. Every syntax has some advantages and some disadvantages, and these can be discussed endlessly. Ruby has a much more flexible syntax than e.g. Pascal. That has big benefits in many cases, but may also have some problems in corner cases. In general, it's impossible for programming language syntax to be easy to write and catch all programmer errors at the same time :-(.