Feature #22132
Updated by shugo (Shugo Maeda) 3 months ago
## Abstract
How about adding an expression form of `for` that desugars into nested `flat_map`/`map` and `filter` calls.
Here's an example, which computes all pairs of numbers between `0` and `n-1` whose sum is equal to a given value `v`:
```ruby
def foo(n, v)
for i in 0...n,
j in 0...n when i + j == v then
[i, j]
end
end
p foo(10, 10) #=> [[1, 9], [2, 8], [3, 7]...]
```
The above code is desugared as follows:
```ruby
def foo(n, v)
(0...n).flat_map { |i|
(0...n).filter { |j|
i + j == v
}.map { |j|
[i, j]
}
}
end
p foo(10, 10)
```
## Background and Motivation
Some other languages have syntactic sugar that flattens nested code.
For example, Scala has for comprehensions:
```scala
def foo(n: Int, v: Int) =
for i <- 0 until n
j <- 0 until n if i + j == v
yield (i, j)
```
Haskell has do notation:
```haskell
foo :: Int -> Int -> [(Int, Int)]
foo n v = do
i <- [0 .. n-1]
j <- [0 .. n-1]
guard (i + j == v)
pure (i, j)
```
Blocks are often nested deeply in Ruby, so such syntactic sugar is useful.
## Use cases
For comprehensions can be used to flatten nested blocks.
For example,
```ruby
(1..).lazy.flat_map { |z|
(1..z).lazy.flat_map { |x|
(x..z).lazy.filter { |y|
x**2 + y**2 == z**2
}.map { |y|
[x, y, z]
}
}
}.take(3).force
```
can be flattened as follows:
```ruby
for z in (1..).lazy,
x in (1..z).lazy,
y in (x..z).lazy when x**2 + y**2 == z**2 then
[x, y, z]
end.take(3).force
```
For comprehensions can be used not only for Enumerable objects, but also for other objects with lawful `flat_map` and `map` (i.e., any [monad](https://wiki.haskell.org/index.php?title=Monad) whose `map` is the functor map derived from `flat_map`). For example, `flat_map` and `map` must cohere so that nested calls can be flattened:
```ruby
m.flat_map(&f).map(&g) == m.flat_map { |x| f.(x).map(&g) }
```
For example, for comprehensions can be used with [a parser-combinator library](https://github.com/shugo/packrat_parser):
```ruby
class SimpleCalcParser < PackratParser
start_symbol :additive
def additive
for x in multitive << term("+"), y in additive then x + y end |
for x in multitive << term("-"), y in additive then x - y end |
multitive
end
```
Here `<<` is equivalent to Scala's `<~`: it runs both parsers but keeps only the left operand's value.
## Why `then` and `when`?
Scala's `yield` conflicts with the existing `yield` keyword in Ruby, so I chose `then`.
While a bare `for ... then` (no guard) reads a little unnaturally in English, Ruby already gives `then` a value-producing meaning (e.g., `Kernel#then`), so I consider it acceptable.
The guard keyword is not `if` but `when`, because `if` after the source would be ambiguous with the modifier `if`.
## Limitations
The right operand of `in` is `arg_value`, not `expr_value`, to avoid conflicts, so unparenthesized method calls (command calls) must be parenthesized.
## Backward compatibility
* `for x in xs do ... end` (and the newline form) is unchanged: it still iterates via `each` and returns the collection.
* All of the new forms (`for ... then`, `for ... ,`, `for ... when`) were `SyntaxError` before, so no existing program changes meaning.
## Implementation
PoC: https://github.com/ruby/ruby/pull/17500
It's currently implemented only in `parse.y`, not in Prism yet, so requires `--parser=parse.y`.
## Open questions
* Variable scope: the loop variables currently leak, same as `for ... do`. Should a comprehension instead scope them like a block?
* Guards desugar to `filter`, not a lazy `withFilter` as in Scala. So in the strict case, each guard builds one intermediate array; users who want fusion can add `.lazy`. Is `filter` acceptable, or should we add a `with_filter`-like method?