Bug #21780: Change the default size of Enumerator.produce back to infinity

• Status: Open
• Assignee: -
• Target version: -
• Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN

[zverok (Victor Shepelev)]

In #21701 a new argument size: was introduced, and its default value is nil (unknown).

While I support the new argument, I'd argue that the default should be Float::INFINITY.

Reasoning: By design, Enumerator.produce is infinite (there is no internal condition to stop iteration), and the simplest, most straightforward usages of the method would produce definitely infinite iterators, which the user than can limit with take, or take_while or similar methods.

To produce the enumerator that will stop by itself requires explicit raising of StopIteration, which I expect to be a (slightly) advanced technique, and those who use it might be more inclined to provide additional arguments to clarify the semantics.

While Enumerator#size is hardly frequently used now (other than in #to_set, which started the discussion), it might be in the future, and I believe it is better to stick with more user-friendly defaults.

Now:

# very trivial enumerator, but if you want it to have "proper" size, you need 
# to not forget to use an elaborate argument and type additional 21 characters
Enumerator.produce(1, size: Float::INFINITY, &:succ)

# already non-trivial enumerator, which is hardly frequently used, but the 
# current defaults correspond to its semantics:
Enumerator.produce(Date.today) {
  raise StopIteration if it.tuesday? && it.day.odd?
  it + 1
}

With my proposal:

# trivial, most widespread case:
Enumerator.produce(1, &:succ).size #=> Infinity

# non-trivial case, with the enumerator designer clarifying their
# intention that "we are sure it stops somewhere":
Enumerator.produce(Date.today, size: nil) {
  raise StopIteration if it.tuesday? && it.day.odd?
  it + 1
}

[Eregon (Benoit Daloze)]

I disagree on this one, as written on https://bugs.ruby-lang.org/issues/21701#note-3

I think Enumerator#size should only be non-nil when it is known to be the exact size.
In this case it is not known if it is infinite, so returning Float::INFINITY for the size is "wrong".

One use case I know of for Enumerator#size is to do a progress bar while iterating the Enumerator.
That can only work reliably if the non-nil size is the exact size.
Returning Float::INFINITY when it is not would be misleading, though of course returning nil won't give the actual size, which might simply be not known.

BTW, your examples use Enumerator.new but the text seems to be about Enumerator.produce.
I think either way it applies to both the same way though.

[Eregon (Benoit Daloze)]

Actually for Enumerator.new, it's trivial to not be infinite and does not even need StopIteration, e.g.:

Enumerator.new { |y| y << 1 }.count # => 1

So I guess you meant to use Enumerator.produce instead in your examples above.

[zverok (Victor Shepelev)]

• Description updated (diff)

I think Enumerator#size should only be non-nil when it is known to be the exact size.
In this case it is not know if it is infinite, so returning Float::INFINITY for the size is "wrong".

I would argue that it is known to be infinite: that's how produce works: loops infinitely, unless explicitly stopped by an exception, there is no other way than an exceptional one (while this might seem to be a dumb pun, I actually think that we have a useful distinction here).

So I would argue that the default expectation of the user to "not think about it and trust Ruby to do the sane thing", and the sane thing is "produce is infinite unless you raise that specific exception" (even break wouldn't work... which is kinda unpleasant, but a discussion for another time).

In a rare situation when they'd question the behavior, there is a clearly documented way to adjust it.

BTW, your examples use Enumerator.new but the text seems to be about Enumerator.produce.

Yes, thank you, fixed. The title said what I meant but the code was broken, sorry.

[zverok (Victor Shepelev)]

There are, by the way, other effects of the current default that are, even if minor, still annoying:

Enumerator.produce(1, &:succ).lazy.take(6).size
# Ruby 3.4: => 6    -- which is correct and useful
# Ruby 4.0: => nil  -- which is ... less useful