Bug #9583: Open3 cannot talk correctly with Ruby when there is a large DATA segment passed through stdin. Broken pipe (Errno::EPIPE)

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 2.2.0dev (2014-03-01 trunk 45219) [x86_64-darwin12.0]
• Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: UNKNOWN

[josh.cheek (Josh Cheek)]

$ # Tested against
$ ruby -v
ruby 2.2.0dev (2014-03-01 trunk 45219) [x86_64-darwin12.0]

Example
$ ruby -e 'puts "END"; puts "."*24_568' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'
-e:1:in write': Broken pipe (Errno::EPIPE) from -e:1:in block in '
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:217:in popen_run' from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:99:in popen3'
from -e:1:in `'

If we reduce the size of the data segment by 1 character, it works:
$ ruby -e 'puts "END"; puts "."*24_567' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'

If we take it out of the data segment, it is fine
$ ruby -e 'puts "#" + "."*100_000' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'

[akr (Akira Tanaka)]

• Status changed from Open to Rejected

EPIPE is raised because ruby doesn't read the script file after END (unless the script read from DATA).
So the ruby invoked by Open3.popen3 can exit before i.write is finished.
It actually happen if the script is long enough.
If it is happen, i.write fails with EPIPE.

If the script doesn't contain END (as your last example), ruby reads the entire script.
So the ruby invoked by Open3.popen3 don't exit before i.close is finished.
So EPIPE is not raised.

[josh.cheek (Josh Cheek)]

Makes sense. Seems like it shouldn't cause ruby -c to break, though, because the syntax is actually valid, and it would pass if it was given in a file:

$ ruby -e 'puts "END"; puts "."*24_568' > f.rb

$ ruby -c f.rb
Syntax OK

$ ruby -r open3 -e 'Open3.popen3("ruby", "-c") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read }'
-e:1:in write': Broken pipe (Errno::EPIPE) from -e:1:in block in '
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:217:in popen_run' from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:99:in popen3'
from -e:1:in `'

As surprising as this behaviour seems, it's actually legit.

What's breaking is not ruby -c - this is behaving correctly. As akr noted, Ruby does not read input files past the __END__ marker. The DATA constant is set to an IO which can be used to read further into the file.

When ruby -c finished reading the file, it checks the syntax and then exits. Any attempts to write more data will raise EPIPE. The bug here is actually in the invoking program, not ruby -c itself. This behaviour can be reproduced by piping data into any child process that terminates before seeing EOF on standard input.

[josh.cheek (Josh Cheek)]

I see, I see. Several stars aligned such that this is correct behaviour on all accounts, even though it isn't immediately obvious. Thank you :)