Bug #9583

Open3 cannot talk correctly with Ruby when there is a large DATA segment passed through stdin. Broken pipe (Errno::EPIPE)

Added by Josh Cheek about 1 year ago. Updated about 1 year ago.

[ruby-core:61156]
Status:Rejected
Priority:Normal
Assignee:-
ruby -v:ruby 2.2.0dev (2014-03-01 trunk 45219) [x86_64-darwin12.0] Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: UNKNOWN

Description

$ # Tested against
$ ruby -v
ruby 2.2.0dev (2014-03-01 trunk 45219) [x86_64-darwin12.0]

Example
$ ruby -e 'puts "__END__"; puts "."*24_568' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'
-e:1:in write': Broken pipe (Errno::EPIPE)
from -e:1:in
block in '
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:217:in popen_run'
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:99:in
popen3'
from -e:1:in `'

If we reduce the size of the data segment by 1 character, it works:
$ ruby -e 'puts "__END__"; puts "."*24_567' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'

If we take it out of the data segment, it is fine
$ ruby -e 'puts "#" + "."*100_000' > f.rb; ruby -r open3 -e 'Open3.popen3("ruby") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read; puts e.read }'

History

#1 Updated by Akira Tanaka about 1 year ago

  • Status changed from Open to Rejected

EPIPE is raised because ruby doesn't read the script file after END (unless the script read from DATA).
So the ruby invoked by Open3.popen3 can exit before i.write is finished.
It actually happen if the script is long enough.
If it is happen, i.write fails with EPIPE.

If the script doesn't contain END (as your last example), ruby reads the entire script.
So the ruby invoked by Open3.popen3 don't exit before i.close is finished.
So EPIPE is not raised.

#2 Updated by Josh Cheek about 1 year ago

Makes sense. Seems like it shouldn't cause ruby -c to break, though, because the syntax is actually valid, and it would pass if it was given in a file:

$ ruby -e 'puts "__END__"; puts "."*24_568' > f.rb

$ ruby -c f.rb
Syntax OK

$ ruby -r open3 -e 'Open3.popen3("ruby", "-c") { |i, o, e| i.write File.read "f.rb"; i.close; puts o.read }'
-e:1:in write': Broken pipe (Errno::EPIPE)
from -e:1:in
block in '
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:217:in popen_run'
from /Users/josh/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/open3.rb:99:in
popen3'
from -e:1:in `'

#3 Updated by Charlie Somerville about 1 year ago

As surprising as this behaviour seems, it's actually legit.

What's breaking is not ruby -c - this is behaving correctly. As akr noted, Ruby does not read input files past the __END__ marker. The DATA constant is set to an IO which can be used to read further into the file.

When ruby -c finished reading the file, it checks the syntax and then exits. Any attempts to write more data will raise EPIPE. The bug here is actually in the invoking program, not ruby -c itself. This behaviour can be reproduced by piping data into any child process that terminates before seeing EOF on standard input.

#4 Updated by Josh Cheek about 1 year ago

I see, I see. Several stars aligned such that this is correct behaviour on all accounts, even though it isn't immediately obvious. Thank you :)

Also available in: Atom PDF