Bug #20514: Open3#capture3 does not receive correct exit code from Heroku but Kernel#system does

• Status: Feedback
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.1 (2024-04-23 revision c56cd86388) [arm64-darwin23]
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[lakehs (Ashley Lake)]

Heroku has a command-line switch for returning the exit code from a detached process to the calling terminal via their CLI. However, Open3 doesn't appear to receive this exit code properly:

irb(main):007> command = 'heroku run "rails db:migrate:status | grep \"down    \"" --exit-code -a remote_app'
=> "heroku run \"rails db:migrate:status | grep \\\"down    \\\"\" --exit-code -a remote_app"
irb(main):008> params = {}
=> {}
irb(main):009> stdout_str, stderr_str, status = Open3.capture3(command, params)
=> ["", "Running rails db:migrate:status | grep \"down    \" on remote_app... provisioning, run.2070 (Shield-M)\n", #<Process::Status: pid 34912 exit 0>]
irb(main):010> status
=> #<Process::Status: pid 34912 exit 0>
irb(main):011> status.exitstatus
=> 0

In this case, since I know that there are no migrations with status "down", grep should return an exit code of 1, not 0. Running the same command with system gives the expected result:

irb(main):025> command = 'heroku run "rails db:migrate:status | grep \"down    \"" --exit-code -a remote_app'
=> "heroku run \"rails db:migrate:status | grep \\\"down    \\\"\" --exit-code -a remote_app"
irb(main):026> res = system(command)
Running rails db:migrate:status | grep "down    " on ⬢ remote_app... provisioning, run.9202 (Shield-M)

# intermediate output snipped

 ›   Error: Process exited with code 1
 ›   Code: 1
=> false

Switching "up" for "down" in the above grep statement yields true for exit code 0 when called with Kernel#system.

This doesn't happen locally, but since Kernel#system gives the right results but Open3#capture3 does not, I don't think the problem is on Heroku's end.

[k0kubun (Takashi Kokubun)]

I don't think the problem is on Heroku's end

Can you reproduce the issue without using Heroku's CLI? If it has nothing to do with Heroku, it should reproduce with non-Heroku CLIs too.

[nobu (Nobuyoshi Nakada)]

For example, like this?

$ ruby -ropen3 -e 'pp Open3.capture3(pp %q[ssh localhost "echo up | grep \"down    \""])'
"ssh localhost \"echo up | grep \\\"down    \\\"\""
["", "", #<Process::Status: pid 51283 exit 1>]

[lakehs (Ashley Lake)]

nobu (Nobuyoshi Nakada) wrote in #note-2:

For example, like this?

$ ruby -ropen3 -e 'pp Open3.capture3(pp %q[ssh localhost "echo up | grep \"down    \""])'
"ssh localhost \"echo up | grep \\\"down    \\\"\""
["", "", #<Process::Status: pid 51283 exit 1>]

That's perfect. I'm still confused as to why system("heroku run -x COMMAND") receives the exit code from Heroku correctly when Open3 does not, but I understand if that's not relevant to this ticket since there's a fix in https://github.com/heroku/cli/pull/2468.

Edit: the Heroku CLI maintainers don't believe that PR will fix it: https://github.com/heroku/cli/pull/2468#issuecomment-2142428387

[mame (Yusuke Endoh)]

• Status changed from Open to Feedback

Some process may behave differently with Open3.capture3 and system, for example, a process that changes its behavior depending on whether stdio is tty or not.
As far as I know, there is no difference in the interpretetion of command-line string between Open3.capture3 and system.
In any case, I don't think we can proceed with this ticket unless you conduct further analysis on your side.