Bug #7070: sometimes backticks can fail to return data

• Status: Third Party's Issue
• Assignee: -
• Target version: -
• ruby -v: ruby 1.9.3p194 (2012-04-20) [i386-mingw32]

[rogerdpack (Roger Pack)]

Basically, if you have ffmpeg for windows, ex: http://ffmpeg.zeranoe.com/builds

Then run it like this:

enum = ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1

"every so often" the value returned is "" (presumably in error).

Example:

39 times you'll get this:

enum = temp\\ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1
=> "ffmpeg version N-44123-g5d55830 Copyright (c) 2000-2012 the FFmpeg developers\n built on Sep 2 2012 20:23:29 with gcc 4.7.1 (GCC)\n configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib\n libavutil 51. 70.100 / 51. 70.100\n libavcodec 54. 55.100 / 54. 55.100\n libavformat 54. 25.104 / 54. 25.104\n libavdevice 54. 2.100 / 54. 2.100\n libavfilter 3. 15.102 / 3. 15.102\n libswscale 2. 1.101 / 2. 1.101\n libswresample 0. 15.100 / 0. 15.100\n libpostproc 52. 0.100 / 52. 0.100\n[dshow @ 003cc3a0] DirectShow video
devices\n[dshow @ 003cc3a0] "screen-capture-recorder"\n[dshow @ 003cc3a0] DirectShow audio devices\n[dshow @ 003cc3a0] "Microphone (USB Audio Device)"\n[dshow @ 003cc3a0] "virtual-audio-capturer"\ndummy: Immediate exit requested\n"
irb(main):036:0> enum = temp\\ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1

then the 40th will be

enum = temp\\ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1
=> ""

Running using C's "popen" seems to work reliably.
https://gist.github.com/3784971
I would note that jruby seems to suffer from the same difficulty for whatever reason. Running it with other executables
Thanks!

[jonforums (Jon Forums)]

While I can repro with this snippet on Win7 32bit, your ffmpeg command (nonexistent 'dummy' input file?) appears to be causing the problem.

file: backtick_loop.rb

CYCLES = (ENV['BT_TEST_CYCLES'] ||= '200').to_i

puts 'backticking curl...'
CYCLES.times do |i|
data = curl --manual
abort "[#{i}] backticked curl returned no data" if data.nil? or data.empty?
end

puts 'backticking ffmpeg...'
CYCLES.times do |i|
data = C:/Users/Jon/Documents/FFmpeg/bin/ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1
abort "[#{i}] backticked ffmpeg returned no data" if data.nil? or data.empty?
end

puts 'OK'

results

C:\Users\Jon\Documents\RubyDev\sandbox>pik ruby backtick_loop.rb

ruby 1.8.7 (2012-06-29 patchlevel 370) [i386-mingw32]

backticking curl...
backticking ffmpeg...
[15] backticked ffmpeg returned no data

ruby 1.9.3p125 (2012-02-16) [i386-mingw32]

backticking curl...
backticking ffmpeg...
[7] backticked ffmpeg returned no data

ruby 1.9.3p277 (2012-09-25 revision 37029) [i386-mingw32]

backticking curl...
backticking ffmpeg...
[4] backticked ffmpeg returned no data

tcs-ruby 1.9.3p231 (2012-05-25, TCS patched 2012-05-27) [i386-mingw32]

backticking curl...
backticking ffmpeg...
[2] backticked ffmpeg returned no data

ruby 2.0.0dev (2012-09-26 trunk 37036) [i386-mingw32]

backticking curl...
backticking ffmpeg...
[30] backticked ffmpeg returned no data

But given the following cmd.exe ffmpeg invocation ends with the error "dummy: Immediate exit requested" I suspect your ffmpeg command is the cause of the problem. When I remove the -i dummy part (nonexistent input file) I haven't been able to repro using the above script. Try it on your system and see if the failure persists.

C:\Users\Jon\Documents\RubyDev\sandbox>C:\Users\Jon\Documents\FFmpeg\bin\ffmpeg.exe -list_devices true -f dshow -i dummy 2>&1
ffmpeg version N-44727-gbbe9fe4 Copyright (c) 2000-2012 the FFmpeg developers
built on Sep 24 2012 16:51:07 with gcc 4.7.1 (GCC)
configuration: --enable-gpl --enable-version3 --disable-pthreads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 58.100 / 54. 58.100
libavformat 54. 28.101 / 54. 28.101
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100
[dshow @ 0036c260] DirectShow video devices
[dshow @ 0036c260] "Integrated Webcam"
[dshow @ 0036c260] "Google Camera Adapter 0"
[dshow @ 0036c260] "Google Camera Adapter 1"
[dshow @ 0036c260] DirectShow audio devices
[dshow @ 0036c260] "Microphone (High Definition Aud"
dummy: Immediate exit requested

C:\Users\Jon\Documents\RubyDev\sandbox>echo %errorlevel%
1

[jonforums (Jon Forums)]

Given your CRT popen and Java Process experiments, does a more verbose IO.popen (non-shell) snippet reliably work?

It's clever that you're parsing ffmpeg's stderr to enum dshow devices, but is the stderr redirect to stdout plus ffmpeg's "immediate exit request" (??) too much for cmd.exe? Wonder if cmd.exe is somehow inconsistently corrupting the streams rather than ruby being the problem. But then your popen (documented to use the command processor) sample with redirects works.

Interesting...maybe Hiroshi's already on it ;)

[jonforums (Jon Forums)]

While I'm interested in discovering root cause on this, ffmpeg enumerates directshow devices like

http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavdevice/dshow.c;h=120ddd9a64ddfb0eaeaa128eee5600e88d42efb0;hb=HEAD#l252

Looks like COM-in-C.

Perhaps you've already tried, but does win32ole + wmi enable you to directly enumerate the video and audio devices?

[rogerdpack (Roger Pack)]

yes in the few limited tests I've tried (I still need to try a c-based popen that also spawn a new cmd shell IIRC) I am not able to get it to fail if I just capture "ffmpeg.exe 2>&1", only with that exact command line does it fail. Interestingly Python's io.popen fails the same way, so I'm also wondering if it's a cmd.exe bug, when cmd.exe is spanwed with redirection. I haven't tried COM-in-C-in-ffi because...dang that sounds hard though I suppose it would be possible... :)
=roger=

[rogerdpack (Roger Pack)]

• Status changed from Open to Third Party's Issue

I'm going to guess for now that this is a windows bug (or cmd.exe bug)--when you run cmd.exe and it also includes redirection, sometimes it returns nothing (even with python). Weird. Work around might be to use ffi or Process.spawn instead, though I haven't tested it.
Thanks. Another possible workaround is to avoid using things like "2>&1" in your command. Having them in there forces ruby to spawn it as a sub command beneath cmd.exe. Not having them there (or using IO.popen instead, etc.) and you can avoid the cmd.exe part, and thus stay steady and stable...