Escaping colon in command line argument

[Bouke]

Trying to do some basic capturing.
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0'] # capturing sound from the first sound device
...
proc.start(cmd, args)
returns exitcode 6, exitstatus 1

I'm pretty sure there is something with the colon, but all escape methods I've tried fail.
running:
ffmpeg -thread_queue_size "2048" -framerate "25" -f "avfoundation" -i :0 /Volumes/Data/tst.mov
works just fine

Any ideas how to make this fly?
Thx,
Bouke

[JonB]

@Bouke
Please actually test the same command line arguments from Qt as you say work on the command line. For example, the command line has :0 but the Qt has none:0. The Qt apparently uses the first sound device, your command line seems to be trying to pass /Volumes/Data/tst.mov. And so on. How are we/you supposed to know whether that makes a difference??

proc.start(cmd, args)

returns exitcode 6, exitstatus 1

QProcess.start() does not return anything. What do you actually mean (which QProcess methods) for each of these? If QProcess::exitStatus() returns 1 that is QProcess::CrashExit.

Colons (:) do not need escaping.

Make your code report any QProcess.errorOccurred signal and print anything which arrives on standard output/error. Often you will be receiving a message there, e.g. a "usage" from ffmpeg.

[Bouke]

@JonB said in Escaping colon in command line argument:

@Bouke
Please actually test the same command line arguments from Qt as you say work on the command line. For example, the command line has :0 but the Qt has none:0. The Qt apparently uses the first sound device, your command line seems to be trying to pass /Volumes/Data/tst.mov. And so on. How are we/you supposed to know whether that makes a difference??

Hi Jon,
Sorry, my bad.
none:0 and :0 are the same, or 'should' behave the same.
Syntax for FFmpeg is videodeviceNo:audiodeviceNo, where it's allowed to leave one blank.

proc.start(cmd, args)

returns exitcode 6, exitstatus 1

QProcess.start() does not return anything. What do you actually mean (which QProcess methods) for each of these? If QProcess::exitStatus() returns 1 that is QProcess::CrashExit.

here is my code :
self.proc = QProcess(self._main)
self.proc.finished.connect(self.renderdone)
self.proc.setProcessChannelMode(self.proc.MergedChannels)
self.proc.start(cmd, args)
self.proc.readyReadStandardOutput.connect(lambda: self.readStdOutput(self.proc))

I 'it return' I ment the output of self.renderdone

Colons (:) do not need escaping.

I hope so, but if I replace "none:0" with just "0", FFmpeg will happily take the first video device.
Hence I'm thinking the colon is the culprit.

Make your code report any QProcess.errorOccurred signal
How do I do that?

and print anything which arrives on standard output/error. Often you will be receiving a message there, e.g. a "usage" from ffmpeg.

self.procMergedChannels returns only the startup of FFmpeg, nothing after that
like:

ffmpeg version 4.3.git Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 12.0.0 (clang-1200.0.32.29)
configuration: --enable-videotoolbox --enable-nonfree --enable-gpl --enable-lv2 --enable-openssl --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libvpx --enable-libxvid --enable-libvidstab --enable-libaom --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libtheora --enable-libfdk-aac --enable-libwebp --enable-libsrt --enable-decklink --disable-debug --disable-doc --disable-shared --enable-pthreads --enable-static --enable-small --enable-version3 --extra-cflags='-I/Users/bouke/workspace/include -I/Users/bouke/ffmpeg-build-script/include' --extra-ldexeflags= --extra-ldflags='-L/Users/bouke/workspace/lib -L/Users/bouke/ffmpeg-build-script/include' --extra-libs='-ldl -lpthread -lm -lz' --pkgconfigdir=/Users/bouke/workspace/lib/pkgconfig --pkg-config-flags=--static --prefix=/Users/bouke/workspace
libavutil 56. 63.101 / 56. 63.101
libavcodec 58.117.101 / 58.117.101
libavformat 58. 65.101 / 58. 65.101
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7. 96.100 / 7. 96.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100

And then crickets....

[JonB]

@Bouke
Like I said, start by testing (and showing) identical commands for QProcess::start() and what works from the command line. Not approximately the same but the same.

Make your code report any QProcess.errorOccurred signal

How do I do that?

By attaching a slot to to it.

[Bouke]

@JonB said in Escaping colon in command line argument:

@Bouke
Like I said, start by testing (and showing) identical commands for QProcess::start() and what works from the command line. Not approximately the same but the same.

Trust me, I've tried with identical commands. Both none:0 and :0 work from te command line, but not from Python.
(I did hundreds of tests...)

Make your code report any QProcess.errorOccurred signal

How do I do that?

By attaching a slot to to it.

Ok,
I've added:
self.proc.errorOccurred.connect(self.erroroccured)

def erroroccured(self, _err):
    print("errorOccured", _err)
    print(type(_err))
    print(dir(_err))

Slot prints:
errorOccured 1
<class 'PyQt5.QtCore.QProcess.ProcessError'>
['abs', 'add', 'and', 'bool', 'ceil', 'class', 'delattr', 'dict', 'dir', 'divmod', 'doc', 'eq', 'float', 'floor', 'floordiv', 'format', 'ge', 'getattribute', 'getnewargs', 'gt', 'hash', 'index', 'init', 'init_subclass', 'int', 'invert', 'le', 'lshift', 'lt', 'mod', 'module', 'mul', 'ne', 'neg', 'new', 'or', 'pos', 'pow', 'radd', 'rand', 'rdivmod', 'reduce', 'reduce_ex', 'repr', 'rfloordiv', 'rlshift', 'rmod', 'rmul', 'ror', 'round', 'rpow', 'rrshift', 'rshift', 'rsub', 'rtruediv', 'rxor', 'setattr', 'sizeof', 'str', 'sub', 'subclasshook', 'truediv', 'trunc', 'xor', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

So, yes, there was an error, but what is unclear to me...

[JonB]

@Bouke said in Escaping colon in command line argument:

Trust me, I've tried with identical commands. Both none:0 and :0 work from te command line, but not from Python.

But this was not the only difference in the commands you showed, it was just one of them.

Please show an actual command line which you have verified works together with the exact Python code you are using to try execute the exact same command.

[Bouke]

my apologies, I've forgot to remove some stuff.

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov

this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

And yes, errorOccurred slot gets 'something' as described.

[JonB]

@Bouke
I am putting what you have written inside Code tags, so we can all read it exactly!

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov
this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

I admit I cannot see any difference!

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:

ffmpeg  -i none:0 /dev/null

and received error

none:0: Protocol not found
Did you mean file:none:0?

And then crickets....

Note that I see this message at the very end, after the same "ffmpeg compilation information" as you show. Are you sure your "crickets" does not have some message in it?

I do not see any examples anywhere using your none:0 construct. That's all I know.

[Bouke]

@JonB said in Escaping colon in command line argument:

@Bouke
I am putting what you have written inside Code tags, so we can all read it exactly!

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov
this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

I admit I cannot see any difference!

It's a mystery....

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I don't know the language in that thread, but I'm pretty sure your suggestion:
args << "-i" << filename << "-vn" << "-c:a" << "copy" << outFileName;
is correct. (Loose the spaces, keep each arg separate. Exception is in FFmpeg '-filter_complex', where the filter is a string containing filters with args.
In a terminal, that should be quoted, but as an arg in QT that is done automagically.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

I have tried a gazillion things, including single / double quotes, slashes, making it a list...
No response (yet) from the FFmpeg user list.

UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:

ffmpeg  -i none:0 /dev/null

and received error

none:0: Protocol not found
Did you mean file:none:0?

I don't know Linux, but this does not fail trough QT, this error is an FFmpeg reply, so the colon did arrive properly I would think.

> And then crickets....

Note that I see this message at the very end, *after* the same "ffmpeg compilation information" as you show.  Are you *sure* your "crickets" does not have some message in it?

You've missed the joke.
The sound of crickets is a 'universal' sound effect (especially in cartoons) that emphasizes an (awkward) silence.

I do not see any examples anywhere using your none:0 construct. That's all I know.

There are tons of examples on the net, and it's in the FFmpeg user manual.
(But this is AVfoundation syntax, so to toy you need a Mac.)

I'll keep toying / asking around, (and report back if I find something.)
Thanks for your help!

[jeremy_k]

@Bouke said in Escaping colon in command line argument:

@JonB said in Escaping colon in command line argument:

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I don't know the language in that thread, but I'm pretty sure your suggestion:
args << "-i" << filename << "-vn" << "-c:a" << "copy" << outFileName;
is correct. (Loose the spaces, keep each arg separate. Exception is in FFmpeg '-filter_complex', where the filter is a string containing filters with args.
In a terminal, that should be quoted, but as an arg in QT that is done automagically.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

I have tried a gazillion things, including single / double quotes, slashes, making it a list...
No response (yet) from the FFmpeg user list.

UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:

ffmpeg  -i none:0 /dev/null

and received error

none:0: Protocol not found
Did you mean file:none:0?

I don't know Linux, but this does not fail trough QT, this error is an FFmpeg reply, so the colon did arrive properly I would think.

I am also not of the opinion that the colon is the problem, but I'll make the same suggestion as in the other thread: If argument passing is suspected to be a problem, have QProcess start a program that logs its arguments.
There is a reencoding that occurs, and It's conceivable that something QString can handle won't survive the conversion.

https://code.woboq.org/qt5/qtbase/src/corelib/io/qprocess_unix.cpp.html#434

    for (int i = 0; i < arguments.count(); ++i)
        argv[i + 1] = ::strdup(QFile::encodeName(arguments.at(i)).constData());

And then crickets....

Note that I see this message at the very end, after the same "ffmpeg compilation information" as you show. Are you sure your "crickets" does not have some message in it?

You've missed the joke.
The sound of crickets is a 'universal' sound effect (especially in cartoons) that emphasizes an (awkward) silence.

Crickets are not in on the joke!

Maybe you have already noticed this, but the output from the slot connected to QProcess::errorOccurred indicates
that ffmpeg "crashed". Don't bother waiting up.

https://doc.qt.io/qt-5/qprocess.html#ProcessError-enum

QProcess::Crashed 1 The process crashed some time after starting successfully.

[JonB]

@jeremy_k said in Escaping colon in command line argument:

Maybe you have already noticed this, but the output from the slot connected to QProcess::errorOccurred indicates
that ffmpeg "crashed". Don't bother waiting up.

@Bouke Indeed as @jeremy_k notes, this indicates a crash. The output you do get back from ffmpeg seems to be printed every time by it? In which case, is it possible that it is getting through your command line arguments OK and actually "crashing" on trying to perform the desired operation? For as yet unknown reason. Can you get the ffmpeg to perform some other operation for you, with different arguments, from Qt, to prove it works OK?

Note also: I believe your MacOS has bash or similar shell. You can run your command via:

cmd = "bash"
args = ["-c", "ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov"]

If the ffmpeg indeed "crashes" you might get more information from a message from bash this way?

[Bouke]

@JonB said in Escaping colon in command line argument:

cmd = "bash"
args = ["-c", "ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov"]

Just tried it, it gives me 4 or 1 (Depending on God knows what) on the errorOccurred slot.
FFmpeg seems to crash the same way...

If the ffmpeg indeed "crashes" you might get more information from a message from bash this way?

How do I get info from bash?
And why is everyone working on a sunday?
(Not that I don't appreciate everyones input.)

Bouke

[JonB]

@Bouke

Just tried it, it gives me 4 or 1 (Depending on God knows what) on the errorOccurred slot.

I don't know quite what is going on here. I think I would expect bash not to "crash", even if it calls an ffmpeg which does.

How do I get info from bash?

With my expectation that bash itself would not "crash", I would expect it to capture any/all output, possibly add a message if what it called "crashed". Then you should be able to read all its output back in the normal way.

If it were me I would be attaching each of the signals you can see from QProcess correctly. You might want to paste that part of your code, just so I can verify exactly what you are doing about reading output, reporting errors etc. I would probably remove self.proc.setProcessChannelMode(self.proc.MergedChannels) at least temporarily and read/report each of stdout/stderr separately.

To diagnose the command line further, I can't type in everything I would try. But I would build up logically. Does a command of just ffmpeg crash? What about ffmpeg - rubbish? And so on through the various arguments you are really wanting to pass. Till you see some point where with something it "crashes" and without that something it works. A thorough process of elimination, till you can find just one thing which makes it malfunction.

And why is everyone working on a sunday?

We are helpful/sad/irreligious. I am interested in any shell/process errors, I go at it like a terrier because (at least under Linux, I don't know MacOS) I would expect to be able to diagnose any shell/process error!

[Bouke]

Ok, thanks to you I've figured out that indeed FFmpeg is crashing.
I've ran the commands with FFmpeg -loglevel trace -report, and the error log shows the audio input is opened, that's the moment it crashes.
So, it has nothing to do with the colon, it's something else.

Still, running the exact same command from the terminal works as expected.
Very weird...

@JonB said in Escaping colon in command line argument:

We are helpful/sad/irreligious.

Same here :-)

I am interested in any shell/process errors, I go at it like a terrier because (at least under Linux, I don't know MacOS) I would expect to be able to diagnose any shell/process error!

I'm that way if it's about Timecode...

[Bouke]

@Bouke

Stupid Bouke.
Turned out to be the 'Security & Privacy' settings that 'should' ask if it's OK to use the mic, but does not...
Curses! 2 days lost on a very stupid thing that I should have known, as I've been bitten by that before!

How can I mark this thread as 'solved'?

Thanks all, sorry for my ignorance!

Bouke