Escaping colon in command line argument

Bouke

@JonB said in Escaping colon in command line argument:

@Bouke
Like I said, start by testing (and showing) identical commands for QProcess::start() and what works from the command line. Not approximately the same but the same.

Trust me, I've tried with identical commands. Both none:0 and :0 work from te command line, but not from Python.
(I did hundreds of tests...)

Make your code report any QProcess.errorOccurred signal

How do I do that?

By attaching a slot to to it.

Ok,
I've added:
self.proc.errorOccurred.connect(self.erroroccured)

def erroroccured(self, _err):
    print("errorOccured", _err)
    print(type(_err))
    print(dir(_err))

Slot prints:
errorOccured 1
<class 'PyQt5.QtCore.QProcess.ProcessError'>
['abs', 'add', 'and', 'bool', 'ceil', 'class', 'delattr', 'dict', 'dir', 'divmod', 'doc', 'eq', 'float', 'floor', 'floordiv', 'format', 'ge', 'getattribute', 'getnewargs', 'gt', 'hash', 'index', 'init', 'init_subclass', 'int', 'invert', 'le', 'lshift', 'lt', 'mod', 'module', 'mul', 'ne', 'neg', 'new', 'or', 'pos', 'pow', 'radd', 'rand', 'rdivmod', 'reduce', 'reduce_ex', 'repr', 'rfloordiv', 'rlshift', 'rmod', 'rmul', 'ror', 'round', 'rpow', 'rrshift', 'rshift', 'rsub', 'rtruediv', 'rxor', 'setattr', 'sizeof', 'str', 'sub', 'subclasshook', 'truediv', 'trunc', 'xor', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

So, yes, there was an error, but what is unclear to me...

JonB

@Bouke said in Escaping colon in command line argument:

Trust me, I've tried with identical commands. Both none:0 and :0 work from te command line, but not from Python.

But this was not the only difference in the commands you showed, it was just one of them.

Please show an actual command line which you have verified works together with the exact Python code you are using to try execute the exact same command.

Bouke

my apologies, I've forgot to remove some stuff.

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov

this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

And yes, errorOccurred slot gets 'something' as described.

JonB

@Bouke
I am putting what you have written inside Code tags, so we can all read it exactly!

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov
this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

I admit I cannot see any difference!

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:

ffmpeg  -i none:0 /dev/null

and received error

none:0: Protocol not found
Did you mean file:none:0?

And then crickets....

Note that I see this message at the very end, after the same "ffmpeg compilation information" as you show. Are you sure your "crickets" does not have some message in it?

I do not see any examples anywhere using your none:0 construct. That's all I know.

Bouke

@JonB said in Escaping colon in command line argument:

@Bouke
I am putting what you have written inside Code tags, so we can all read it exactly!

this works from the terminal:
ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov
this fails from Python
cmd = "ffmpeg"
args = ['-f', 'avfoundation', '-i', 'none:0', '/Volumes/Data/16-05-2021_07-06-18_1.mov']

I admit I cannot see any difference!

It's a mystery....

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I don't know the language in that thread, but I'm pretty sure your suggestion:
args << "-i" << filename << "-vn" << "-c:a" << "copy" << outFileName;
is correct. (Loose the spaces, keep each arg separate. Exception is in FFmpeg '-filter_complex', where the filter is a string containing filters with args.
In a terminal, that should be quoted, but as an arg in QT that is done automagically.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

I have tried a gazillion things, including single / double quotes, slashes, making it a list...
No response (yet) from the FFmpeg user list.

UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:
ffmpeg  -i none:0 /dev/null
and received error
none:0: Protocol not found
Did you mean file:none:0?

I don't know Linux, but this does not fail trough QT, this error is an FFmpeg reply, so the colon did arrive properly I would think.

> And then crickets....

Note that I see this message at the very end, *after* the same "ffmpeg compilation information" as you show.  Are you *sure* your "crickets" does not have some message in it?

You've missed the joke.
The sound of crickets is a 'universal' sound effect (especially in cartoons) that emphasizes an (awkward) silence.

I do not see any examples anywhere using your none:0 construct. That's all I know.

There are tons of examples on the net, and it's in the FFmpeg user manual.
(But this is AVfoundation syntax, so to toy you need a Mac.)

I'll keep toying / asking around, (and report back if I find something.)
Thanks for your help!

jeremy_k

@Bouke said in Escaping colon in command line argument:

@JonB said in Escaping colon in command line argument:

I have just been answering a similar-ish question in https://forum.qt.io/topic/126693/qprocess-start-filename-args/6. Waiting for that OP to report back if what I suggested does work for him.

I don't know the language in that thread, but I'm pretty sure your suggestion:
args << "-i" << filename << "-vn" << "-c:a" << "copy" << outFileName;
is correct. (Loose the spaces, keep each arg separate. Exception is in FFmpeg '-filter_complex', where the filter is a string containing filters with args.
In a terminal, that should be quoted, but as an arg in QT that is done automagically.

I can only suggest for you that you play with your command-line till you find something which does work, something which doesn't work, and compare them. You need to discover a minimal change which moves it from working to non-working from Python/Qt.

I have tried a gazillion things, including single / double quotes, slashes, making it a list...
No response (yet) from the FFmpeg user list.
UPDATE
I know nothing about ffmpeg. But I installed it under Ubuntu. I tried:
ffmpeg  -i none:0 /dev/null
and received error
none:0: Protocol not found
Did you mean file:none:0?
I don't know Linux, but this does not fail trough QT, this error is an FFmpeg reply, so the colon did arrive properly I would think.

I am also not of the opinion that the colon is the problem, but I'll make the same suggestion as in the other thread: If argument passing is suspected to be a problem, have QProcess start a program that logs its arguments.
There is a reencoding that occurs, and It's conceivable that something QString can handle won't survive the conversion.

https://code.woboq.org/qt5/qtbase/src/corelib/io/qprocess_unix.cpp.html#434

    for (int i = 0; i < arguments.count(); ++i)
        argv[i + 1] = ::strdup(QFile::encodeName(arguments.at(i)).constData());

And then crickets....

Note that I see this message at the very end, after the same "ffmpeg compilation information" as you show. Are you sure your "crickets" does not have some message in it?

You've missed the joke.
The sound of crickets is a 'universal' sound effect (especially in cartoons) that emphasizes an (awkward) silence.

Crickets are not in on the joke!

Maybe you have already noticed this, but the output from the slot connected to QProcess::errorOccurred indicates
that ffmpeg "crashed". Don't bother waiting up.

https://doc.qt.io/qt-5/qprocess.html#ProcessError-enum

QProcess::Crashed 1 The process crashed some time after starting successfully.

JonB

@jeremy_k said in Escaping colon in command line argument:

Maybe you have already noticed this, but the output from the slot connected to QProcess::errorOccurred indicates
that ffmpeg "crashed". Don't bother waiting up.

@Bouke Indeed as @jeremy_k notes, this indicates a crash. The output you do get back from ffmpeg seems to be printed every time by it? In which case, is it possible that it is getting through your command line arguments OK and actually "crashing" on trying to perform the desired operation? For as yet unknown reason. Can you get the ffmpeg to perform some other operation for you, with different arguments, from Qt, to prove it works OK?

Note also: I believe your MacOS has bash or similar shell. You can run your command via:

cmd = "bash"
args = ["-c", "ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov"]

If the ffmpeg indeed "crashes" you might get more information from a message from bash this way?

Bouke

@JonB said in Escaping colon in command line argument:

cmd = "bash"
args = ["-c", "ffmpeg -f avfoundation -i none:0 /Volumes/Data/16-05-2021_07-06-18_1.mov"]

Just tried it, it gives me 4 or 1 (Depending on God knows what) on the errorOccurred slot.
FFmpeg seems to crash the same way...

If the ffmpeg indeed "crashes" you might get more information from a message from bash this way?

How do I get info from bash?
And why is everyone working on a sunday?
(Not that I don't appreciate everyones input.)

Bouke

JonB

@Bouke

Just tried it, it gives me 4 or 1 (Depending on God knows what) on the errorOccurred slot.

I don't know quite what is going on here. I think I would expect bash not to "crash", even if it calls an ffmpeg which does.

How do I get info from bash?

With my expectation that bash itself would not "crash", I would expect it to capture any/all output, possibly add a message if what it called "crashed". Then you should be able to read all its output back in the normal way.

If it were me I would be attaching each of the signals you can see from QProcess correctly. You might want to paste that part of your code, just so I can verify exactly what you are doing about reading output, reporting errors etc. I would probably remove self.proc.setProcessChannelMode(self.proc.MergedChannels) at least temporarily and read/report each of stdout/stderr separately.

To diagnose the command line further, I can't type in everything I would try. But I would build up logically. Does a command of just ffmpeg crash? What about ffmpeg - rubbish? And so on through the various arguments you are really wanting to pass. Till you see some point where with something it "crashes" and without that something it works. A thorough process of elimination, till you can find just one thing which makes it malfunction.

And why is everyone working on a sunday?

We are helpful/sad/irreligious. I am interested in any shell/process errors, I go at it like a terrier because (at least under Linux, I don't know MacOS) I would expect to be able to diagnose any shell/process error!

Bouke

Ok, thanks to you I've figured out that indeed FFmpeg is crashing.
I've ran the commands with FFmpeg -loglevel trace -report, and the error log shows the audio input is opened, that's the moment it crashes.
So, it has nothing to do with the colon, it's something else.

Still, running the exact same command from the terminal works as expected.
Very weird...

@JonB said in Escaping colon in command line argument:

We are helpful/sad/irreligious.

Same here :-)

I am interested in any shell/process errors, I go at it like a terrier because (at least under Linux, I don't know MacOS) I would expect to be able to diagnose any shell/process error!

I'm that way if it's about Timecode...

Bouke

@Bouke

Stupid Bouke.
Turned out to be the 'Security & Privacy' settings that 'should' ask if it's OK to use the mic, but does not...
Curses! 2 days lost on a very stupid thing that I should have known, as I've been bitten by that before!

How can I mark this thread as 'solved'?

Thanks all, sorry for my ignorance!

Bouke