QProcess and stdout

Devolution

Hello, I'm trying to call a binary (namely youtube-dl) using QProcess.

Relevant code is the following:

    QProcess * ytdl = new QProcess();


    QObject::connect(ytdl,&QProcess::started, [=]{
        qDebug() << "STARTED";
    });

    QObject::connect(ytdl, &QProcess::readyReadStandardOutput, [=]{
         qDebug() << ytdl->readAllStandardOutput();
    });
    QObject::connect(ytdl, &QProcess::readyReadStandardError, [=]{
          qDebug() << ytdl->readAllStandardError();
    });
    QObject::connect(ytdl,QOverload<int>::of(&QProcess::finished), [=]{
        qDebug() << "Youtubedl ended nam";
        ytdl->deleteLater();
    });

            ytdl->setWorkingDirectory(QApplication::applicationDirPath());
            ytdl->setProcessChannelMode( QProcess::MergedChannels ); // merge stderr and stdout

            ytdl->setReadChannel(QProcess::StandardOutput);
            ytdl->start("cmd" , QStringList() <<"/C" << "D:\\youtube-dl.exe") ;

If i run the code as is, i only get the STARTED output in the qDebug terminal.
However, if i add waitForFinished() i get all the output text at once.
The start command i used is very small and should return the usage of youtube-dl, it's basically instant, and even after waiting a few minutes nothing is printed in the qDebug window without waitForFinished().

The thing is i want this to be asynchronous, and not blocking.

Could you provide help on this issue? Thanks in advance for any hints :)

eyllanesc

@Devolution try changing to ytdl->start("D:\\youtube-dl.exe") ;

SGaist

Hi and welcome to devnet,

You forgot to connect the errorOccured signal in case something went wrong.

Devolution

@eyllanesc said in QProcess and stdout:

@Devolution try changing to ytdl->start("D:\\youtube-dl.exe") ;

Hi,
I tried with that, the result is the same.

@SGaist said in QProcess and stdout:

Hi and welcome to devnet,

You forgot to connect the errorOccured signal in case something went wrong.

Hello, i've now connected the signal, with the following:

QObject::connect(ytdl,&QProcess::errorOccurred, [=](QProcess::ProcessError error){
     qDebug() << "Error:" << error;
 });

No error is shown and still no output from stderr or stdout.

JonB

@Devolution
Well there out be some difference, because your original cmd D:\\youtube-dl.exe would not have run it, and should have just produced a Command Prompt message or hung. At least D:\\youtube-dl.exe should run it.

When you run it (D:\youtube-dl.exe) yourself (outside Qt), does it produce output and exit, or does it stay around?

Devolution

My bad,
i forgot the /C in my original post, i edited it. Regardless of cmd /C D:\youtube.dl.exe or D:\youtube-dl.exe there is no output.

D:\youtube-dl.exe ecuted outside qt (in the execcute window) produces usage output and then shuts off.

JonB

@Devolution
Just to make sure and kinda simulate what is going on with the redirection, please go into a Command Prompt and just verify how D:\youtube-dl.exe | more behaves? Also, just in case, D:\youtube-dl.exe <nul

Then, comment out all your redirection code. First things first, we are wanting to just get the finished signal, which you are saying you aren't getting.

Also check when it "hangs" that you can see the youtube-dl.exe process running in Task Manager.

These are the things I can think of that I would verify.

Devolution

@JonB said in QProcess and stdout:

@Devolution
Just to make sure and kinda simulate what is going on with the redirection, please go into a Command Prompt and just verify how D:\youtube-dl.exe | more behaves? Also, just in case, D:\youtube-dl.exe <nul

Then, comment out all your redirection code. First things first, we are wanting to just get the finished signal, which you are saying you aren't getting.

Also check when it "hangs" that you can see the youtube-dl.exe process running in Task Manager.

These are the things I can think of that I would verify

D:`youtube-dl.exe | more executed in a cmd behaves the same as without | more, that is printing the usage message.

I commented the stdout and stderr redirections and the lines relative to the channels, still i got no signal emitted for process done. I can't see any process hanging in the task manager, probably youtube-dl going to fast to shutdown..

JonB

@Devolution

probably youtube-dl going to fast to shutdown

It's not supposed to work that way. So long as you have your slots set up before you start(), which you seem to have done.

Hmm. I don't get that you are not seeing finished, yet, say, waitForFinished() is concluding.

I haven't used it myself but there is also signal QProcess::stateChanged, https://doc.qt.io/qt-5/qprocess.html#stateChanged. See what states that reports it going through?

There is no chance that you are deleting that QProcess elsewhere than in finished, is there?

I'm not a C++ expert, certainly not on lambdas. Example at https://doc.qt.io/qt-5/qprocess.html#finished shows:

connect(process, QOverload<int, QProcess::ExitStatus>::of(&QProcess::finished),
    [=](int exitCode, QProcess::ExitStatus exitStatus){ /* ... */ });

The QOverload<int, QProcess::ExitStatus> does not seem to match your QOverload<int>? Try with theirs?

Finally, check with some quite different .exe elsewhere (preferably producing some output), is the behaviour different from with youtube-dl.exe?

Devolution

@JonB

The overload is simply a deprecated one, i can surely try with theirs, nothing happened, still no finished signal.

I added a piece of code for the stateChanged signal, here's the output:

my state is now: QProcess::Starting
my state is now: QProcess::Running
STARTED

As you suggested i tried with other stuff. Notepad opens notepad as expected.
However, gcc, which behave the same without arguments (that is priting a message about no input file) does the exact same thing: goes through starting and running phase, but i'm not able to capture the output nor does it send the finished signal.

If i add waitForFinished with the gcc process call, it does put the process in the NotRunning state and fires the event, but still, no output is shown.

JonB

@Devolution
Dunno! :( I have used all this stuff and it works OK. Your code looks OK to me. So long as you guarantee your QProcess * ytdl = new QProcess() is not allowing delete ytdl from anywhere after you have called the ytdl->start() :) And, your code does return to the main Qt event loop after the start(), doesn't it?

If it were me, this would be bugging me now! Replace your redirection code & readyRead handlers with setStandardOutput/ErrorFile() calls. Send them to separate files/channels, for simplicity. See whether you ever get the output in those files or not....

Devolution

@JonB

Well to be sure i wasn't killing the process anywhere i basically moved its creation in the main just to test the outcome of this 'bug'.

This is my 'main.cpp' code



int main(int argc, char *argv[])
{
      QApplication app(argc, argv);


    SoundboardMainUI container;

    QProcess * ytdl = new QProcess();

    QObject::connect(ytdl,&QProcess::started, [=]{
        qDebug() << "STARTED";
    });
    QObject::connect(ytdl, QOverload<int, QProcess::ExitStatus>::of(&QProcess::finished), [=]{
        qDebug() << "Youtubedl ended nam";
        ytdl->deleteLater();
    });

    QObject::connect(ytdl,&QProcess::errorOccurred, [=](QProcess::ProcessError error){
        qDebug() << "Error:" << error;
    });

    QObject::connect(ytdl, &QProcess::stateChanged, [=](QProcess::ProcessState newState){
        qDebug() << "my state is now:" << newState;
    });

          
    ytdl->setWorkingDirectory(QApplication::applicationDirPath());

    ytdl->start("gcc");



    QFile css_dark(":/css/resources/darkorange.css");
    css_dark.open(QFile::ReadOnly);
    app.setStyleSheet(css_dark.readAll());
    css_dark.close();

      MSG msg;
      QPixmap pixmap(":/icon/resources/forsenAim.png");
 
      app.processEvents();
 
      while(GetMessage(&msg,nullptr,0,0))
      {
          TranslateMessage(&msg);
          DispatchMessage(&msg);
          if (msg.message == WM_HOTKEY)
              container.winHotKeyPressed(static_cast<int>(msg.wParam));

      }

    // Re-send post quit message or the app runs as a daemon for some reason
    PostQuitMessage(0);

    return app.exec();
}

Trying with output and error files.
gcc seems to be correct:

//stderr.txt
gcc: fatal error: no input files
compilation terminated.

stdout.txt is empty

With youtube-dl.exe:

//stderr.txt
Usage: youtube-dl.exe [OPTIONS] URL [URL...]

youtube-dl.exe: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.

stdout is empty.

Once again the output with file is correct...

What could be going on ? :/

JonB

@Devolution said in QProcess and stdout:

What could be going on ? :/

What's all this

app.processEvents();

(not so bad) but then horror:

while(GetMessage(&msg,nullptr,0,0))

?! :(

Get rid of all that Windows-y stuff, test your code as it should be just returning to the main event loop. That must surely be interfering with Qt messages?

(E.g. the "process finished" Windows event is getting eaten in the GetMessage() loop? You don't ever exit the GetMessage() loop till quit anyway, so you don't reach app.exec(); till after quit! I don't understand how you're expecting the code to behave, or why you're doing that stuff, and I don't want to know!)

Your code will not be what waitForFinished() does, which you have said works. You did not say anything about this, isn't it likely to be relevant? You're also doing your start() before going into app.exec(), do it from within. When you have things working that way correctly, then is the time to re-introduce and play with the code as it is now....

Devolution

@JonB

It didn't interfere with any of my signal and slots so i don't think it's the troublemaker here. I tried it tho, and you were right. This windows shenanigans IS actually messing with the stderr and stdout signals.

I don' t know if that's expected behavior or not (from windows/qt).

The windows-y event loop from windows is there for a reason,as i need it to capture system events: i need capture key stroke, but from everywhere, and not only when the qt application is in focus, which is the behavior QShortcut has, and i didn't find a way native to qt to register "global" shortcuts.

Seems i hit a wall there.

Could i workaround this using another QEventLoop used just for this process?

JonB

@Devolution
You have:

while (GetMessage())
   ....
...
return app.exec();

That is a blocking loop, till the user quits. Only then do you allow the Qt main event loop to run (at which point it presumably exits, but that's not relevant). It's not surprising there are problems.

I don't know what you want for what you want to achieve, but clearly this interferes with your process reading/finishing. A very knowledgeable Qt expert better than me suggests you look at https://doc.qt.io/qt-5/qabstractnativeeventfilter.html ... :)

Devolution

@JonB

Oh thanks, i never found this event filter thing. I'll try to play around with it and replace the sketchy windows loop. If i can render this more platform-agnostic i'd be happy.

Thanks to you and your friend :) I'll mark this as resolved