read data continuously from QProcess

jgxy1123

I want to read the continuous data from QProcess.

The problem is that I can not read continuously the compressing progress of 7z.exe, but can only get the final result after the process is finished. It seems that the signal of "readyReadStandardOutput" will be emitted only once when QProcess is finished.

My code logic is below

make an instance of QProcess : QProcess proc
set readChannel : proc.setReadChannel(QProcess::StandardOutput)
connect signal and slots:
. connect( &proc, &QProcess::readyReadStandardOutput,[&](){
qDebug()<<QString::fromLocal8bit(proc.readAllStandardOutput());
}
.connect(&proc,&QProcess::finished,[&](){
qDebug()<<"Process finished";
}
set program start the Process:
.proc.setProgram("cmd.exe")
.proc.startCommand("7z.exe a myfolder.zip myfolder")
.proc.waitForFinished(-1)
.proc.close();

JonB

@jgxy1123
Is the progress even written on stdout? I would have thought stderr?

Add readyReadStandardError() signal/slot.

In case they interfere, get rid of both setReadChannel() and waitForFinished() calls. So you can't close(). proc must remain in scope, from where you are now either make it a member variable or new it.

7z.exe may detect output is not to terminal and not send progress messages. Try running it from command line with output redirected to file or pipe and see.

jeremy_k

@jgxy1123 said in read data continuously from QProcess:

My code logic is below

Rather than present the logic as a set of bullet points, just show us the code.

set program start the Process:
.proc.setProgram("cmd.exe")
.proc.startCommand("7z.exe a myfolder.zip myfolder")

There's no need to call QProcess::setProgram if QProcess::startCommand is used.

jgxy1123

@jeremy_k

Thank you!
The zip file can be made successully, but the realtime progress of 7z.exe still can not be retrived from the output of the QProcess, which means I can not read the current progress percentage marked by red in the below picture.

jsulm

@jgxy1123 Maybe 7zip prints the progress on stderr? You should also subscribe to it and see what you get.

SamiV123

It might be that 7zip doesn't flush but uses carriage return (or even Win32 console API directly) to print the progress and without flushing (new line) the signal is not emitted.

That being said you don't need the signal to read, you can always just try to read from the standard output (or the standard err) for example on a timer, i.e. polling.

You might also check if there's some line buffering type of setting in QProcess.

Also be aware of this caveat

"Note: Windows intentionally suppresses output from GUI-only applications to inherited consoles. This does not apply to output redirected to files or pipes. To forward the output of GUI-only applications on the console nonetheless, you must use SeparateChannels and do the forwarding yourself by reading the output and writing it to the appropriate output channels."

JonB

@jgxy1123
Having written earlier what you need to try to see whether you can get output from 7zip successfully.

Just a heads up: rather than running an external command (you/end-users need 7z.exe installed, and code only works under Windows) you might consider doing the "zipping" directly in your Qt code. zlib is a library written in C that many people use. See also e.g. https://forum.qt.io/topic/74306/how-to-manage-zip-file.

Though your example passes a directory/folder to zip, this can absolutely be done but does require a lot more work than just zipping one file/stream. See e.g. https://github.com/sebastiandev/zipper.

May not apply to you (you may want simplicity of just running 7z.exe command) but here for others reading this.

mranger90

Here's how I parse the output of 7zip and send it to a progress bar object:

   // 7z a -bso0 -bsp1 <destination> <f1> <f2> <f3>
    auto cmd = m_7ZipCommand + "a -bso0 -bsp1 " + destFile + " " + Folder1+ " " +
               Folder2 + " " +  Folder3;

...
  // connect the QProcess finished signal
    connect(&m_zipProcess, QOverload<int, QProcess::ExitStatus>::of(&QProcess::finished), this,
            &ProjectionExporter::onExportFinished);
...
   // Parse the output reported by 7Zip to report the progress percentage
    connect(&m_zipProcess, &QProcess::readyRead, this, [this]() {
        QString output = m_zipProcess.readAllStandardOutput();
        QRegularExpression percentageMessage("^(\\s*)(\\d+)(%.*)$");
        QRegularExpressionMatch match = percentageMessage.match(output);
        if (match.hasMatch())
        {
            const int percentageExported = match.captured(2).toInt();
            const float percentAsFloat = static_cast<float>(percentageExported) / 100.0f;
            m_exportProgress.complatedFiles =
                static_cast<int>(static_cast<float>(m_filesToBeExported) * percentAsFloat);
            m_exportProgress.progressInPercentage = percentageExported;
            emit exportProgress(m_exportProgress);
        }
    });

jsulm

@mranger90 said in read data continuously from QProcess:

connect(&m_zipProcess, &QProcess::readyRead, this, this {

So, is the slot called?

JonB

@mranger90
Looks much like you said originally. Does it get the output to parse? If not, have you acted on the suggestions to your issue?

It would be "odd" to connect slot &QProcess::readyRead with code which only calls readAllStandardOutput().

mranger90

@jsulm
Yes, this slot gets called regularly. It's how I update a progress bar for the operator.

JonB

@mranger90
So are you saying it is resolved and you get what you want, or do you still have any issue?

mranger90

@JonB
I'm not the one with the issue. I posted some working code to help out.

jsulm

@mranger90 If it is called did you check what "output" is? Maybe your regexp is wrong...

JonB

@mranger90 LOL, I'm sorry :)

jsulm

@mranger90 said in read data continuously from QProcess:

I'm not the one with the issue

Oh, I also did not realise :-D