need guidance on proper launching of external process to Qt C++ app

MarkMcC

I am writing an application in C++ (Qt creator application ver 6.0.4), and I have embedded a call to an external AVRDUDE script. The script launches an app called AVRDUDE, which communicates with an AVR micro-controller over USB port. The script executes perfectly, however control is returned to the spawning C++ program immediately after the call, and NOT upon completion. here is the function call

void MainWindow::call_AVRDUDE_read() {
QProcess CommandPrompt;
QStringList Arguments;
Arguments <<"C/ avrdude -c arduino -P com3 -b 115200 -p ATmega328P -e -U eeprom:r:fromEEPROM.bin:r";
CommandPrompt.startDetached("cmd",Arguments);

The script runs fine and does exactly as it should, but reading the EEPROM of course takes a few seconds to execute the task. In the meantime, control has been returned to the c++ program which spawned the AVRDUDE script and is off trying to evaluate the contents of "fromEEPROM.bin" prior to its arrival. I know this to be true as I have placed a breakpoint in the code shortly after the call to the AVRDUDE read function. The breakpoint is hit immediately after the call to the above script.
So the crux of the problem is how to suspend execution of the calling code while the AVRDUDE script is in process. Then, when the EEPROM read is completed, return control to the host process so it can proceed with its process.
One last thing: I did try inserting "CommandPrompt.waitForFinished();" in the above function at the tail end, but it did nothing.
Is my problem in the use of "startDetached"? If so, what should I be using here? Any thoughts or recommendations?
Regards,
Mark

JonB

@MarkMcC
I am surprised this works as you claim. You start your argument with C/, I do not believe that works or is what you intend. Is this genuinely a copy-paste of the code you have?

You will indeed need to wait for the command to finish. One way is CommandPrompt.waitForFinished(). But that will not work with startDetached(). Use start().

You have two ways to wait for the command to complete:

Use waitForFinished(). This blocks until the QProcess exits. I'm not sure, but I think it blocks your UI too, so it will appear "frozen" while waiting.
Use the signals emitted from QProcess like finished(); write your code which must come after execution in a slot attached to that. There are also signals for errors, output being produced by the sub-process, and so on. This will definitely not block the UI.

MarkMcC

Hi Jon. Yes, this excerpt is indeed verbatim what my code is. Also, the script as shown actually DOES work, and as a stand-alone entity, there is no issue at all with it.
The arguments passed in CommandPrompt invoke an external "AVRDUDE" script, which reads the contents of the EEPROM of an ATmega328P processor (off an Arduino "nano" board) and write the contents to a file I have named "fromEEPROM.bin". This works just as it should.
My problem is that once this CommandPrompt is executed, it immediately returns control to my code, which in turn attempts to open this file "fromEEPROM.bin" and parse it, etc. This reading and processing I am attempting to do is prior to the completion of the task of reading/storing the EEPROM, so this file has not yet been updated. This is the core issue.
I have made a really kludgy "fix", but not only is it inelegant, but unreliable due to a possible race at the end. Changes as follows...

void MainWindow::call_AVRDUDE_read()
{
    remove("fromEEPROM.bin");    // trash the file before re-recreating here in tHe avrdude script
    QProcess CommandPrompt;
    QStringList Arguments;
    Arguments << "/C avrdude -c arduino -P com3 -b 115200 -p ATmega328P -e  -U eeprom:r:fromEEPROM.bin:r"; // " /C avrdude -c
    CommandPrompt.startDetached("cmd",Arguments);
}

and the code that calls it ..

oid MainWindow::on_READ_FROM_EEPROM_clicked() // Read EEPROM into fike "fromEEPROM.bin", then output values to UI
{
    call_AVRDUDE_read();
    int mBufferLength = 1024;
    char mBuffer[mBufferLength];
    float fB[mBufferLength/4];
    QString filename = "fromEEPROM.bin"; // QFileDialog::getOpenFileName(this,"Choose Fie");
    QFile mFile(filename);
    while (!mFile.exists());  // NOP. Wait here until the file is written so we can go on and process it
    if (mFile.exists())
    {
        if (mFile.open(QFile::ReadOnly))
        {
            while (!mFile.atEnd())
                mFile.read(mBuffer,sizeof(mBuffer));
        }
     mFile.close();

In the first code excerpt above (call_AVRDUDE_read) Ihave deleted the file "fromEEPROM.bin" prior to the AVRDUDE script.
In the "on_READ_FROM_EEPROM_clicked() " code segment, I have the process held up by the line

    while (!mFile.exists());  // NOP. Wait here until the file is written so we can go on and process it

Therefore, the code past that point will not execute until the file is present (i.e. has been successfully been written by the AVRDUDE script).
I realize this is a kludge, and not a working solution, but it does bring the problem to the foreground, and it does "fix" (term used loosely) the problem

Regarding the using the QProcess "start()" rather than "startDetached()", using just "start()", the AVRDUDE script will NOT execute. Also, if I omit the "/C' at the head of my arguments, the AVRDUDE script will NOT execute. Adding the line "CommandPrompt.waitForFinished()" seems to do nothing. I'll keep researching QProcess, but for now, I am still stymied. Reards..Maek

JonB

@MarkMcC said in need guidance on proper launching of external process to Qt C++ app:

Also, if I omit the "/C' at the head of my arguments, the AVRDUDE script will NOT execute

Arguments <<"C/ avrdude -c arduino -P com3 -b 115200 -p ATmega328P -e -U eeprom:r:fromEEPROM.bin:r";

Your argument string does not have /C at the beginning of it. It has C/. Which is why I am surprised it works and asked if it was verbatim.

MarkMcC

I Believe I found the resolution to my problem!
The CommandPrompt.startDetached(), or the CommandPromt.start(): that I later tried (which didn't work at all) I haver replaced with CommandPrompt.execute(); This not only executes the AVRDUDE script successfully, but also responds to the CommandPrompt.waitForFinished(); and behaves as I would expect it to. The last lines of the call are now..

        CommandPrompt.execute("cmd",Arguments)
        CommandPrompt.waitForFinished(5000);

This works great. Control is held until the script completes, and I don't have to do any of that hokey stuff I described in my last post, deleting files and waiting for then to again become re-written.. Thanks JonB for helping on this and giving me some things to think about! The Qt documentation on the QProcess class was (and is) over my head, but I don't see any problems using .execute(), and it works exactly as I would like it too. Mark

Anonymous_Banned275

@MarkMcC I got stuck with similar issue - it seems that "start" should not be an option or the doc is really misleading one to believe that "start" and "execute " function same way .
I did not read much further , but does "execute" emits any "progress' SIGNAL(s) ? (started , running ...)

It obviously emits "finished" ...,

MarkMcC

I really don't know. This stuff is all pretty transparent to me. What I can say is control is not returned to the calling process until the EEPROM is read and the file is generated by the AVRDUDE process.
Control is returned to the caller only after CommandPrompt.waitForFinished() is satisfied. What the mechanism is under the hood, I do not know. Any further details are way over my head ;)

Kent-Dorfman

As hinted above, you don't want to do anything with the avrdude output until the QProcess::finished() has been emited.

J.Hilk

@MarkMcC said in need guidance on proper launching of external process to Qt C++ app:

Regarding the using the QProcess "start()" rather than "startDetached()", using just "start()", the AVRDUDE script will NOT execute.

yes of course, because it's a lifetime issue, your QProcess instance is stack allocated and therefore limited to the scope of call_AVRDUDE_read it will get detroyed at the end of the function and that will happen before your process is finished running.

JonB

@MarkMcC said in need guidance on proper launching of external process to Qt C++ app:

    CommandPrompt.execute("cmd",Arguments)
    CommandPrompt.waitForFinished(5000);

Please be aware, or for anyone else who comes to read this thread, there is muddle/misinformation here.

int QProcess::execute(const QString &program, const QStringList &arguments) is a static method. That means it does not use your CommandPrompt instance, it could just as well be written CommandPrompt::execute("cmd",Arguments) or QProcess::execute("cmd",Arguments).

Given that it is static, I have no idea what your CommandPrompt.waitForFinished(5000); does on the following line, if anything, since your CommandPrompt instance has not started any process.

QProcess::execute() code can be seen at https://code.woboq.org/qt5/qtbase/src/corelib/io/qprocess.cpp.html#_ZN8QProcess7executeERK7QStringRK11QStringList. It is not magic, all it does is:

int QProcess::execute(const QString &program, const QStringList &arguments)
{
    QProcess process;
    process.setProcessChannelMode(ForwardedChannels);
    process.start(program, arguments);
    if (!process.waitForFinished(-1) || process.error() == FailedToStart)
        return -2;
    return process.exitStatus() == QProcess::NormalExit ? process.exitCode() : -1;
}

So you can see that is all you had to do in code if you wanted to achieve the same. You can also see that it uses its own private QProcess instance, not yours. And it calls waitForFinished(). So you could have done this.

To wait for a QProcess to finish you can either use waitForFinished() or you can slot onto the finished() signal, which is what waitForFinished() does internally. Only a QProcess::start() can be waited on, not a startDetached().

I do not know what you did originally which stopped whatever code you had from working. The only code you showed (a) used startDetached() instead of start() and (b) did not attempt to do any waiting.

Finally you have said you have "copied verbatim" from your source code and it reads

Arguments <<"C/ avrdude -c arduino -P com3 -b 115200 -p ATmega328P -e -U eeprom:r:fromEEPROM.bin:r";

You have also stated

Also, if I omit the "/C' at the head of my arguments, the AVRDUDE script will NOT execute.

I have challenged you on this earlier. I do not believe your "verbatim" copy can start with the C/ you claim it has, rather it must be /C. Do you have a comment on this?

JonB

@AnneRanch said in need guidance on proper launching of external process to Qt C++ app:

it seems that "start" should not be an option or the doc is really misleading one to believe that "start" and "execute " function same way

Of course QProcess::start() must be "an option", it is an essential method, it is the one which actually sets off the subprocess running. You can now see from the code above that QProcess::execute() is a static method with its own QProcess instance which does a start() followed by waitForFinished().

the doc is really misleading one to believe that "start" and "execute " function same way

The docs for start() state:

Starts the given program in a new process, passing the command line arguments in arguments.

and for execute() state

Starts the program program with the arguments arguments in a new process, waits for it to finish, and then returns the exit code of the process.

[My bold] Which is precisely what they each do. They do not do "the same" as each other. There is nothing "really misleading" here, execute() does more than just start() just as the docs explain. I cannot see how it could have been documented any clearer.

It obviously emits "finished" ...,

The internal QProcess process; used in execute() does itself emit finished() and all other signals. However, since it is an internal variable in the static function you cannot access any of those signals, be notified or put slots on them.

MarkMcC

Thanks for the help Jon. Sorry, but yes..this was a bit of a muddle. Using " QProcess::execute()" alone solved my issue, and I removed the unnecessary "waitForFinished" line.

Looking at the code you gave for the QProcess:: execute() above, I noticed that the process.start() line was preceded by another rather important line..

  process.setProcessChannelMode(ForwardedChannels);
  process.start(program, arguments);

When I initially tried using just QProcess::start() in my code, it failed to execute my AVRDUDE script. Now, if I precede start() with the setProcessChannelMode(ForwardChannels) as above, my AVRDUDE process launches. That appeared to be the missing link there with using start().

In any case, QProcess::execute has it all in a nice neat bundle including the wait for completion, so I'm good. I'm just going with that Thanks again.

JonB

@MarkMcC said in need guidance on proper launching of external process to Qt C++ app:

When I initially tried using just QProcess::start() in my code, it failed to execute my AVRDUDE script. Now, if I precede start() with the setProcessChannelMode(ForwardChannels) as above, my AVRDUDE process launches.

That is interesting. I only thought setProcessChannelMode() should affect how your Qt host program handles any output the child produces. I would not have expected that to affect the ability to launch/run the child.

Anyway, yes, execute() is a "convenience" static method only introduced a few releases ago.