qprocess docker problem
-
Hi,
I have an amd64 machine and I want to cross-compile a Qt program for arm32 with docker(using qemu-arm-static). I tried many images like arm32v7 and others but at last I use official ubuntu docker image and then use apt-get to install necessary Qt libraries. everything looks fine but when I try to start a process using QProcess, it alwayse crashes (ProcessError::Crashed).
I am pretty sure that I use QProcess correctly, becuse if I copy the compiled binary on arm machine, it just works fine and problem only happens in container in amd64 machine.
Any clues how can i fix or debug that? apparently strace and gdb do not work on qemu -
Hi,
I have an amd64 machine and I want to cross-compile a Qt program for arm32 with docker(using qemu-arm-static). I tried many images like arm32v7 and others but at last I use official ubuntu docker image and then use apt-get to install necessary Qt libraries. everything looks fine but when I try to start a process using QProcess, it alwayse crashes (ProcessError::Crashed).
I am pretty sure that I use QProcess correctly, becuse if I copy the compiled binary on arm machine, it just works fine and problem only happens in container in amd64 machine.
Any clues how can i fix or debug that? apparently strace and gdb do not work on qemu@noahlopezdev
I have never used Docker. My colleague tells me how great it is, all I read up is people stuck when it goes wrong :)This is such a low-level and unusual error that --- if you don't get a better suggestion --- I would look for
ProcessError::Crashed
in Qt/Linux source code (e.g. woboq if you need to) and see what can actually raise that. May not tell you what the cause is, but at least you'll know where it's happening from.QProcess
is only a wrapper to the Linux system calls. -
Hi and welcome to devnet,
What exactly are you running with QProcess ?
-
@SGaist
Thank you fpr your response.
I tried many commands and they all crashed. my original use case wasldd <PROGRAM_PATH>
but something likeecho
andls
also crashes.
P.S. ldd works fine in bash (in emulated container) -
@noahlopezdev
I have never used Docker. My colleague tells me how great it is, all I read up is people stuck when it goes wrong :)This is such a low-level and unusual error that --- if you don't get a better suggestion --- I would look for
ProcessError::Crashed
in Qt/Linux source code (e.g. woboq if you need to) and see what can actually raise that. May not tell you what the cause is, but at least you'll know where it's happening from.QProcess
is only a wrapper to the Linux system calls.@JonB
Thank you for your response, well I inspected the code and if I'm not mistaking, Qt usesfork()
in Linux to create processes. I usedfork()
in a simple program and it works :( -
Can you provide a minimal project that shows the behaviour ?
-
@JonB
Thank you for your response, well I inspected the code and if I'm not mistaking, Qt usesfork()
in Linux to create processes. I usedfork()
in a simple program and it works :(@noahlopezdev
Yes it will usefork()
. That will not "crash" your parent. I suggested you look to see what path Qt code would have to follow to encounterProcessError::Crashed
in the sources, so that you could reason back from there. -
@noahlopezdev
You will probably get the same result (unfortunately), but what about trying for your command:ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");
just in case the
/usr/bin/sh
actually runs (though I'm guessing it won't, it will just cause the same error itself) and produces some output which might tell us what is going on?You might also try attaching a slot to
QProcess::stateChanged(QProcess::ProcessState newState)
which prints outnewState
so that you can retrieve it in yourreadAllStandard...()
. We might know which steps it got through before it "crashed". There is also signalQProcess::errorOccurred
.I used fork() in a simple program and it works :(
In a program compiled with Qt libraries, or a non-Qt standalone program? I wouldn't mind seeing the source you used which you say works? Don't forget it would not surprise me that in whatever goes wrong it is not on the
fork()
statement --- which really ought work (unless you have no memory left!) --- but more likely on the subsequentexec...()
call in the child, or maybe in thewait...()
call in the parent?If you suspect it is something about Qt or
QProcess
which is problematic you could always try the equivalent of the code you show but just usingfork
/exec
/wait()
so nothing Qt-y in it. -
@noahlopezdev
You will probably get the same result (unfortunately), but what about trying for your command:ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");
just in case the
/usr/bin/sh
actually runs (though I'm guessing it won't, it will just cause the same error itself) and produces some output which might tell us what is going on?You might also try attaching a slot to
QProcess::stateChanged(QProcess::ProcessState newState)
which prints outnewState
so that you can retrieve it in yourreadAllStandard...()
. We might know which steps it got through before it "crashed". There is also signalQProcess::errorOccurred
.I used fork() in a simple program and it works :(
In a program compiled with Qt libraries, or a non-Qt standalone program? I wouldn't mind seeing the source you used which you say works? Don't forget it would not surprise me that in whatever goes wrong it is not on the
fork()
statement --- which really ought work (unless you have no memory left!) --- but more likely on the subsequentexec...()
call in the child, or maybe in thewait...()
call in the parent?If you suspect it is something about Qt or
QProcess
which is problematic you could always try the equivalent of the code you show but just usingfork
/exec
/wait()
so nothing Qt-y in it.@JonB
Updated the repository with requested changes (this commit )I tried
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");
but nothing changed.
my simplefork
andexec
programs are included in repo, they don't use Qt and work fine. here are outputs:root@a78391006ad6:/problematic_src/build# ./qt_process_arm Hi This is qprocess simultation state: QProcess::Running error: QProcess::Crashed state: QProcess::NotRunning ls.readAllStandardOutput: "" ls.readAllStandardError: ""
root@a78391006ad6:/problematic_src/build# ./simple_fork Parent has x = 0 Child has x = 2
root@a78391006ad6:/problematic_src/build# ./simple_exec Parent Of parent process, pid = 1 child process, pid = 3058 parent process, pid = 3055 parent of child process, pid = 3055 program execution successful
-
@JonB
Updated the repository with requested changes (this commit )I tried
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");
but nothing changed.
my simplefork
andexec
programs are included in repo, they don't use Qt and work fine. here are outputs:root@a78391006ad6:/problematic_src/build# ./qt_process_arm Hi This is qprocess simultation state: QProcess::Running error: QProcess::Crashed state: QProcess::NotRunning ls.readAllStandardOutput: "" ls.readAllStandardError: ""
root@a78391006ad6:/problematic_src/build# ./simple_fork Parent has x = 0 Child has x = 2
root@a78391006ad6:/problematic_src/build# ./simple_exec Parent Of parent process, pid = 1 child process, pid = 3058 parent process, pid = 3055 parent of child process, pid = 3055 program execution successful
@noahlopezdev
Yep, you have done a good job of testing thefork
/exec
/wait()
pattern.I don't know what the issue with your Qt/
QProcess
code is then. I don't know how you get the output back from docker or whatever, but did neither of your newstateChanged
orerrorOccurred
slots get hit/produce any output? Try putting them before thels.start()
, not after. I would expect at minimumstateChanged
to be emitted.BTW, I'm not 100% sure you can use
QProcess
/waitForFinished()
/signals given that you have no Qt main event loop running (noQApplication::exec()
) in your code. I think you can, but not certain.WAIT, HANG ON!!!
You have not crated anQ[Core]Application
object in yourmain()
. Basically nothing will work in Qt till you do! I bet that is your issue.... -
@noahlopezdev
Yep, you have done a good job of testing thefork
/exec
/wait()
pattern.I don't know what the issue with your Qt/
QProcess
code is then. I don't know how you get the output back from docker or whatever, but did neither of your newstateChanged
orerrorOccurred
slots get hit/produce any output? Try putting them before thels.start()
, not after. I would expect at minimumstateChanged
to be emitted.BTW, I'm not 100% sure you can use
QProcess
/waitForFinished()
/signals given that you have no Qt main event loop running (noQApplication::exec()
) in your code. I think you can, but not certain.WAIT, HANG ON!!!
You have not crated anQ[Core]Application
object in yourmain()
. Basically nothing will work in Qt till you do! I bet that is your issue....@JonB
slots produce output, (as mentioned above)
Here is stateChanged output:state: QProcess::Running state: QProcess::NotRunning
here is errorOccurred output:
error: QProcess::Crashed
-
@JonB
slots produce output, (as mentioned above)
Here is stateChanged output:state: QProcess::Running state: QProcess::NotRunning
here is errorOccurred output:
error: QProcess::Crashed
@noahlopezdev Start with my latest update:
You have not crated an Q[Core]Application object in your main(). Basically nothing will work in Qt till you do! I bet that is your issue....
-
@noahlopezdev Start with my latest update:
You have not crated an Q[Core]Application object in your main(). Basically nothing will work in Qt till you do! I bet that is your issue....
@JonB
Sorry for late response, I am new user and can't post in sooner :(
Changed the code, Is that better now?
I Still get the same output -
@JonB
Sorry for late response, I am new user and can't post in sooner :(
Changed the code, Is that better now?
I Still get the same output@noahlopezdev said in qprocess docker problem:
Changed the code, Is that better now?
Yep!
I Still get the same output
Damn! :)
I have a feeling that
QProcess::Crashed
is emitted by Qt just when it cannot find the child PID to wait on. It is possible nothing is actually "crashing". The output ofstate: QProcess::Running state: QProcess::NotRunning
might indicate it ran and exited. But we don't see
QProcess::Starting
. I asked you to move theconnect()
s to immediately afterQProcess ls;
, before thestart()
, please.Let's test when the command actually does run, no matter what is reported. Your programming seems good so I leave you to write the code :)
- Can you somehow see in the docker/host/whatever so you can do things while your Qt app is running? Change the command to like
ls.start("/usr/bin/sleep", QStringList() << "10");
Can you then see that
sleep
process running? Like viaps
ortop
?- Change it to e.g.
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
After all finished does that file exist? Does it contain any output?
- Get rid of
waitForFinished()
. Do everything with the signals & slots fromQProcess
(includingstarted
&finished
signals), nowait...()
calls. Any difference in behaviour? What steps does it/does it not go through? (This probably means removing yoursleep(10);
and letting code enter theapp.exec()
.)
Otherwise I'm running out of ideas, especially since your own C
fork
/exec
/wait()
does work.... -
@noahlopezdev said in qprocess docker problem:
Changed the code, Is that better now?
Yep!
I Still get the same output
Damn! :)
I have a feeling that
QProcess::Crashed
is emitted by Qt just when it cannot find the child PID to wait on. It is possible nothing is actually "crashing". The output ofstate: QProcess::Running state: QProcess::NotRunning
might indicate it ran and exited. But we don't see
QProcess::Starting
. I asked you to move theconnect()
s to immediately afterQProcess ls;
, before thestart()
, please.Let's test when the command actually does run, no matter what is reported. Your programming seems good so I leave you to write the code :)
- Can you somehow see in the docker/host/whatever so you can do things while your Qt app is running? Change the command to like
ls.start("/usr/bin/sleep", QStringList() << "10");
Can you then see that
sleep
process running? Like viaps
ortop
?- Change it to e.g.
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
After all finished does that file exist? Does it contain any output?
- Get rid of
waitForFinished()
. Do everything with the signals & slots fromQProcess
(includingstarted
&finished
signals), nowait...()
calls. Any difference in behaviour? What steps does it/does it not go through? (This probably means removing yoursleep(10);
and letting code enter theapp.exec()
.)
Otherwise I'm running out of ideas, especially since your own C
fork
/exec
/wait()
does work....@JonB
refactored the code, please have a look at here.
I have monitored the environment withtop
, nothing shows up but in/proc/<PID>/task/
i see another pid.
here is what I think is parent process:root@a78391006ad6:/proc/6104/task/6104# ls -la ... lrwxrwxrwx 1 root root 0 Jul 9 11:52 exe -> /usr/bin/qemu-arm-static ...
and i think this is child process.
root@a78391006ad6:/proc/6104/task/6106# ls -la ... lrwxrwxrwx 1 root root 0 Jul 9 11:52 exe -> /usr/bin/qemu-arm-static ...
Are there any usefull files in
proc
that I can inspect?ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
This was a NICE idea, but sadly,
output
is not created :/
Here is full output of the program:root@a78391006ad6:/problematic_src/build# ./qt_process_arm Hi This is qprocess simultation stateChanged: QProcess::Starting stateChanged: QProcess::Running Process started errorOccurred: QProcess::Crashed stateChanged: QProcess::NotRunning Process finished: code: 1059250828 status: QProcess::CrashExit
exit code is always
1059250828
no matter what the input command is.Here is output of the code on my amd64 machine(a proof that code works fine, and something is wrong with
qemu
)oem@oem:~/programming/$ ./qt_process_arm Hi This is qprocess simultation stateChanged: QProcess::Starting stateChanged: QProcess::Running Process started stdout: "bin\nboot\ndev\netc\nhome\nlib\nlib32\nlib64\nlibx32\nmedia\nmnt\nopt\nproc\nroot\nrun\nsbin\nsnap\nsrv\nstorage\nsys\ntmp\nusr\nvar\n" stateChanged: QProcess::NotRunning Process finished: code: 0 status: QProcess::NormalExit
-
@JonB
refactored the code, please have a look at here.
I have monitored the environment withtop
, nothing shows up but in/proc/<PID>/task/
i see another pid.
here is what I think is parent process:root@a78391006ad6:/proc/6104/task/6104# ls -la ... lrwxrwxrwx 1 root root 0 Jul 9 11:52 exe -> /usr/bin/qemu-arm-static ...
and i think this is child process.
root@a78391006ad6:/proc/6104/task/6106# ls -la ... lrwxrwxrwx 1 root root 0 Jul 9 11:52 exe -> /usr/bin/qemu-arm-static ...
Are there any usefull files in
proc
that I can inspect?ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
This was a NICE idea, but sadly,
output
is not created :/
Here is full output of the program:root@a78391006ad6:/problematic_src/build# ./qt_process_arm Hi This is qprocess simultation stateChanged: QProcess::Starting stateChanged: QProcess::Running Process started errorOccurred: QProcess::Crashed stateChanged: QProcess::NotRunning Process finished: code: 1059250828 status: QProcess::CrashExit
exit code is always
1059250828
no matter what the input command is.Here is output of the code on my amd64 machine(a proof that code works fine, and something is wrong with
qemu
)oem@oem:~/programming/$ ./qt_process_arm Hi This is qprocess simultation stateChanged: QProcess::Starting stateChanged: QProcess::Running Process started stdout: "bin\nboot\ndev\netc\nhome\nlib\nlib32\nlib64\nlibx32\nmedia\nmnt\nopt\nproc\nroot\nrun\nsbin\nsnap\nsrv\nstorage\nsys\ntmp\nusr\nvar\n" stateChanged: QProcess::NotRunning Process finished: code: 0 status: QProcess::NormalExit
@noahlopezdev said in qprocess docker problem:
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
This was a NICE idea, but sadly, output is not created :/That tells us that it did not even start
/usr/bin/sh
. If it had it would have created the/tmp/output
redirection file. Also you never found thesleep
process running. The implication is that it cannot successfully spawn the/any child process. I do not know why.