qprocess docker problem

noahlopezdev

Hi,
I have an amd64 machine and I want to cross-compile a Qt program for arm32 with docker(using qemu-arm-static). I tried many images like arm32v7 and others but at last I use official ubuntu docker image and then use apt-get to install necessary Qt libraries. everything looks fine but when I try to start a process using QProcess, it alwayse crashes (ProcessError::Crashed).
I am pretty sure that I use QProcess correctly, becuse if I copy the compiled binary on arm machine, it just works fine and problem only happens in container in amd64 machine.
Any clues how can i fix or debug that? apparently strace and gdb do not work on qemu

JonB

@noahlopezdev
I have never used Docker. My colleague tells me how great it is, all I read up is people stuck when it goes wrong :)

This is such a low-level and unusual error that --- if you don't get a better suggestion --- I would look for ProcessError::Crashed in Qt/Linux source code (e.g. woboq if you need to) and see what can actually raise that. May not tell you what the cause is, but at least you'll know where it's happening from. QProcess is only a wrapper to the Linux system calls.

SGaist

Hi and welcome to devnet,

What exactly are you running with QProcess ?

noahlopezdev

@SGaist
Thank you fpr your response.
I tried many commands and they all crashed. my original use case was ldd <PROGRAM_PATH> but something like echo and ls also crashes.
P.S. ldd works fine in bash (in emulated container)

noahlopezdev

@JonB
Thank you for your response, well I inspected the code and if I'm not mistaking, Qt uses fork() in Linux to create processes. I used fork() in a simple program and it works :(

SGaist

Can you provide a minimal project that shows the behaviour ?

JonB

@noahlopezdev
Yes it will use fork(). That will not "crash" your parent. I suggested you look to see what path Qt code would have to follow to encounter ProcessError::Crashed in the sources, so that you could reason back from there.

noahlopezdev

@SGaist
Yes, This github repository reproduces the bug.

JonB

@noahlopezdev
You will probably get the same result (unfortunately), but what about trying for your command:

ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");

just in case the /usr/bin/sh actually runs (though I'm guessing it won't, it will just cause the same error itself) and produces some output which might tell us what is going on?

You might also try attaching a slot to QProcess::stateChanged(QProcess::ProcessState newState) which prints out newState so that you can retrieve it in your readAllStandard...(). We might know which steps it got through before it "crashed". There is also signal QProcess::errorOccurred.

I used fork() in a simple program and it works :(

In a program compiled with Qt libraries, or a non-Qt standalone program? I wouldn't mind seeing the source you used which you say works? Don't forget it would not surprise me that in whatever goes wrong it is not on the fork() statement --- which really ought work (unless you have no memory left!) --- but more likely on the subsequent exec...() call in the child, or maybe in the wait...() call in the parent?

If you suspect it is something about Qt or QProcess which is problematic you could always try the equivalent of the code you show but just using fork/exec/wait() so nothing Qt-y in it.

noahlopezdev

@JonB
Updated the repository with requested changes (this commit )

I tried
ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls /");
but nothing changed.
my simple fork and exec programs are included in repo, they don't use Qt and work fine. here are outputs:

root@a78391006ad6:/problematic_src/build# ./qt_process_arm
Hi This is qprocess simultation
state: QProcess::Running 
error: QProcess::Crashed 
state: QProcess::NotRunning 
ls.readAllStandardOutput: ""
ls.readAllStandardError: ""

root@a78391006ad6:/problematic_src/build# ./simple_fork
Parent has x = 0
Child has x = 2

root@a78391006ad6:/problematic_src/build# ./simple_exec
Parent Of parent process, pid = 1
child process, pid = 3058
parent process, pid = 3055
parent of child process, pid = 3055
program execution successful

JonB

@noahlopezdev
Yep, you have done a good job of testing the fork/exec/wait() pattern.

I don't know what the issue with your Qt/QProcess code is then. I don't know how you get the output back from docker or whatever, but did neither of your new stateChanged or errorOccurred slots get hit/produce any output? Try putting them before the ls.start(), not after. I would expect at minimum stateChanged to be emitted.

BTW, I'm not 100% sure you can use QProcess/waitForFinished()/signals given that you have no Qt main event loop running (no QApplication::exec()) in your code. I think you can, but not certain.

WAIT, HANG ON!!!
You have not crated an Q[Core]Application object in your main(). Basically nothing will work in Qt till you do! I bet that is your issue....

noahlopezdev

@JonB
slots produce output, (as mentioned above)
Here is stateChanged output:

state: QProcess::Running 
state: QProcess::NotRunning

here is errorOccurred output:

error: QProcess::Crashed

JonB

@noahlopezdev Start with my latest update:

You have not crated an Q[Core]Application object in your main(). Basically nothing will work in Qt till you do! I bet that is your issue....

noahlopezdev

@JonB
Sorry for late response, I am new user and can't post in sooner :(
Changed the code, Is that better now?
I Still get the same output

JonB

@noahlopezdev said in qprocess docker problem:

Changed the code, Is that better now?

Yep!

I Still get the same output

Damn! :)

I have a feeling that QProcess::Crashed is emitted by Qt just when it cannot find the child PID to wait on. It is possible nothing is actually "crashing". The output of

state: QProcess::Running 
state: QProcess::NotRunning

might indicate it ran and exited. But we don't see QProcess::Starting. I asked you to move the connect()s to immediately after QProcess ls;, before the start(), please.

Let's test when the command actually does run, no matter what is reported. Your programming seems good so I leave you to write the code :)

Can you somehow see in the docker/host/whatever so you can do things while your Qt app is running? Change the command to like

ls.start("/usr/bin/sleep", QStringList() << "10");

Can you then see that sleep process running? Like via ps or top?

Change it to e.g.

ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");

After all finished does that file exist? Does it contain any output?

Get rid of waitForFinished(). Do everything with the signals & slots from QProcess (including started & finished signals), no wait...() calls. Any difference in behaviour? What steps does it/does it not go through? (This probably means removing your sleep(10); and letting code enter the app.exec().)

Otherwise I'm running out of ideas, especially since your own C fork/exec/wait() does work....

noahlopezdev

@JonB
refactored the code, please have a look at here.
I have monitored the environment with top, nothing shows up but in /proc/<PID>/task/ i see another pid.
here is what I think is parent process:

root@a78391006ad6:/proc/6104/task/6104# ls -la
...
lrwxrwxrwx  1 root root 0 Jul  9 11:52 exe -> /usr/bin/qemu-arm-static
...

and i think this is child process.

root@a78391006ad6:/proc/6104/task/6106# ls -la
...
lrwxrwxrwx  1 root root 0 Jul  9 11:52 exe -> /usr/bin/qemu-arm-static
...

Are there any usefull files in proc that I can inspect?

ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");

This was a NICE idea, but sadly, output is not created :/
Here is full output of the program:

root@a78391006ad6:/problematic_src/build# ./qt_process_arm
Hi This is qprocess simultation
stateChanged: QProcess::Starting 
stateChanged: QProcess::Running 
Process started 
errorOccurred: QProcess::Crashed 
stateChanged: QProcess::NotRunning 
Process finished: code:  1059250828 status:  QProcess::CrashExit

exit code is always 1059250828 no matter what the input command is.

Here is output of the code on my amd64 machine(a proof that code works fine, and something is wrong with qemu)

oem@oem:~/programming/$ ./qt_process_arm
Hi This is qprocess simultation
stateChanged: QProcess::Starting 
stateChanged: QProcess::Running 
Process started 
stdout: "bin\nboot\ndev\netc\nhome\nlib\nlib32\nlib64\nlibx32\nmedia\nmnt\nopt\nproc\nroot\nrun\nsbin\nsnap\nsrv\nstorage\nsys\ntmp\nusr\nvar\n" 
stateChanged: QProcess::NotRunning 
Process finished: code:  0 status:  QProcess::NormalExit

JonB

@noahlopezdev said in qprocess docker problem:

ls.start("/usr/bin/sh", QStringList() << "-c" << "/usr/bin/ls / > /tmp/output 2>&1");
This was a NICE idea, but sadly, output is not created :/

That tells us that it did not even start /usr/bin/sh. If it had it would have created the /tmp/output redirection file. Also you never found the sleep process running. The implication is that it cannot successfully spawn the/any child process. I do not know why.