Running into problems using a simple gui multithreaded application. FRUSTRATED AND NEED HELP !

alonhalawi

I am working on other projects so it took me a while to go back to this one.

Chris Kawa, I agree with most of what you say, and I'll talk about those stuff later on another post ..

I tried emitting a signal from a process I created, to call a slot to change a picture in one of my controls on my main dialog. Still doesn't work. Other experiments do work, but again, none if them are what I desire.

Some little insights from what I tried.

1:
I see that the main dialog has an event loop, and it is a single threaded object. According to my experiments, if I click a button on the main dialog to change a picture (I use setStyleSheet,) it works. But again, it's not what I seek to do.

Basicly, the function does have to end if I do that, but I can call QApplication.processEvents() to force the chage before it ends. So, in a way, this is a function I might use. I bet this is not the common practice anyway ... (correct me if I'm wrong on this one)

Here is the code related to this insight:

2:
When I use the tester process to emit the signal to the gui process, the tester goes blocking, this is strange, only when the emitted slot call ends, the tester process continues. Now WHYYY does this phenomenon happends, those are not even the same process, aren't they !?

main.py
@
import threading, sys, Queue
import multiprocessing as mp
from PySide import *
from pagex import *

import AA_Tester

shared queue object between my threads

mqGui2AA = mp.Queue()

control dictionary

shared_objects = {'action': 'init', 'mqGui2AA':mqGui2AA}

AA_Instance=AA_Tester.MainAA(shared_objects)

qt gui

app = QtGui.QApplication(sys.argv)
form = MainDialog(shared_objects=shared_objects, AA_Instance=AA_Instance)
form.show()
app.exec_()
@

maindialog.py
@
class MainDialog(QtGui.QDialog, pages.Ui_Pages):
def init(self, parent = None, shared_objects = None, AA_Instance = None):
super(MainDialog, self).init(parent)
self.setupUi(self)

      self.next.clicked.connect(self.next_clicked)
      self.retry.released.connect(self.retry_call)

################ it spawns many threads, so I chosed to put the signal on the AA file #############
AA_Instance.emit_gui_update.connect(self.parse_gui_update)

      self.AA = AA_Instance
      self.process = mp.Process(target=self.AA.test)
      self.process.start()

@

pages.py
@
class Ui_Pages(object):
def next_clicked(self):
self.shr_objs['mqGui2AA'].put('next')

def retry_call(self):
self.modify_text( ... modifying text ... )
self.modify_pic( ... modifying pictures ... )

@

AATester.py
@
class MainAA(QtCore.QObject):

emit_gui_update = QtCore.Signal(list)

def init(self, shared_objects):

print "in MainAA::init"
QtCore.QObject.init(self, parent=None)

self.shr_objs = shared_objects

def test(self):

example of a test

while (True):
message = self.shr_objs["mqGui2AA"].get()
print "test(): page 1, got message",message

loading config

self.do_some_test()
#####################3 this should emit a gui update ################ it doesn't do it !!!!
ret = self.emit_gui_update.emit(['/dev/ttyUSB0', 1, 1])

if message == 'next':
break

@

I have removed most of the debug prints so you could see the code.

- The first insite I wrote is just more frustrating, couse its a progress, and helpful ..

I am not able to explain the second insite, according to all the posts here, this should not happend in Qt !!!*

Chris Kawa

My Python is a little rusty, but I believe this part is the problem:
@
AA_Instance.emit_gui_update.connect(self.parse_gui_update)
self.AA = AA_Instance
self.process = mp.Process(target=self.AA.test)
self.process.start()
@
It's not immediately obvious but for performance reasons there are different types of connect. This type is controlled by the type parameter in the connect() call.
The default value is AutoConnection that is evaluated to either DirectConnection or QueuedConnection (or others that are of no interest here).

When DirectConnection is used an emit is basically a simple function call. As every function call this is blocking.
When QueuedConnection is used emit puts a "marker" in the event loop of the thread that hosts the target object and the target thread calls the slot the next time it gets to process events. This is non-blocking and to be thread safe it uses mutexes (so it's a bit slower).

Now, another thing I said is "the thread that hosts the target object". In Qt each QObject is "assigned" to a specific thread (the main ui thread by default). Every slot is executed in the thread that object lives in, no matter from which thread the signal came.

The AutoConnection chooses the type of connection based on the threads the caller and receiver live in. Same thread -> direct, different -> queued.

In your case AA_Instance lives in the main thread. Now the tricky part is that a thread object itself also lives in the main thread, even though its run method (called by start) is started in the worker thread. Thread object is not in the same thread as the code it runs in start(). Because both - thread object and the receiver live in the same thread, Qt establishes a direct, blocking connection.

The usual way to deal with that is (in pseudo code)
@
UIObject ui = ...; //the receiver
QThread* t = ...; //a thread
Worker* w = ...; //a worker object
w->moveToThread(t); //assign the worker to the thread
connect(t, QThread::started, w, Worker::doSomeWork);
connect(w, Worker::someSignal, ui, UIObject::updateUi);
t->start(); //run the thread
@
Because the worker is moved to thread before the connect() the connection chosen is QueuedConnection.
Sorry for the C++'ish syntax. I know how to read Python but I'm terrible at writing it. I hope you can translate.

alonhalawi

Thank you Chris for the reply, I am trying it, I have already changed the connect param to QueuedConnection, but it didn't work.

I tried to do it with processes to make sure it won't be blocked, but it goes blocking even with processes.

the solution of moveToThread was tried before, and also didn't work, though I'm not sure I used the QueuedConnection attribute.

alonhalawi

Tried it, didn't work.

Now there must be some small detail I am missing -- again --.
But right now basically the thread doesn't start.

I really didn't want to go in to one of those tricky parts, that's why I chosed processes.

bq. In your case AA_Instance lives in the main thread. Now the tricky part is that a thread object itself also lives in the main thread, even though its run method (called by start) is started in the worker thread. Thread object is not in the same thread as the code it runs in start(). Because both – thread object and the receiver live in the same thread, Qt establishes a direct, blocking connection.

Now this is EXACTLY why I say, and you can and should quote me on this one, Qt TAKES ITS OWN PREFERENCE OF CONTROL !!

The whole idea of processes is that they all have their own domain of objects and whatever, and making something shared is simply shm get and attach, if you want it as direct as can be.

The whole idea of threads is to have a WHOLE DATA SEGMENT SHARED, but still, my responsibility to sync them.

The WHOLE IDEA OF 'MOVE TO THREAD' IS TO GET ME PISSED OFF !!

Chris Kawa

We should not mix processes with threads. They are completely different concepts. Processes know nothing about each other (unless using shared memory concepts - e.g. QSharedMemory). You can't connect objects in different processes.

Now as for threading - you easily loose your nerve, don't you? :) Threading is complicated and each language/lib has a different take on it. Qt's is that objects have thread affinity. Data is not that important, but access to it is (what thread reads/writes). To me it's quite elegant. You just mark an object as belonging to thread X and all its related code runs there. You still need to synchronize shared data access with mutexes and what not, don't get me wrong.

I guess Qt is just not for you if it makes you that mad. I've been using it for almost 7 years now and it's a love story :) Have you considered switching to a language/lib more to your liking?

alonhalawi

The reason of me loosing my nerve is that it supposed to be very simple.

If I had the ability to draw a rectangle on the screen and paint it according to anything, I have many ways to do it.

The only reason I chose Qt is for the graphics.

And it takes me a lot of digging, and getting to know the UI, the objects and the threading mechanisms, I mean, forcing me to read all of this stuff, then experiment, then I see what I have to define to make this simple stuff to work.

I'm positive I have missed something, but I am not going to port all my python work in PySide.

Look at the requirement for making this work:

manage signals and slots
1.1 define slots for functions you want to invoke
1.2 define signals (so you could emit the slots)
1.3 connect the signal(s) to the slot(s)
1.4 make sure it's a queued connection (it wouldn't work otherwise)
use a thread, not a process [1]
2.1 use the moveToThread when creating
2.2 use signal to emit changes on the gui (&make sure the func ends)
2.3 use QtGui.QApplication.processEvents() to force the updates when the func doesn't end

I can go on ..

All this stuff to handle, none of this belongs to the code itself, only preparations ..

[1] BTW in c, syncing threads is with mutexes, and using globals, syncing process is different but they communicate with ipc e.g. mq, sockets, shm ..

How is any of this comfortable ?

alonhalawi

I bet something is wrong with this code:

main.py
@
AA_Instance=AA_Tester.MainAA(shared_objects)

app = QtGui.QApplication(sys.argv)
form = MainDialog(shared_objects=shared_objects, AA_Instance=AA_Instance)
form.show()
app.exec_()
@

maindialog.py
@
class MainDialog(QtGui.QDialog, pages5.Ui_Pages):

def init(self, parent = None, shared_objects, AA_Instance):

self.AA = AA_Instance

self.thread = QtCore.QThread()
self.AA.moveToThread(self.thread)
self.AA.run_test.connect(self.AA.test)
self.thread.start()
@

the thread doesn't start

Chris Kawa

[quote author="alonhalawi" date="1405262476"]
Look at the requirement for making this work: (...)[/quote]

Yes, you need to learn library to properly use library. What's the problem?
1.1 one line
1.2 one line
1.3 one line
1.4 no code (if you know the library)
If you're a programmer you should know the difference anyway. You can actually make it work with both. It's just different.
2.1 QObject belongs to a thread it was created in. If you create the worker inside the running thread no need to use moveToThread, one line otherwise
2.2 one line. What do you mean make sure it ends? It ends. It's a function call. If you mess up the setup and it directly calls lengthy slot then of course it doesn't end. What is surprising about that?
2.3.No, you should not use processEvents. It's for cases when your code is so messed up you can't find other ways to fix it. It's a lifeboat, not a yaht. Should not happen in such a simple case. And again - a UI update should be a small function that ends right away. Don't do lengthy stuff in UI slots.

[quote]I can go on ..[/quote]Please do. (Denholm Reynholm ref. :) )

So, I counted 3-4 lines of preparation code (not counting the actual user code that does stuff) and one line (the emit) to make an asynchronous data exchange between threads. Sounds simple to me, but I can agree to disagree.

alonhalawi

I must have forgot the target, but as according to the examples I have, the placing of the function that I want to call as a thread is not done like this

QtCore.QThread(target= ***)
*

kenchan

Hey, why don't you point your questions to the Python group? Maybe some of those Python experts over there can help you.

alonhalawi

I'm considering moving the graphics to python. It's just exhausting missing some property, or widget attribute, or another feature that forces you to read a whole document of an object in qt ..

I do not go lengthly on the slot. I make a simple short change and go out, when it doesn't work I try looking for more examples that look similar to what I need.

I think I'll just do it with python, one line + one line + one line + noline with feature parameter queue-connection + reading another library kinda making me change my mind.

alonhalawi

Got it figured out, and done ...
with python.

Sorry Qt fans, I admit, I am a noob to Qt, I find it very non-scalable, however I truly recommend python (for graphics as well), and did it with less than a day (I studied most of the gui stuff)

simple, scalable, comfortable ... and most importantly, can adapt to more then one design option of mine.

Chris Kawa

If it's right for you then... well, it's right for you :)

Just as a comment - You made it in less then a day. Yup, for that size of apps Python might be the easier and faster option. But claiming Qt is not scalable... you clearly didn't do your research :)

Chris Kawa

Out of curiosity (and some boredom) I implemented a thread changing a color of a button every second. It took me < 5 minutes and there are exactly the 5 lines I was talking about needed to make threading work.
If you're not busy I would really like to see what it looks like in Python(without Qt). I don't mean to bash, I'm really curious.

@
class Worker : public QObject {
Q_OBJECT
public:
void doWork() {
while(true) {
QThread::currentThread()->sleep(1);
// 5 - emit the signal
emit foo(rand());
}
}
signals:
void foo(int);
};

class UI : public QPushButton {
public:
UI(QString text, QWidget* parent = nullptr) : QPushButton(text, parent) {
// 1 - start the thread
connect(this, &UI::clicked, &{ workerThread.start(); });
// 2 - fire up doWork in the thread
connect(&workerThread, &QThread::started, &worker, &Worker::doWork);
// 3 - modify ui in response to signal
connect(&worker, &Worker::foo, this, &UI::changeColor);
// 4 set thread affinity of worker object
worker.moveToThread(&workerThread);
}
private:
void changeColor(int color) {
setStyleSheet(QString("background-color: ") +
QColor::fromRgba(color).name());
}
Worker worker;
QThread workerThread;
};

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
UI ui("hello");
ui.show();
return a.exec();
}@

alonhalawi

sure, I'll post it, and also, out of curiosity, I'll go over the code you've posted to see what I've missed.

Right now, I have to get the whole project done, and also I am going to a trip in Europe tonight, so I will post the code when I get back.

It will probably be next Saturday.

I didn't mean to be such an **hole in this post, but I really don't do gui, so ..

I guess I need to invest my time to learn Qt the way it should, before I judge the lang.

alonhalawi

Will upload it very soon. Just got back from Prague :)

alonhalawi

Sorry for the delay, as I promised I am uploading the code.

Gui.py
@
import Tkinter as Tk
from PIL import Image, ImageTk
import tkMessageBox as tkMsgBox

def version_click(shr_objs):
print "version_click is called"
print 'this is my global segment:', shr_objs
tkMsgBox.showinfo('say hello', 'hello world')

def setupUi(MainDlg, shr_objs = None):
#setting up all the images and other stuff
raw_img_good = Image.open('images/good.jpg')
MainDlg.img_good = ImageTk.PhotoImage(raw_img_good)

#setting up all the widgets, and if needed, putting pictures to them
MainDlg.bg = Tk.Label(MainDlg)
MainDlg.bg.pack(expand=Tk.YES, fill=Tk.BOTH)
MainDlg.bg['image'] = MainDlg.img_bg

MainDlg.btnVer = Tk.Button(MainDlg, text='start', command = (lambda: start_click(shr_objs) ) )

widgets can be arranged using three main geometry managers: pack, grid and place, for this example I chosed pack (the recommended and simplest)

MainDlg.btnVer.pack()

MainDlg.btnVer.place(x=620, y=650, width=100, height=40)

if name == 'main':
MainDlg = Tk.Tk()
setupUi(MainDlg, {'action': 'init'})

MainDlg.mainloop() # for a gui app only enable this, or just run with -i option (interactive mode)

@

main.py
@
#!/usr/bin/python

import threading
import sys, time
import Gui
import Tkinter as Tk

import AA_Tester

shared queue object

mqGui2AA = mp.Queue()

global vars dictionary, I can choose any IPC I want here ..

shared_objects = {'action': 'init', 'mqGui2AA':mqGui2AA}
#shared_objects = {'action': 'init'}

creating main gui form

MainDlg = Tk.Tk()
Gui.setupUi(MainDlg, shr_objs = shared_objects)

shared_objects['MainDlg'] = MainDlg

AA = AA_Tester.MainAA(shared_objects)

MainDlg.mainloop()
@

partial file from AA_Tester.py
@
MainDlg = shr_objs['MainDlg']
MainDlg.bg['image'] = MainDlg.img_good
@