GUI operations across threads (again)



  • The subject of GUI operations across threads is heavily discussed, but while studying it I have found two conflicting basic answers :

    1. GUI operations across threads are impossible - use signals and slots
    2. GUI operations across threads are possible - enclose GUI operations in the thread within qApp->lock() and qApp->unlock()

    I am faced with porting to Qt a large Windows product which has only one GUI thread and multiple worker threads. Using the first method would require creating hundreds of signals and slots, so I would prefer to avoid it. My problem is that the second method is only mentioned in relatively older documentation of Qt.

    Should using the second method be possible, I could limit the locking to small fragments of code that manipulate GUI object attributes or do show/hide, and for GUI objects created in a thread use moveToThread to move them to the GUI thread.

    So my question is : Does the qApp->lock() method still exist, or only signals and slots ? If it exists, can it be used as described above ?


  • Qt Champions 2016

    Hi
    I was wondering. what does the current large Windows product do now?

    use events?

    Normally gui operations is never permitted from
    other thread so I wonder how it works now?

    ps. I could not find lock() in Qt5



  • @mrjj

    Under Windows, GUI operations work across threads - it's just good practice to use a mutex, which is why the port this way would be much easier since everything is already in place.

    And it seems like lock() is now gone in Qt5, which explains the lack of newer documentation. Is there any other way of blocking the GUI event loop from the thread for a short while ?


  • Qt Champions 2016

    @Harry123

    So my question is : Does the qApp->lock() method still exist, or only signals and slots ? If it exists, can it be used as described above ?

    http://doc.qt.io/qt-5/qapplication-members.html
    It appears it doesn't, and I personally don't remember such a method at all. In what version of Qt you saw that method?

    Under Windows, GUI operations work across threads

    I really, really doubt that. Can you provide a reference for this claim?

    it's just good practice to use a mutex

    Nope. It's simply necessary when you need a mutual exclusion lock. In a single thread a mutex is simply unnecessary overhead.

    Is there any other way of blocking the GUI event loop from the thread for a short while ?

    Possibly. It depends. When you want to block it, and from which thread?



  • @kshegunov

    lock() was apparently last seen in QT3.

    Windows GUI operations do work across threads and we do a lot of them - within reason of course. This is pragmatic, as much of the documentation of the older parts of the SDK still dates from Windows 3.1 and doesn't include the better multi-threading introduced with the NT kernel. Object handles in Windows NT are a global resource, not limited to one thread.

    From your strong objection I take it that it wouldn't work with Qt (or was never tested) ? Would you have any idea of a technique that can minimize the number of signals/slots ?


  • Qt Champions 2016

    @Harry123 said:

    lock() was apparently last seen in QT3.

    This explains it. Qt 3 is really, really ancient.

    Windows GUI operations do work across threads and we do a lot of them

    Okay, I'll assume you're correct, although MSDN mentions nothing of this.

    From your strong objection I take it that it wouldn't work with Qt (or was never tested) ?

    Objection to? I was just asking when and from where you want to block the event loop. If you're talking about Qt, it doesn't matter if the win API is thread-safe, the Qt GUI is not, and it's not even reentrant.

    Would you have any idea of a technique that can minimize the number of signals/slots ?

    Direct function calls. But why would you want to minimize them in the first place? I'd think you should want to maximize them ...


  • Qt Champions 2016

    @Harry123
    Hi
    I was wondering the windows app u are porting. Is it qt3 ? or a native win app?

    Must you must re-create the GUI in Qt and the threads or can you reuse those?



  • @Harry123 Can't you just use QMetaObject::invokeMethod? This will do the right thing.



  • @mrjj : It's a native win app.

    @cheezus : Does QMetaObject::invokeMethod require calling connect, or can it just call any slot? Not requiring connect will reduce the amount of coding.


  • Qt Champions 2016

    @Harry123 said:

    Does QMetaObject::invokeMethod require calling connect, or can it just call any slot?

    You can call any slot and/or invokable function (i.e. one that has been declared with the Q_INVOKABLE macro).



  • @kshegunov :
    @cheezus :

    That's nice to know.
    I assume that QMetaObject::invokeMethod is the best solution, and will try it in a limited way over the next few days.



  • @Harry123 No, it can call any function exposed to the meta object system.


  • Qt Champions 2016

    @Harry123
    QMetaObject::invokeMethod has somewhat limited uses, and I really doubt this will solve your problem.
    As I told @mrjj over the chat you have much bigger problems than the threading. Qt's widgets are alien (non-native) and the painting is done over a single window's surface (device context handle). Each of the child widgets manages its own geometry and does its painting, but only the top-level widget actually holds a handle to the window surface. When there's a paint event the top-level window will delegate the painting to the child widgets who actually paint on the top-level windows's surface. This I don't see how you're going to be able to work around easily. Possibly by enforcing native widgets all along the object tree, but this'd cause a bit hit on performance.



  • @kshegunov I don't see how that's relevant but I missed the chat so perhaps there is some missing context.


  • Qt Champions 2016

    @cheezus said:

    I don't see how that's relevant but I missed the chat so perhaps there is some missing context.

    Nope, you didn't miss much. The relevance is this:
    The OP is asking how he can paint using the win API from multiple threads onto widgets. So there are two problems, one how to thread the GUI code (which as established would be iffy at best), and secondly how to paint with the win API over widgets. I think the second problem is much more hard to work around.



  • @kshegunov I take OP to mean "I have an existing WINAPI project that draws on multiple windows from multiple threads and I need to port this to Qt"

    The real response to this thread is what does this code look like? If it is simply setting labels or adding stuff to lists. Then yes, invokeMethod will work fine. If he is literally using GDI calls to paint onto a surface, then that requires a different approach. There are multiple options here but the real answer depends on what this magic WINAPI application looks like. It also depends on the target platforms.

    OP, hire someone who knows what they're doing to guide you otherwise you'll just be creating future bugs/crashes if you guess your way through it.


  • Qt Champions 2016

    @cheezus

    If he is literally using GDI calls to paint onto a surface, then that requires a different approach

    This is how I understand it.



  • @kshegunov Then heaven help us all



  • @kshegunov :
    @cheezus :

    NO! Absolutely NOT! No drawing from multiple threads. This one-million line application was (somewhat) ported to Qt, but for the moment the multi-thread part is #define'd out, which disables many functions.

    About multi-thread support in Windows :

    Some Windows objects, like timers, need to be created in their thread, but can then be killed from any other, in spite of what the outdated documentation says.

    But don't forget that practically all Windows objects have a message interface via SendMessage and PostMessage. Even when the SDK offers seemingly direct functions, internally many are implemented as sent messages. This includes practically all objects : push-buttons, checkboxes, comboboxes, tree objects etc. Which allows Windows to be completely thread-agnostic for very many types of operations. Windows since NT is a real multi-threaded OS whose SDK is (almost) completely re-entrant and thread-safe. The "main" thread has no further significance than that of the system library automatically installing there an event loop.

    By comparison, Qt uses direct function calls which almost always precludes inter-thread operations. While being much more esthetic and powerful than Windows SDK, it also requires some acrobatics when programming multiple threads.

    It would be nice if some future version of Qt could also become thread-agnostic for these types of operations, automatically crossing thread boundaries with invokeMethod or other. This will in effect supersede much of the current mechanical need for signals/slots, leaving them for more sophisticated needs.

    For a dummy example : If a thread wants to call QPushButton::setText, a signal and a slot are required, and maybe also sub-classing QPushButton, which I see as much too much bother for such a trivial thing.


  • Qt Champions 2016

    @Harry123 said:

    But don't forget that practically all Windows objects have a message interface via SendMessage and PostMessage.

    Qt's objects have that message interface as well. QCoreApplication::sendMessage and QCoreApplicaiton::postMessage.

    PS
    The win API is C. It has no objects (in the C++ meaning), it has handles (which are opaque pointers to some internal structure). Its message processing is also synchronous, which X11's is not. But I don't want to enter a debate on the win API and its design, or lack thereof.

    The "main" thread has no further significance than that of the system library automatically installing there an event loop.

    As recently asked, you can in principle create Qt's application object (thus the main event loop) in a thread different from the main one.

    By comparison, Qt uses direct function calls

    It uses signals to notify of state changes, and slots to respond to those changes. This however does not preclude any threading and/or doesn't require acrobatics, only some understanding of how the framework actually works.

    It would be nice if some future version of Qt could also become thread-agnostic for these types of operations, automatically crossing thread boundaries with invokeMethod or other.

    QMetaObject::invokeMethod is safe across thread boundaries (with auto/queued/blocking queued connection type), so that's already in place.

    This will in effect supersede much of the current mechanical need for signals/slots ... For a dummy example ...

    It will most certainly supersede nothing. Your thread is not supposed to know of the existence of push buttons at all. The worker object that sits in that thread is supposed to just notify anyone that's interested something has happened. That's what decoupling means in the first place. The moment your thread or worker object know of the push button you couple them up, and if something changes (e.g. the push button becomes a label) you go about changing/inspecting every place where that object was used.



  • @kshegunov said:

    Qt's objects have that message interface as well. QCoreApplication::sendMessage and QCoreApplicaiton::postMessage.

    Yes, but how do you send a QPushButton::setText ?

    Your thread is not supposed to know of the existence of push buttons at all.

    Why not ? Why should I need slots that do nothing useful except call basic functions ?

    Please understand that I'm coming from another context and from a general-purpose OS, whose SDK, while ugly, is really flexible. With Qt multi-threading I don't have this freedom. I feel that the developers are imposing their own paradigms of what is multi-threading by force of action.

    Multi-threading in Qt currently has an overhead and a complexity. I understand that changing this would require lots of development work and may not be justified because there are not that many heavily multi-threaded graphical applications using Qt.

    For myself, my future is now settled : I'm going to be writing lots and lots of calls to invokeMethod.


  • Moderators

    @Harry123 "Why not ? Why should I need slots that do nothing useful except call basic functions ?" to separate logic from UI. @kshegunov already said: decoupling. Why should your logic (business logic) know UI details? Both are not related to each other and should be decoupled. This is nothing Qt specific, it's software design.


  • Qt Champions 2016

    @Harry123

    Yes, but how do you send a QPushButton::setText ?

    By posting a meta call event, like done here, which is ultimately what QMetaObject::invokeMethod does when used with queued type of connection.

    Why not ? Why should I need slots that do nothing useful except call basic functions ?

    The punchline is the last sentence of my previous post. Basically, because that creates a spagetti C code where everyone knows about everyone else( and thread access serialization is done by hand, but that's a minor detail).

    Please understand that I'm coming from another context

    I gathered that.

    and from a general-purpose OS

    The world doesn't begin with windows, and it certainly doesn't end there. Linux and OSX are no less "general-purpose".

    whose SDK, while ugly, is really flexible.

    It's a matter of opinion and of some debate, but I don't want to get into it, as stated.

    With Qt multi-threading I don't have this freedom. I feel that the developers are imposing their own paradigms of what is multi-threading by force of action.

    Not at all, you can still derive from QThread override QThread::run and do the synchronization by hand, no one is stopping you from doing that. The only thing that's of matter in this case is that the GUI is not thread-safe or reentrant, so it's your responsibly as with any object/library that is not thread safe by default to work around it.

    I understand that changing this would require lots of development work and may not be justified because there are not that many heavily multi-threaded graphical applications using Qt.

    It most probably won't change, as Qt is not a MS windows only toolkit. And there're considerations going beyond what the win API might or might not provide.

    Also there are enough Qt applications using threads, most of them with OpenGL which can be threaded by design. For those that use widgets and don't employ opengl the GUI thread is pretty much enough, as the heavy lifting (i.e. calculations, processing and the such) is moved to worker threads and the GUI is only signaled when changes should be reflected.



  • @kshegunov :

    the GUI is only signaled when changes should be reflected

    Well, that's the paradigm. To be clear, I'm not saying that GUI operations should work across threads, just that some basic functions could do their own invokeMethod so I wouldn't have to code that much :;

    When all is said and done - all I wanted was to shake some dust out of some paradigms, even if that wouldn't help me now.


  • Qt Champions 2016

    Hi, just a side note.
    Using lambdas for slots
    can reduce the coding for signal&slot to some
    degree for those trivial cases.

    Also, those gui call you have now.
    That would be like
    windows API call with handles to the widgets ?
    or was it possible in qt3 to pause main loop
    and do directly ui->label->setText() ?



  • @mrjj :

    Lambdas would require C+11, not so ? I would prefer to require as little as possible.

    And yes, in qt3 I found just such examples. This approach would work for most modern OS, since qt3 did work across many environments, but maybe Qt developers later encountered some OS that were more limiting.


  • Qt Champions 2016

    @Harry123
    yes it does. But even for small embedded boards the compiler can do c++11
    (often) so not sure it still makes sense to avoid. Even I do get your point.

    Well I have old kylix program in linux. they used Qt3 for that.
    I do have that sync function and its not a good solution as if your worker threads
    are very busy updating/talking to GUI, the whole app get slow as each time it
    suspend main loop. So while convenient back then, i have issues
    with it now as we have 3 times the number of threads and
    they all talk and pause main loop.

    So while you have my sympathies
    for having to port it all,
    it might not even have worked really well anyway. :)



  • @Harry123 OK, so your application's current model is as follows:

    Thread N: User code -> Call windows API (say SetText on button) -> SendMessage(HWND for button,WM_SETTEXT)

    GUI Thread: --> PostMessage(HWND for button,WM_SETTEXT)

    The equivalent in Qt is:

    Thread N: User code -> QMetaObject::invokeMethod(button,"setText",text);

    GUI Thread: --> Button::setText(text)

    Since you're just doing simple property setters, I'd suggest trying invoke method first. The only problem with invokeMethod as I see it is it isn't statically typed. But then again, neither is the Windows API.



  • @cheezus :

    setText was just a simple example given for the discussion.
    I had a little look at some Qt headers, and Q_INVOKABLE is not used very much with the method declarations.

    @mrjj :

    The program performs fine on Windows, since the majority of SDK calls are just messages, others do work across threads, while only some require freezing for an unnoticeable small time. And thanks for your much needed sympathies.


  • Qt Champions 2016

    @Harry123 said:

    just that some basic functions could do their own invokeMethod so I wouldn't have to code that much

    They could, but they shouldn't. In 99% of cases that'd be a terrible overhead for a simple property change. As established, use QMetaObject::invokeMethod to work around it.

    Lambdas would require C+11, not so ?

    Qt 5.7, I believe, will require C++11 compatible compiler, so in a few months it wouldn't matter anyway.

    I had a little look at some Qt headers, and Q_INVOKABLE is not used very much with the method declarations.

    Possibly because everything that's declared as a slot is already invokable. So that macro appears only for non-slot members that should be known to the meta-object system.

    while only some require freezing for an unnoticeable small time

    As I said, it's possible to block the event loop, but you never said from which thread you want to do that. In any case, here's the simplest way of doing it:

    class EventLoopBlocker : public QObject
    {
        Q_OBJECT
    
    public:
        EventLoopBlocker(QObject * parent)
            : QObject(parent), lock(0), waiter(0)
        {
        }
    
        Q_INVOKABLE void block()
        {
            waiter.release();
            lock.acquire();
        }
    
        void unblock()
        {
            lock.release();
        }
    
        void wait()
        {
            waiter.acquire();
        }
    
    private:
        QSemaphore lock;
        QSemaphore waiter;
    }
    

    Usage is simple:

    1. Create the object in the main/GUI thread (for example as child of QApplication).
    2. Block the event loop (from thread other than the main one):
    EventLoopBlocker * blocker; //< You can pass the pointer when you create the threads.
    
    QMetaObject::invokeMethod(blocker, "block", Qt::QueuedConnection);
    blocker->wait();
    
    1. Unblock the event loop (from thread other than the main one obviously):
    EventLoopBlocker * blocker; //< You can pass the pointer when you create the threads.
    blocker->unblock();
    

    This is all there is to it. However @mrjj's note on responsiveness applies.

    PS.
    The sample code will work with one thread controlling the GUI thread, for universal solution (i.e. that takes into accounts multiple calls to block()) you should fiddle a bit with the logic and the samaphores.



  • @kshegunov :

    Thanks for this very ingenious freeze function.

    everything that's declared as a slot is already invokable

    This is the missing piece in the puzzle, which will minimize the required code much more.

    @cheezus :
    @mrjj :

    Many thanks to you all for your much appreciated help.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.