Unique blocking connection or somethig like it needed

Gourmand

Here is the situation: one thread rapidly sends QString in signals. Main thread receives these signals and appends QStrings to QPlainTextEditor. With simple queued connection this greatly slows down interface. What needed to force this work? A buffer with warranty to block thread until each QString will be added, then it releases thread, takes next QString from queue and so on. This is Qt::BlockingQueuedConnection. With it all works fine. Until thread is single. If connect two threads with Qt::BlockingQueuedConnection-s to one slot - we get dead lock. Second thread is not locked it does not know about processing of QString from 1st thread. It sends signal and locks application. Of course some sync can be made between two threads but this complicates application design. Especially if threads run in plugins (like in my case).

Anybody have the idea how to workaround this gracefully? I would be glad if some 7th connection type would exist - for example Qt::WaitBlockingQueuedConnection. It should work something like this: when connection transmits signal it blocks only the thread which it sent. If it was sent from another thread - then signal is stored to queue and another thread is blocked. After first signal processed first thread released, and second thread's signal is being processed.

Now looks like BlockingQueuedConnection locks not the sending thread but entire event processing. If there are two different slots and they are connected to two different signals but both connections are Blocking - then second signal deadly locks if first is being processed.

So why not threads are locked in this case but entire event processing? Signal processing knows who was the sender and can lock only the sender but leave other parts unlocked.

dangelog

I'm not getting most of your explanation. BlockingQueuedConnection blocks the emitting thread until all the connected slots have been invoked (which of course deadlocks if those slots are intended to be invoked from the current thread).

Instead of immediately appending the strings, can't you just buffer them somewhere and then append them in batches? Consider that your UI won't refresh faster than 60Hz anyhow.

Or: if you're reallying sending lots of strings, and flooding main the event loop (thus causing your UI to be unresponsive), don't use cross-thread signals and slots, but some other producer/consumer solution.

Gourmand

bq. I’m not getting most of your explanation. BlockingQueuedConnection blocks the emitting thread until all the connected slots have been invoked (which of course deadlocks if those slots are intended to be invoked from the current thread).

Now I see it blocks not the current thread only. I tested it twice. If another thread sends signal with BlockingQueuedConnection while first is being processed - then deadlock appears.

BlockingQueuedConnection blocks not until slot invocation but until end of signal processing ie return from slot ("from Namspaces description":http://doc.qt.nokia.com/4.7/qt.html): Qt::BlockingQueuedConnection 4 Same as QueuedConnection, except the current thread blocks until the slot returns.

bq. Instead of immediately appending the strings, can’t you just buffer them somewhere and then append them in batches?

But the queue is this ready to use buffer. Double buffering?

bq. flooding main the event loop (thus causing your UI to be unresponsive

BlockingQueuedConnection solves that.

bq. don’t use cross-thread signals and slots, but some other producer/consumer solution

Signals/slots is better way for me to make messaging system between threads. It works but sometimes worse than I expected. Again - threads are in different plugins. If I will use some another method then I'll have to repudiate from Qt...

andre

I think you might want to considder this:
In order for threads to work effectively, I think they need operate as independently from each other as possible. That means limiting the communication between the threads to a minimum. An added benefit is that it will lighten the load on the cross thread communication mechanism. It is quite easy to overload the mechanism to send signals between threads. If you add waiting for a reply and blocking, matters grow worse.

So... please re-think your design. Is it really necesairy that the two threads communicate with each other? Considder at least throtling the message stream from your provider thread. Like peppe said: once 1/60th second is the maximum rate you need, but there is a good chance that a much lower rate still results in snappy performance for you.

dangelog

[quote author="Gourmand" date="1314815365"]
Now I see it blocks not the current thread only. I tested it twice. If another thread sends signal with BlockingQueuedConnection while first is being processed - then deadlock appears.
[/quote]

Another thread than which one? Can you make an example or a testcase?

[quote]
BlockingQueuedConnection blocks not until slot invocation but until end of signal processing ie return from slot ("from Namspaces description":http://doc.qt.nokia.com/4.7/qt.html): Qt::BlockingQueuedConnection 4 Same as QueuedConnection, except the current thread blocks until the slot returns.
[/quote]

With "until all the connected slots have been invoked" I meant exactly that they are invoked and they returned. No problem there...

[quote]
bq. Instead of immediately appending the strings, can’t you just buffer them somewhere and then append them in batches?

But the queue is this ready to use buffer. Double buffering?
[/quote]

Which queue are you talking about? I was talking about the fact that it's nonsense to append every single string immediately to the textedit, and suggesting instead to buffer them and append them no more often than 50-60 times per second.

[quote]

bq. flooding main the event loop (thus causing your UI to be unresponsive

BlockingQueuedConnection solves that.
[/quote]

That's not a solution IMHO: you're blocking the thread which is generating the data, thus defeating the purpose of it being a separate thread.

[quote]
bq. don’t use cross-thread signals and slots, but some other producer/consumer solution

Signals/slots is better way for me to make messaging system between threads. It works but sometimes worse than I expected. Again - threads are in different plugins. If I will use some another method then I'll have to repudiate from Qt...
[/quote]

Of course they're are just a better way -- they're already available, and they work. I was talking about this:

[quote]
With simple queued connection this greatly slows down interface
[/quote]

Why is that? Is it because the main event loop has too many events to process (and/or a single event takes too long to be processed) and can't keep up? If so, you must reduce the pressure on the main event loop. I suggested both processing the strings in batches instead of one at a time, and eventually using another producer/consumer pattern for sending them to the objects in the main thread.

goetz

[quote author="Gourmand" date="1314815365"]Again - threads are in different plugins.
[/quote]

As you emphasize this aspect:
You've "been told before":http://developer.qt.nokia.com/forums/viewthread/6939/ that thread affinity has absolutely nothing to do with plugins, neither do plugins introduce any threading at all. Nothing in respect of this has changed in Qt in the meantime.

Gourmand

bq. Can you make an example or a testcase?

If I'll have time then I'll make. For while you can it by yourself. I have full hands with main job.

bq. Which queue are you talking about?

The one used in Queue dConnection or Blocking Queue edConnection

bq. That’s not a solution IMHO: you’re blocking the thread which is generating the data

This is your's HO only... In my task temporary blocking a thread is not a crime.

bq. thread affinity has absolutely nothing to do with plugins

Threads affinity itself - ok, nothing. But to organize data exchange between threads in different plugins lots of additional coding needed. Signal/slot is most useful ready made system for this. I need exchange not only QStrings - but all possible data included into QVariant.

lgeyer

Signals/slots are not the only way to communicate in multithreaded applications. When transferring large amounts of data bulk transfers or shared memory are usually more adequate solutions (both supported by Qt out of the box).

In addition, not every problem can be split into multiple threads. If your thread generates data as fast that the transfer of this data to another thread becomes a serious bottleneck this is usually a perfect indicator that you just encountered such a problem.

andre

Gourmand, if you are not willing to take any pointers, or provide clear test cases, then perhaps you should not ask for help here. I think we have been through this before...

The Qt signal slot connections that have the word Queue in them, are called that way because they end up in the event_queue_ that is being handled by the event loop. That queue however, is not to be confused with a general purpose queue that is OK to just fill up and forget about. At least, not if you want your application to stay performant.

goetz

[quote author="Gourmand" date="1314865363"]
bq. thread affinity has absolutely nothing to do with plugins

Threads affinity itself - ok, nothing. But to organize data exchange between threads in different plugins lots of additional coding needed. Signal/slot is most useful ready made system for this. I need exchange not only QStrings - but all possible data included into QVariant.
[/quote]

You have a thread problem. It is completely irrelevant if that is caused by threads created by some plugins or by threads from on single application binary. A QThread is as QThread, it even doesn't know if it's from the main app or a plugin.

If you're not willing to invest some minutes for a good test case, why should we bother wasting our times for solving so called "problems" for you? You may want to read "here":http://www.catb.org/~esr/faqs/smart-questions.html and understand why if you do not get the rationale yourself and if you can spend some secondes of your valuable time to read such stuff.

Good luck with solving your problem, though.