Infinite loop and signal deluge



  • I have a problem with a loop in my code.
    Basically I do this in a thread:
    @while( !stop )
    {
    textResult = someLongProcessing(); // this takes ~50ms
    emit displayText(textResult);
    }@ The signal is connected to the setValue() slot of a QLabel, so nothing really time consuming.

    The problem is that sometimes someLongProcessing() returns immediately, causing the loop to emit a lot of signals.
    My ram usage then increases a lot and the software crashes.

    I can't call processEvents() because I'm not in the main thread. Then what should I do?
    I can not handle the issue slot-side, since it's a qt slot.
    If I add a sleep(5) in the loop it's better, but how would I know that the computer it's running on will need 5 or 10ms or even more to handle the setValue()...?

    Thanks a lot for the help.

    • EDIT: I just found about Qt::BlockingQueuedConnection, could this be a good idea?


  • I would simply make sure that you don't emit your signal if it should not emit. Best place to stop such an overflow it as the source...

    Is there some way for your process to know if the result is new or updated or something like that? If not, you could use a technique like "this":http://developer.qt.nokia.com/wiki/Delay_action_to_wait_for_user_interaction to throttle your signal emissions, I guess.



  • [quote author="Andre" date="1304344127"]I would simply make sure that you don't emit your signal if it should not emit.[/quote]In fact I always want to emit this signal. I "rephrased" my question in the first post.



  • maybe this?

    @while ( !stop ){

    bool finished = false;
    finished = someLongProcessing(*textResult); // if you finshed parsing your text return true
    if(finished)
    emit displayText(textResult);
    }@



  • I don't get it. With your finished boolean you just turn someLongProcessing() into a blocking call. But it is already blocking.



  • Sorry forgot the stop = true; line.

    @while ( !stop ){

    bool finished = false;
    finished = someLongProcessing(*textResult); // if you finshed parsing your text return true
    if(finished){
    emit displayText(textResult);
    stop = true;
    }
    }@



  • This will break the loop. I don't want to break the loop when someLongProcessing() returns true. You obviously did not understood my problem.



  • It is generally only useful to update a text if it has actually changed, so assuming that some successive calls may produce the same text result:
    @QString previousText;
    while (true) {
    QString text = someLongProcessing();
    if (previousText != text) {
    previousText = text;
    emit displayText(text);
    }
    }@



  • I'm sorry but successive calls wont produce the same result.
    There's nothing to do with the event queue?



  • QApplication::processEvents() helps?



  • Hi neFast,

    is this loop executed in a thread?
    If yes, processEvents will never help.



  • Yes it is in a thread. What do you suggest?



  • From that thread you have no influence on the main thread directly by calling processEvents etc. If you fill up the queue of one thread by another one, you have to think, whether it is the correct way for doing so.

    Perhaps you should change the logic to something like this:

    thread --> setText on intermediatObject --> emit signal

    intermediatObject is a thread save object, which stores the last text and stores, whether the text was already read. If a text is set, it emits the signal, if after the last read there came no new text.

    But that changes the logic completely...



  • Another consideration is that the text might change quickly enough for the user not even to notice it. It would probably make sense to not let the longProcessThingy() control the text the user sees unless there's an error of sorts. The current approach seems indeed too naive for successful operation.



  • Sorry to bump this thread, but are there any best practice that I could find to solve my problem?

    • EDIT: I just found about Qt::BlockingQueuedConnection, could this be a good idea?


  • If your thread can wait, sure, try that.



  • I'm affraid it will affect performances. I'll try that and report here.



  • It will slow down your thread somewhat, yes. If the thread cannot wait, you should probably go for Gerolf's solution or something like it.



  • [quote author="Franzk" date="1306420801"]Gerolf's solution or something like it.[/quote] Any details on this?
    Gerolf method could work with text (because it does not always change), but I also emit pointer to images. And after each loop the image changes, so I need to either:

    • wait for the event to be processed (in a performance friendly fashion)
    • trash events if too many of them are fired

    I cannot use this "emit only if required" idea, because I can not decide if it's required based on the content of the image.



  • Hmno, that was Andre's suggestion. Read "Gerolf's suggestion":https://developer.qt.nokia.com/forums/viewreply/33804/ again.



  • Yes your right, I will end up doing something like this if I don't find anything simpler.
    I'm now considering this:
    adding a new emit processEventsInMain() in the loop,
    and connect this signal in a blocking way to a slot of my main thread that will just consist in a processEvents() call.



  • Nonono, bad idea. the queued signal will be called if the main thread enters the event loop, so that will not solve the problem, you will get others. Opening the event loop is not a good idea, you get asynchronous behavior where you don't expect it.

    What is wrong with the intermediate object? That also works with any kind of data. Simple but powerfull pattern:

    The object stores the current data (or perhaps the current and the last). It has a member bNewDataEmitted. If this value is false, a dataChanged signal is emitted and the value is set to true. On each consecutive call of set data, no signal is emitted until the value goes to false.

    Each getData sets the value to false, so a dataChanged can be emitted on next setData.



  • [quote author="Gerolf" date="1306432144"]Nonono, bad idea. the queued signal will be called if the main thread enters the event loop, so that will not solve the problem, you will get others.[/quote] Even with a Qt::BlockingQueuedConnection?



  • Yes. BlockingQueuedConnection just waits until the event loop has finished executing the slot. If the slot is starting an event loop, you are getting weird results. Different example, but same case: http://labs.qt.nokia.com/2010/02/23/unpredictable-exec/.



  • Is there really no way to know if you need to act on the result of your longProcess or not? So far, it is not very clear what your long process entains. You started out with a string (I guess as an example), and you are talking about pointers to images. However, you also said that sometimes your longProcess returns very very quick. Does it still return valid and new data if that happens?



  • My veryLongProcess() performs segmentation on an image.
    If it work the result is an image with an overlay, if it fails it returns the original image.
    In both cases it will also result a string containing some timers and error codes.

    The original image comes from a live camera, so the output should be as real-time as possible.



  • So... couldn't you parse the error codes to see if you need to emit, and if so, what?



  • I always need to emit, even if it's an error I will emit the original image, the timers and the error code ...



  • Then, sorry, I don't know how to optimize further. I would say that at most, you'd need to emit a signal every 1/60th of a second (and probably half that or even only every 1/24th of a second), but if you say that you need to draw everything, then I believe you.

    I think that if you really need to pass information that fast, that threads may not be the ideal way to do it. It also depends on how much time processing the information at the other end takes. I mean: it doesn't make much sense to send a load of new images to display as a video if the system is still busy with drawing the first image. That will only clog the event queue, which will result in even more delays.



  • I tried to implement Andre's suggestion:
    Since I can send many signals I added a int _sentSignals to my class:
    @
    private:
    QMutex canEmitMutex;
    int _sentSignals; //!< Safe-guard to avoid signal deluge
    bool canEmit() {return _sentSignals<=0;}

    public:
    void signalProcessed();
    void signalEmited();

    // cpp
    void ProcessingThread::signalProcessed()
    {
    canEmitMutex.lock();
    _sentSignals--;
    canEmitMutex.unlock();
    }

    void ProcessingThread::signalEmited()
    {
    canEmitMutex.lock();
    _sentSignals++;
    canEmitMutex.unlock();
    }@

    And now in my processing loop I have
    @
    while( someCondition )
    {
    iCanEmit = canEmit();

    //code
    if( iCanEmit )
    {
    emit someSignal();
    signalEmited();
    }
    //repeat for other signals
    }@

    someSignal is connected to a slot in my main:
    @
    void MainWindow::slotDisplayStatusMsg(QString text)
    {
    ui->statusLabel->setText(text);
    // Tell processing thread that we processed its signal
    if( _procThread ) _procThread->signalProcessed();
    }@

    The problem is that, sometimes, some slot seems to be not executed. Hence _signalSent is not decremented and stays to 1. And my loop stop emiting.
    Is it possible that some events are sometimes ignored?



  • no, it can't.
    but between canEmit and signalEmitted in your while loop, there is a whole, where your raise condition may happen.
    So I suggets doing it like this:

    @
    class ProcessingThread
    {
    private:
    QMutex canEmitMutex;
    int _sentSignals; //!< Safe-guard to avoid signal deluge
    bool canEmit() {return _sentSignals<=0;}
    // some more stuff

    public:
    // some more stuff
    void signalProcessed();
    };

    void ProcessingThread::signalProcessed()
    {
    QMutexLocker lockObj(&canEmitMutex);
    _sentSignals--;
    }

    void ProcessingThread::run()
    {
    // do some stuff
    while( someCondition )
    {
    // emit lock
    {
    QMutexLocker lockObj(&canEmitMutex);
    if(canEmit())
    {
    emit someSignal();
    _sentSignals++;
    }
    }
    }
    // do some stuff
    }
    @

    EDIT: corrected code, Gerolf



  • Thanks for your help. Will try that tomorrow.
    Hum, in your example I guess I should replace:
    @void ProcessingThread::signalEmited()
    {
    QMutexLocker lockObj(&canEmitMutex);
    _sentSignals++;
    }@ with
    @void ProcessingThread::signalProcessed()
    {
    QMutexLocker lockObj(&canEmitMutex);
    _sentSignals--;
    }@



  • perhaps, yes, it was late :-)



  • You could even use a boolean variable, as you only emit once....



  • No in fact I emit more than once, I just simplified my sample code.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.