Building multi progress bar & getting slowdown.

Dariusz

Hey

I figuret it. It was the QMap[key]->setValue() search/ hash tables I suppose or something. I changed my keys to ints and use QVector[key]->setValue(). that speeded up everything to nearly as fast as with no gui updated. I guess I just needed to post it to re-think it. TIA in advance.

Leason to learn, if you gonna run 100mil loop dont use Qmap, use either QHash or vector[index] to find data.

mrjj

Ok super :)
yeah for some use cases / data set QHash will be faster.
Sadly it soften hard to find out thats the real issue.

Dariusz

Ok I must have been high or I managed to break it again. But its slow again! I made a simple test to show the case:

https://ufile.io/7gqey
weetransfer:
https://we.tl/3OewiuUJ9G

mrjj

Hi
Such massive loops will strangulate the app
and to have any effect qApp->processEvents(); should
be used inside the loops.
However loops not optimal and cause issue as u see,
and Qtimer or other method should be used. ( that also fixes the slowdown)

mainWindow::mainWindow() {

    myBars = new multiProgressBar();
    myBars->show();
    qApp->processEvents();

    for (int x = 0; x < 500000; ++x) {
        myBars->setVal(0, x);
   qApp->processEvents();
    }
    for (int x = 1; x < 500000; ++x) {
        myBars->setVal(0, x);
    qApp->processEvents();
    }
    for (int x = 2; x < 500000; ++x) {
        myBars->setVal(0, x);
...
    }
    for (int x = 3; x < 500000; ++x) {
        myBars->setVal(0, x);
...
    }
}

If you do like this, its pretty fast

#include <QTimer>
#include <QDebug>

int main(int argc, char* argv[]) {

  QApplication a(argc, argv);

  multiProgressBar* myBars;
  myBars = new multiProgressBar();
  myBars->show();
  // this is sign something not good with design -> qApp->processEvents();

  //Instead of loops that blocks the app, we use the signal processing.
  QTimer* timer = new QTimer;
  QObject::connect(qApp, &QApplication::aboutToQuit, timer, &QTimer::deleteLater); // to delete the timer.

  QObject::connect(timer, &QTimer::timeout, [myBars, timer]() {
    static int val = 0;
    myBars->setVal(0, val);
    myBars->setVal(1, val + 1);
    myBars->setVal(2, val + 5);
    myBars->setVal(3, val + 10);
    myBars->setVal(4, val + 15);
    qDebug() << "TIMER" << val;
    val++;
    if (val > 500 )   val = 0;
  });
  timer->start(1);
  return a.exec();
}

alt text

Dariusz

Hey

Humh that's interesting approach with using a timer. But even so, when I use a timer to loop over the 500k items it's still slow as hell. Or am I missing something?

It don't block the application, but the operation now is limited to qtimer ticks?

I tried q progress dialog as well but that is slow too.

Perhaps I need to write some kind of a system that if sees values being changed in less than ms from last change, I should ignore the value and pass every 10nth value or something like that... So that I call update on GUI every 10 items or every 50 items to not slow down the GUI updates. Or perhaps I should run it in a separate thread the calculation and then just emit a signal and let QT handle update signals in main thread himself humh...

#include <QTimer>
#include <QDebug>
#include "mainWindow.h"

#include <QProgressDialog>

int main(int argc, char* argv[]) {

    QApplication a(argc, argv);

    multiProgressBar* myBars;
    myBars = new multiProgressBar();
    myBars->show();

    QProgressDialog *dial = new QProgressDialog();
    dial->setMaximum(500000);
    // this is sign something not good with design -> qApp->processEvents();

    //Instead of loops that blocks the app, we use the signal processing.
    qDebug()<<"im being called, id";
    int b=0;
    for (int x=0;x<500000;++x){
        dial->setValue(x);
        b++;
        qApp->processEvents();
    }
    qDebug()<<"im being called, id";



    QTimer* timer = new QTimer;
    QObject::connect(qApp, &QApplication::aboutToQuit, timer, &QTimer::deleteLater); // to delete the timer.


    QObject::connect(timer, &QTimer::timeout, [myBars, timer]() {
        static int val = 0;
        myBars->setVal(0, val);
        myBars->setVal(1, val + 1);
        myBars->setVal(2, val + 5);
        myBars->setVal(3, val + 10);
        myBars->setVal(4, val + 15);
        //qDebug() << "TIMER" << val;
        val++;
        if (val > 500000 )   timer->stop();
        if (val > 500000 )   val = 0;
    });
    timer->start();
    return a.exec();
}

Dariusz

Here is my test using concurrent:
https://we.tl/vMNEGbjL9a

The loop/calculation is very fast now, but the GUI update is super slow as it has 500k signals to process I think. I recon I have to come up with a way to bulk up the signal/progress setValue emits.

Ok I think I may have got it...
Simple test, will need to drop it in main app see how it works :- )

    time_t seconds = time(NULL);
    std::cout << seconds << "  " << time(NULL) << std::endl;
    for (int x = 0; x < 500000; ++x) {
        delay(); // delays 50ms
        if (time(NULL) - seconds > 1) { // if last update was more than 1 sec ago
            myBars->setVal(0, x);
            seconds = time(NULL);
        }
    }

Found even better way > http://www.cplusplus.com/reference/chrono/steady_clock/

This way I can control in float range. from 0.01- to whatever I want. So if I have long calculation I can use 0.1 as timeout, but if I have fast calcs and I need to "bulk" up the update then I can use 1-2 seconds etc. Did a test seems to work great - yay :D

t1 and t2 are declared in header:

    t2 = steady_clock::now();
    duration<double> time_span = duration_cast<duration<double>>(t2 - t1);
    if (time_span.count() > 2.0) {
        _slaveBars[key]->setValue(value);
        t1 = t2;
        qApp->processEvents();
    }

mrjj

Hi
Just to be sure we are clear.
The a.exec(); runs a message loop that make it possible for the application
to draw on screen and receive user input.
If you make huge loops of any kind,
then this exec() do not get a chance to run and the app
stops redrawing and its not possible to interface with it.

So it has nothing to do with the very little time it takes to call slaveBars[key]->setValue
or that it calls the paintEvent to draw the new value. Its simply how your codes does it that makes it work bad.

If you do anything like that in a loop it will simply run like shit.
Its not a performance issue. simply just bad design for
a event driven application.

If you need long lasting operations, you should move them to a thread and use signals to update info in mainwindow.

but yes, calling qApp->processEvents(); in the tight loops , often makes it work ok.
But its really not the best design and might not always be the cure.

Its always better to use the signal slot system and timers to repeat a task than to use for loops.

Dariusz

Hey

Thanks for info !

In the last post I have used QtConcurrent, that should run in separate thread/app loop yes? I also connected signal/slot to main Qt app to update the qbars. Is that what you mean when it comes to make a "good design"? Because that calc runs intantly. But QT gets 500k signals which then it slowly process, or did I mess up something there?

I attached weetransfer file too.

mrjj

Hi
Yes QtConcurrent would be a fine way to offload
processing from main thread.

Im not sure how the 500.000 signals comes into play but normally not an issue unless you made them with loops so it had no chance to process them.

I attached weetransfer file too.
To last post or ?

Dariusz

@mrjj https://we.tl/vMNEGbjL9a here. It runs calculation in concurrent finished in like 1 second, but then QT takes ages to update/process the signals the concurrent emitted.

mrjj

Hi
you spam the message loop as its not really doing anything.

for (int x=0;x<500000;++x){          
        emit updateX(x);        
    }

It does count as expected here ( as fast as it can) but
on other pc it might be too massive for it.

Normally the calculation takes time and only updates from time to time.

jsulm

@Dariusz To add to @mrjj : it doesn't really make sense to update the UI 500000 times! Such a bar goes from 0% to 100%, so you only need 100 updates. You should optimise your app emitting the signal only for each 1% step.

mrjj

@jsulm
hi
he does call
http://doc.qt.io/qt-5/qprogressbar.html#maximum-prop
to set it to 50.000.
The main concern is he feels it lags/do slow drawing the full range.