Avoid "Not Responding" when adding rows to QTableWidget

hbatalha

I have a QTableWidget that after I add a new row I also add a new row to a database table. Because of this when I add a large number of rows to QTableWidget to at once the app becomes unresponsive.
I want to have something like a loading spinner while the operations occurs. I tried using this spinner widget but it does not work as expected.

A possible solution I can think of is to add the data to the database after I add the rows to QTableWidget and doing it in a another thread, perhaps using QtConcurrent::run

JonB

@hbatalha
What sort of "large number of rows" in one go? How long is it unresponsive for? What database backend are you using, and where is the database housed? Are the rows you are adding to a QTableWidget the actual rows for the database? In which case why use a QTableWidget rather than a QTableView and (hopefully) bound to a QSql... model?

hbatalha

@JonB said in Avoid "Not Responding" when adding rows to QTableWidget:

What sort of "large number of rows" in one go? How long is it unresponsive for?

It starts becoming unresponsive after the 200th row.

What database backend are you using, and where is the database housed?

I am using SQLite, stored in a file in my computer.

Are the rows you are adding to a QTableWidget the actual rows for the database?

Sort of, the rows for the database contain more columns.

In which case why use a QTableWidget rather than a QTableView and (hopefully) bound to a QSql... model?

Because when I first started using Qt and I need a table so the first one I found was QTableWidget, so I didn't feel the need to switch to QTableView. And I only recently started using Sql.

JonB

@hbatalha
As it stands now, you must be copying the results between the QTableWidget and your SQL queries a lot. Up to you. I can't say whether that might be contributing to the time delay. I am "surprised" that 200 row insertions into a local SQlite file cause a "Not Responding", but I have no experience to say otherwise.

You can wait for someone to answer about putting up an "hour-glass" cursor.

I don't know whether showing a QProgressBar or QProgressDialog would satisfy you and deal with the "not responding"?

You could go down the other-thread route. My own advice/preference is to steer clear of threads as long as you can. Once you start doing database in another thread for this case, you need to do all other database things in that same thread....

You might use a QTimer to run the insertions in "batches"of 20 or whatever instead.

You might consider why you are accumulating 200+ rows and inserting them at all at one time instead of as you go along. But I do not know your situation.

hbatalha

@JonB said in Avoid "Not Responding" when adding rows to QTableWidget:

You might consider why you are accumulating 200+ rows and inserting them at all at one time instead of as you go along. But I do not know your situation.

Actually it's pretty unlikely that the user will be adding 200 rows at once but I want to avoid the bad UX when that happens. Also my laptop is a gaming laptop so a user with a low specs pc might have a bad experience with lower number of insertions.

Worth noting that when I add a new row to the table I am also setting two cell widgets and that contributes greatly to the app unresponsiveness.

JonB

@hbatalha
If you are now committed to a QSqL... model, you might test out responsiveness if you attached a QTableView directly to it instead. QSqlTableModel also has support for various EditStrategys and submitting,

hbatalha

@JonB I found a nice solution in an open source project Shotcut
longuitask.h
longuitask.cpp

JonB

@hbatalha Yup that's a QProgressDialog.

hbatalha

@JonB yeah when you mentioned it earlier I remembered seeing it in the Shotcut.

fcarney

Maybe I am reading this wrong, but wouldn't it be appropriate to do this in batches?
Each batch can be called by a zero timer that processes a few at a time. It in turn calls a zero time timer to get the next batch processed on the next event loop cycle. So that app stays responsive and you don't have threads.

In this what longuitask does?

jeremy_k

@hbatalha said in Avoid "Not Responding" when adding rows to QTableWidget:

@JonB I found a nice solution in an open source project Shotcut
longuitask.h
longuitask.cpp

That doesn't look like a viable solution for this task. QWidget-derived classes should only be accessed from QApplication's thread. QtConcurrent::run will violate that requirement.

JonB

@jeremy_k
I didn't even look in the .h file, just the .cpp. No mention there of threads! :)

hbatalha

@fcarney said in Avoid "Not Responding" when adding rows to QTableWidget:

In this what longuitask does?

No, longuitask just avoids the app unresponsiveness.

Each batch can be called by a zero timer that processes a few at a time. It in turn calls a zero time timer to get the next batch processed on the next event loop cycle.

From what I am picturing this would be the same as longuitask regarding app responsiveness since longuitask successfully avoids the "Not Responding" problem mentioned in OP.

But if you think it's different could you provide a simple code sketch that demonstrates your idea.

hbatalha

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

That doesn't look like a viable solution for this task. QWidget-derived classes should only be accessed from QApplication's thread. QtConcurrent::run will violate that requirement.

Using QtConcurrent::run was just an idea, a likely last resort solution in case none better was found.

hbatalha

@JonB Actually in '.h' file there's QtConcurrent. I just noticed it because I was focusing in the '.cpp'.
.h

#ifndef LONGUITASK_H
#define LONGUITASK_H

#include <QFuture>
#include <QProgressDialog>
#include <QtConcurrent/QtConcurrent>

class LongUiTask : public QProgressDialog
{
public:
    explicit LongUiTask(QString title);
    ~LongUiTask();

    template <class Ret>
    Ret wait(QString text, const QFuture<Ret>& future)
    {
        setLabelText(text);
        setRange(0, 0);
        while (!future.isFinished()) {
            setValue(0);
            QCoreApplication::processEvents();
            QThread::msleep(100);
        }
        return future.result();
    }

    template <class Ret, class Func, class Arg>
    Ret runAsync(QString text, Func&& f, Arg&& arg)
    {
        QFuture<Ret> future = QtConcurrent::run(f, arg);
        return wait<Ret>(text, future);
    }

    void reportProgress(QString text, int value, int max);
    static void cancel();
};

#endif // LONGUITASK_H

But I don't really understand how the wait and runAsync play a role in reporting progress.

jeremy_k

@JonB said in Avoid "Not Responding" when adding rows to QTableWidget:

@jeremy_k
I didn't even look in the .h file, just the .cpp. No mention there of threads! :)

I usually take the opposite approach, and look at the header for structure first.

In this case the .cpp doesn't mention threads, but also doesn't have anything that would help with a long running function call.

jeremy_k

@hbatalha said in Avoid "Not Responding" when adding rows to QTableWidget:

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

That doesn't look like a viable solution for this task. QWidget-derived classes should only be accessed from QApplication's thread. QtConcurrent::run will violate that requirement.

Using QtConcurrent::run was just an idea, a likely last resort solution in case none better was found.

I misread the problem description.

Using QtConcurrent to add rows to a database sounds fine, as long as it doesn't read the rows from the QTableWidget. It may be necessary to serialize the QtConcurrent tasks.

hbatalha

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

...but also doesn't have anything that would help with a long running function call.

It actually helps as it just solved my problem described in the OP.

    LongUiTask longTask(tr("Adding downloads"));
    for(int i = 0, len = titlesList.size(); i < len; ++i)
    {
        longTask.reportProgress(tr("Adding"), i, len);

// adding rows to the QTableWidget
    }

If I remove it the app becomes unresponsive starting from a certain number of rows.

hbatalha

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

@hbatalha said in Avoid "Not Responding" when adding rows to QTableWidget:

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

That doesn't look like a viable solution for this task. QWidget-derived classes should only be accessed from QApplication's thread. QtConcurrent::run will violate that requirement.

Using QtConcurrent::run was just an idea, a likely last resort solution in case none better was found.

I misread the problem description.

Using QtConcurrent to add rows to a database sounds fine, as long as it doesn't read the rows from the QTableWidget. It may be necessary to serialize the QtConcurrent tasks.

Probably but I will be using longuitask for now because aside from adding rows to database I still have two operations when adding rows to QTableWidget that I still haven't found a solution:
Add widget right aligned to a QTableWidget cell and
Add QPushButton to QTableWidget cell without using QTableWidget::setCellWidget

Edit: these two operations are very slow.

jeremy_k

@hbatalha said in Avoid "Not Responding" when adding rows to QTableWidget:

@jeremy_k said in Avoid "Not Responding" when adding rows to QTableWidget:

...but also doesn't have anything that would help with a long running function call.

It actually helps as it just solved my problem described in the OP.
    for(int i = 0, len = titlesList.size(); i < len; ++i)
    {
        longTask.reportProgress(tr("Adding"), i, len);

If I remove it the app becomes unresponsive starting from a certain number of rows.

LongUiTask::reportProgress() spins the event loop via QCoreApplication::processEvents(). It is making your app responsive because the function that it is called within doesn't return to the event loop quickly. It won't do anything for individual function calls within the loop. The preferred solution is to not do this, and instead return to the event loop.

If a large volume of records need to be processed, @fcarney's suggestion of using a zero timer works well.