QTableView memory usage

Andeol

I am discovering that QTableView uses some memory depending on the number of rows in the associated model. Even if only a small portion of those rows are displayed.

To test that, I used the dummy model as follows :

class DummyModel : public QAbstractTableModel {

        Q_OBJECT;
    public:
        DummyModel(int nbLine, QObject *parent = 0) : QAbstractTableModel(parent)
            , m_nbLine(nbLine) 
        {};

        virtual int rowCount(const QModelIndex &parent = QModelIndex()) const override
        {
            return m_nbLine;
        }

        virtual int columnCount(const QModelIndex &parent = QModelIndex()) const override
        {
            return 5;
        }

        virtual QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override
        {
            if (!index.isValid()) return QVariant();        // if index is invalid, return empty QVariant

            switch (role)
            {
            case Qt::DisplayRole:
            case Qt::EditRole: {
                return "test";
            }
            }

            // return default for a table widget item
            return QTableWidgetItem().data(role);
        }

    private: 
        int m_nbLine;
    };

And then I simply observe the memory usage of my application before and after executing the following lines :

DummyModel* testModel = new DummyModel(10*1000000);
QTableView* testTable = new QTableView(this);
testTable->setModel(testModel);

Here, for 10 million lines, the memory usage jumps by 128Mo, the moment setModel is called. That jump in memory seems to be proportional to the number of lines.

Is there any way to work around this memory use, or should I completely give up the use of QTableView for large tables ?

Christian Ehrlicher

@JonB said in QTableView memory usage:

I assume the table view holds a pointer or so for each row in the model

Nearly - it has to maintain e.g. one section item per row (4 byte * 10mio) and others

btw: return QTableWidgetItem().data(role); ? Useless, simply return a QVariant()

JonB

@Andeol
I assume the table view holds a pointer or so for each row in the model. Are you intending the QTableView to display 10 million rows to the user? (And, are you intending the model to hold 10 million rows in memory, how does that compare to table view memory usage of 128MB when you have read in all the rows?) If not, why not interpose a proxy model or similar between large model and view to reduce what is shown/implement a "paging" mechanism?

Christian Ehrlicher

You have to live with it - but as always with those high numbers. Do you really expect that someone is seeing something useful when you display 10 mio lines?

Christian Ehrlicher

@JonB said in QTableView memory usage:

I assume the table view holds a pointer or so for each row in the model

Nearly - it has to maintain e.g. one section item per row (4 byte * 10mio) and others

btw: return QTableWidgetItem().data(role); ? Useless, simply return a QVariant()

artwaw

@Andeol why not switch to sqlite backend and use QSqlTableModel (or descendant if you need some extra features), which already has lazy population built-in?

Andeol

@artwaw We might end up doing that. But note that the issue is not really with the model part. We already have all we need to efficiently access the data on-demand. Typically here in my example, the model is super-light. It's the view that takes all this memory, even before making a single call to data().

Otherwise, as JonB suggested, we are probably going to limit the number of rows in the actual table, by using a proxy. I'm not personally expecting the user to see something useful by scrolling such data, but I'm not the one making the specifications. The goal until now was to be able to scroll in the full table if the user so desires. I'll have to negociate that. Or we may do our own view widget, if it appears QTableView is not suitable for our needs.

artwaw

@Andeol It is my impression that using a model with lazy population will not force the view to grow so much, as it will see only 256 items at the time as this is what lazy population does?
Of course, I might be wrong.

JonB

@artwaw
I am assuming that the overhead in the QTableView is not to do with records have been read in. Rather, it is proportionate to what model->rowCount() returns. I too may be wrong :)

Andeol

@JonB Considering that the spike in memory use happens before any call to data() is done, but just after the first call to rowCount(), I think you're right.