Large set of data in QTableView

FroZtiZ

Thank you for your contributions. I found how to proceed by storing my data in a QList<QList<float>>.
Please find below a working code that can probably be improved:

CPP FILE


#include "filelistmodel.h"

#include <QGuiApplication>
#include <QDir>
#include <QPalette>
#include "qdebug.h"

FileListModel::FileListModel(QObject *parent)
    : QAbstractTableModel(parent), rowNumber(0)
{}

int FileListModel::rowCount(const QModelIndex &parent) const
{
    return parent.isValid() ? 0 : rowNumber;
}

int FileListModel::columnCount(const QModelIndex &parent) const
{
    return parent.isValid() ? 0 : colNumber;
}

void FileListModel::setColumnNumber(const int x)
{
    colNumber = x;
}

QVariant FileListModel::data(const QModelIndex &index, int role) const
{
    if (!index.isValid())
    {
        return QVariant();
    }
    if (index.row() >= fileList[0].size() || index.row() < 0)
    {
        return QVariant();
    }
    if (role == Qt::DisplayRole)
    {
        return fileList[index.column()][index.row()];
    }
    return QVariant();
}

bool FileListModel::canFetchMore(const QModelIndex &parent) const
{
    if (parent.isValid())
    {
        return false;
    }
    return (rowNumber < fileList[0].size());
}

void FileListModel::fetchMore(const QModelIndex &parent)
{
    if (parent.isValid())
    {
        return;
    }
    int remainder = fileList[0].size() - rowNumber;
    int itemsToFetch = qMin(100, remainder);

    if (itemsToFetch <= 0)
    {
        return;
    }
    beginInsertRows(QModelIndex(), rowNumber, rowNumber + itemsToFetch - 1);

    rowNumber += itemsToFetch;

    endInsertRows();
}

void FileListModel::setDataToTable(const QList<QList<float>> &data)
{
    beginResetModel();
    fileList = data;
    rowNumber = 0;
    endResetModel();
}

H FILE


#ifndef FILELISTMODEL_H
#define FILELISTMODEL_H

#include <QAbstractListModel>
#include <QStringList>

class FileListModel : public QAbstractTableModel
{
    Q_OBJECT

public:
    FileListModel(QObject *parent = nullptr);

    int rowCount(const QModelIndex &parent = QModelIndex()) const override;
    int columnCount(const QModelIndex &parent = QModelIndex()) const override;
    QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
    void setColumnNumber(const int );
    void setDataToTable(const QList<QList<float>>&);

protected:
    bool canFetchMore(const QModelIndex &parent) const override;
    void fetchMore(const QModelIndex &parent) override;

private:
    QList<QList<float>> fileList;
    int rowNumber;
    int colNumber;
};

#endif // FILELISTMODEL_H

BUILDING MODEL

FileListModel *pModel =new FileListModel(this);
QList<QList<float>> a;
pModel->setColumnNumber(5);
a.append(qlist1);
a.append(qlist2);
a.append(qlist3);
a.append(qlist4);
a.append(qlist5);
pModel->setDataToTable(a);

Christian Ehrlicher

You're still using the wrong container. You want to use a vector, not a list. Your fetchMore() is trivial - I doubt this is your real workcase, or?

FroZtiZ

Why should I use a QVector instead of a QList?
My fetch is maybe trivial but it is what I was trying to do. I am not a QT expert, especially, I dont know at all the abstract classes.

Christian Ehrlicher

@FroZtiZ said in Large set of data in QTableView:

Why should I use a QVector instead of a QList?

Because QList is a list (more or less) and QVector a vector. See also the QList documentation - since you're using floats, the memory needed for QList<float> is exactly double the memory needed for QVector<float> on 64 bit systems.

FroZtiZ

Thanks I will perform the modification

JonB

@FroZtiZ
Excuse my chiming in again. I have to confess I am lost as to whether you are now asking about which model to use, columns vs rows, efficiency of adding, or ...

@FroZtiZ about 18 hours ago

Minimum 5 columns of 20 000 rows

@Christian-Ehrlicher
20k items is not that larger that lazy loading is needed I would say

First, always heed what @Christian-Ehrlicher asks/suggests, as he knows more than I. However, I am slightly surprised, I would have said 20k (x 5 columns) is quite large. I don't know what your "very very slow" actually means. But my thought would be: if you are presenting a UI like a QTableWidget or whatever, why would you want 20k items in it? The user cannot handle that many items to scroll through....

FroZtiZ

I am doing what Christian suggested. And now, with the last modifications, everything is working. So I am not asking anymore about how to proceed with columns, rows or model.
When I say very very slow, I mean I need ~5 seconds to update/construct/destroy the table.
I agree that 100k items is quite large for a user to read everything. In fact, these items are spread in different tables that represent x,y coordinates of a function. For sure the user won't read everything simultaneously. However, I need to be able to provide the data on request inside the user interface.

JonB

@FroZtiZ
OK. Sorry if I am supposed to understand this from the foregoing, but I don't understand where your "~5 seconds to update/construct/destroy the table" is coming from? You have implemented a fetchMore() which only fetches 100 at a time, so that should be good. Is it being called repeatedly to read all 20/100k? Is there any overhead in physically getting the items? I don't understand where your 5 seconds is being spent? You need to discover that.

FroZtiZ

@FroZtiZ said in Large set of data in QTableView:

The construction and destruction of QStandardItem in a QTableWidget is very slow. That is the reason why I decided to construct a QAbstractTableModel.

My code was slow to execute when I was using a QTableWidget with QStandardItem.

@FroZtiZ said in Large set of data in QTableView:

The current model that is described in my code above is very fast so, it seems to be the solution. Inside, I want to put 5 QList, each being a column of this model.

@FroZtiZ said in Large set of data in QTableView:

Thank you for your contributions. I found how to proceed by storing my data in a QList<QList<float>>.
Please find below a working code that can probably be improved:

With the model I provided above, including "fetch", the problem is solved and the execution is fast.

Christian Ehrlicher

@FroZtiZ said in Large set of data in QTableView:

the problem is solved and the execution is fast.

Then please mark this topic as solved, thx.