@Daniella said in How to create a QTreeView that can handle very large amount of data?:
I'm adding data loaded form a file on disk to a QTreeView , the file contains more than 60.000 rows and new 'rows'/data will be constantly added to it while its being executed, so each time the GUI is reloaded/reopened the file will contain more data.
Is 60.000 records a lot of data to hold in memory? 60k integers is probably not a big deal for a currently produced desktop or smartphone level device. 60k 4K HDR movies is a different story.
I tried to load this data using another thread just to figure out that QAbstractItemModel is not thread safe.
Then I tried to created a 'lazy load' subclass of a QAbstractItemModel , that 'process' the items as they get visible in the
QTreeview.
At the beginning it worked great, as it doesn't freeze the entire GUI for like 10~ seconds at runtime anymore.
However when i scroll the QTreeView it get frozen for many seconds, i think its 'loading' the data.
Profile the execution. Otherwise you're likely engaging (and asking us to engage) in premature optimization.
I was reading the documentation of QAbstractItemModel and see that it have these two functions: fetchMore() and canFetchMore(), to control how the data is loaded, but i didn't understand how to use it and if they could help in this case.
The view calls these in situations where more data might be useful. Look at them as an opportunity to queue asynchronous loading.
if (view.scrollBar.value() == view.scrollBar.maximum())
if (canFetchMore())
fetchMore();
...
struct Model: public QAbstractItemModel {
int pageCount;
int pagesLoaded;
QNetworkManager netman;
Model() {
connect(&netman, &QNetworkManager::finished, this, &Model::addData);
}
bool canFetchMore(QModelIndex &parent) {
return pageCount > pagesLoaded;
}
void fetchMore(QModelIndex &parent) {
if (pageCount > pagesLoaded)
netman.get(QUrl(...));
}
};
How I could dynamically load/unload the data as it get visible in the QTreeView? and free the memory of the rows that are not visible/not visible anymore, instead of having the entire file loaded into the memory.
QAbstractItemView::indexAt() can help answer this question, but be careful. If the user scrolls and the view attempts to display what it thinks is available data, the model will need to either block in QAbstractItemModel::data() while retrieving it, or present dummy data to later be corrected with QAbstractItemMode::dataChanged(). Is unloading necessary?
void addData(const QVector<QStringList> &row_info)
{
beginInsertRows(QModelIndex(), m_rows.count(), m_rows.count() + row_info.count() - 1);
for (const auto &row : row_info)
m_rows.append(row);
endInsertRows();
}
Engaging in a little premature optimization, consider QList::append(QList&&) or QList::append::(QList &) instead of a for loop picking apart the list.
