How can I speed up QStandardItemModel::clear() ?

Kill-Animals

In my particular table, I have 47 columns and 6768 rows. Calling .clear(); so I can repopulate it according to a filter, takes nearly 30 seconds, and I find that bizarre, and have to assume that there was something that I am doing that is causing such a lag.

Any ideas?

Christian Ehrlicher

Please provide a minimal, compilable example - clear() should not take that long (and I haven't heard about such an issue). Also you maybe switch over to an own custom model.

Kill-Animals

Took me awhile to track it down, but here is the offending member:

	QWidget w;
	QHBoxLayout hbl(&w);
	QTreeView tv;
	{
		hbl.addWidget(&tv);
	}
	QStringList headers;
	QStandardItemModel sim;
	for ( int i = 0; i < 8000; i++ ) {

		QList<QStandardItem *> sil;
		for ( int j = 0; j < 30; j++ ) {
			QStandardItem *si = new QStandardItem(QString::number(i+j) );
			sil <<  si;
		}
		sim.appendRow(sil);
	}
	sim.sort( 3, Qt::SortOrder::DescendingOrder ); // This line here greatly slows it down
	tv.setModel(&sim);

	w.resize(500,500);
	w.showNormal();
	QElapsedTimer et;
	et.start();
	qDebug() << "begin";
	sim.clear();
	qDebug() << "end" << et.elapsed();

On my computer, that one line changes it from 25, to 17002, aka taking 17 seconds to clear everything.

Kill-Animals

Kind of a hacky solution but what I ended up doing was making an additional column far right, naming them 0000 - 9000, and before calling clear, I would sort the table by that column, and it works extremely fast now.
I kind of think this is a bug that should be reported.

JonB

@Kill-Animals
QStandardItemModels are not for efficiency. The usual way to sort is to interpose a QSortFilterProxyModel between the model and the view. I don't know if you tried that whether performance would be better. You might read through e.g. https://forum.qt.io/topic/55272/qtreeview-sort-model-qstandarditemmodel-the-most-efficient-way-to-update-the-model for some observations.

It looks like, for whatever reason, it is re-sorting (or something) as items are cleared. I don't see clearly from the docs whether your sim.sort( 3, Qt::SortOrder::DescendingOrder) can be disabled, you might try sim.sort( -1, Qt::SortOrder::AscendingOrder) immediately above your sim.clear()?

artwaw

@Kill-Animals To add to @JonB note about QSortFilterProxyModel (which is a very sound advice) - I ditched standard item model in the favour of in-memory sqlite db + QSqlTableModel + sort filter proxy model for any kind of data stored in the table as apparently nothing gets faster than that combo.

mrjj

Hi
But it's a bit odd how much longer deallocation takes when sorted.

On my pc it takes
419198
versus
447
o.O

artwaw

@mrjj may I ask you to post your code? I'd like to test on my machine (xeon w, macOS).

JonB

@artwaw Can't you just test the code posted by @Kill-Animals earlier?

artwaw

@JonB @mrjj
I did just that.
31786 with sort
93 without.

JonB

@artwaw
Which just confirms what @Kill-Animals said! :)
Did sim.sort( -1, Qt::SortOrder::AscendingOrder) work/make any difference?

Kill-Animals

@JonB
I tried it, but the -1 value did not do anything for me. Instead I went with my hacky solution.
m_AnimeModelFilter.sort( m_AnimeModelFilter.columnCount() - 1, Qt::SortOrder::AscendingOrder );

Because I had put a hidden column with numbers ranging 0001 to 9999, and sorted by that row. Speed issues were more or less solved by that.
I will look into QSortFilterProxyModel, but it sounds like quite a bit of work.

@artwaw said in How can I speed up QStandardItemModel::clear() ?:

@Kill-Animals To add to @JonB note about QSortFilterProxyModel (which is a very sound advice) - I ditched standard item model in the favour of in-memory sqlite db + QSqlTableModel + sort filter proxy model for any kind of data stored in the table as apparently nothing gets faster than that combo.

I'll look into that. I just finished an sqlite library which replaces QSettings for me.

@JonB said in How can I speed up QStandardItemModel::clear() ?:

@artwaw
Which just confirms what @Kill-Animals said! :)
Did sim.sort( -1, Qt::SortOrder::AscendingOrder) work/make any difference?

No.

artwaw

@JonB
88 - so faster than even commenting out the sort O_O
EDIT - but not if original sort stays in place.

Kill-Animals

@artwaw said in How can I speed up QStandardItemModel::clear() ?:

@JonB
88 - so faster than even commenting out the sort O_O

It didn't work for me. It makes things way worse on my real world usage, where the first column is more or less the same data. Therefore I suspect that the -1 example will be dependent upon the type of data you keep in there.

JonB

@artwaw , @Kill-Animals
I just looked briefly at the source code on woboq. According to me, first test is if column to sort on less then 0 then return. So I don't know how one of you found it better and the other found it worse, I think it's a NO-OP! :)

This may be way off-target, but how does QStandardItemModel::setRowCount(0) behave instead of clear()? Does it do the same job of freeing the items, or does it leave stuff around?

artwaw

@JonB I will test in a moment.
I did an experiment, built a table with over 30 columns, table model, sort filter proxy model, and populated it with 10k rows.
Sorting and clear took 2ms.
I will test setRowCount(0) in a sec.

artwaw

@JonB 33124 for setRowCount(0); so it is actually slower a bit.

Kill-Animals

@JonB In my previous testing, I can attest that deleting the raw pointer ( and calling the destructor ) was arduously long. Setting the row count to 0 similarly took a long time.