Unsolved QStandardItemModel.clear() is very slow (60 sec or more)
-
I have a
QTreeView
that is using aQStandardItemModel
. When the user selects a category (via a combo box), the script loads data from a REST API (running locally; this request usually takes less than 0.1 seconds for a large payload) and populates the model with anywhere from 1,000 to 10,000 rows (and ~10 columns per row) of data. Here's the method (slightly stripped-down) that takes the data from the REST call (parsed into ashotlist
list andheaders
list), and loads it into the model/treeview:def setShotList(self, shotlist, headers): self.tree_alldailies.setSortingEnabled(False) print("Sorting is enabled?", self.tree_alldailies.isSortingEnabled()) print("Clearing model") self.model_dailies.clear() print("Adding headers") self.model_dailies.setHorizontalHeaderLabels(headers) item_root = self.model_dailies.invisibleRootItem() print("Adding items") for shot in shotlist: row = [str(shot.metadata.get(x,"")) for x in headers] item_root.appendRow([QtGui.QStandardItem(x) for x in row]) print("Done adding items") self.lbl_summary.setText(f"Showing all {self.model_dailies.rowCount():,} shots") self.tree_alldailies.resizeColumnToContents(0) self.tree_alldailies.setSortingEnabled(True) print("Sorting is enabled?", self.tree_alldailies.isSortingEnabled()) print("Done")
When the program runs this the first time, the entire thing from program launch to data loaded takes about 1.2 seconds to execute, which is completely acceptable to me, given the large amount of data. But once the user chooses a different category and this method is called again, the delay can be around 1-2 minutes(!) before the old data is cleared out and the new data is loaded in.
I put those
print()
functions in to track it down, and the delay seems to be coming from theQStandardItemModel.clear()
method. Indeed, I tried commenting out the.clear()
line, so that data just keeps getting added to the model without the old stuff being removed, and I was back down to <2 seconds execution time once again, even when the number of rows was growing to be 50,000.I also tried just unsetting the data model using
self.tree_alldailies.setModel(None)
at the beginning of the method, then loading the new data into a newQStandardItemModel
and setting the treeview to the new one after the new data is loaded withself.tree_alldailies.setModel(theNewItemModel)
, but that doesn't help at all; presumably because in the background, the old item model calls.clear()
when it's destroyed or something.The delay also occurs when I close the main window of the application, presumably because of the same reason.
Am I doing something wrong? How can I help
.clear()
execute faster?I understand I'm filling it with a lot of data, but every other operation (even loading the data in) goes literally orders of magnitude faster.
-
@mjiggidy
I'm afraid it looks like you are not alone in this....
https://forum.qt.io/topic/104647/it-s-extremely-slow-to-remove-all-rows-from-qtableview
https://www.qtcentre.org/threads/65616-Slow-performance-Removing-huge-amount-of-rows-from-qstandarditemmodelDo make sure you are not getting multiple signals, or disconnect any signals, if that helps.
All I can suggest is you profile the performance and see whether there is anything there you can do anything about. Maybe, for all I know,
QStandardItemModel
is a poor choice, and you would be better with your own model? -
Thank you for your reply. I have tried setting
blockSignals(True)
on both the data model and the tree view before clearing rows (experimentally, with the understanding that it's not a great thing to do), and I see no difference. Also triedmodel.beginResetModel()
andendResetModel()
around theclear()
method, and no difference there either.I have also tried this amalgamation just to try to dig into the problem:
self.model_dailies.blockSignals(True) self.tree_alldailies.blockSignals(True) for row in range(self.model_dailies.rowCount()): print("Removing", row) self.model_dailies.removeRow(row) self.tree_alldailies.blockSignals(False) self.model_dailies.blockSignals(False)
From the print statement there, I can see that it starts out pretty quickly (removing approx 100 rows/second) then around 400-500 rows in, it slows down gradually to about 1 per second for the next ~500, and then fluctuates from there. This is true for any data that is being loaded/unloaded.
What really gets me is that I'm re-writing this program from a version I wrote with tkinter, and in the tkinter program, switching between the exact same data sets in the treeview is instantaneous even with the REST call.
I am admittedly very new to PySide2 and I'm sure I have much to learn (like implementing my own model from QAbstractItemModel), but I don't understand why merely adding text rows of data with Qt's own built-in QStandardItemModel yields such terrible performance!