Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

QStandardItemModel.clear() is very slow (60 sec or more)



  • I have a QTreeView that is using a QStandardItemModel. When the user selects a category (via a combo box), the script loads data from a REST API (running locally; this request usually takes less than 0.1 seconds for a large payload) and populates the model with anywhere from 1,000 to 10,000 rows (and ~10 columns per row) of data. Here's the method (slightly stripped-down) that takes the data from the REST call (parsed into a shotlist list and headers list), and loads it into the model/treeview:

    def setShotList(self, shotlist, headers):
    
        self.tree_alldailies.setSortingEnabled(False)
        print("Sorting is enabled?", self.tree_alldailies.isSortingEnabled())
    
        print("Clearing model")
        self.model_dailies.clear()
    
        print("Adding headers")
        self.model_dailies.setHorizontalHeaderLabels(headers)
    
        item_root = self.model_dailies.invisibleRootItem()
        print("Adding items")
        for shot in shotlist:
                row = [str(shot.metadata.get(x,"")) for x in headers]
                item_root.appendRow([QtGui.QStandardItem(x) for x in row])
    
        print("Done adding items")
        self.lbl_summary.setText(f"Showing all {self.model_dailies.rowCount():,} shots")
    
        self.tree_alldailies.resizeColumnToContents(0)
        self.tree_alldailies.setSortingEnabled(True)
        print("Sorting is enabled?", self.tree_alldailies.isSortingEnabled())
        print("Done")
    

    When the program runs this the first time, the entire thing from program launch to data loaded takes about 1.2 seconds to execute, which is completely acceptable to me, given the large amount of data. But once the user chooses a different category and this method is called again, the delay can be around 1-2 minutes(!) before the old data is cleared out and the new data is loaded in.

    I put those print() functions in to track it down, and the delay seems to be coming from the QStandardItemModel.clear() method. Indeed, I tried commenting out the .clear() line, so that data just keeps getting added to the model without the old stuff being removed, and I was back down to <2 seconds execution time once again, even when the number of rows was growing to be 50,000.

    I also tried just unsetting the data model using self.tree_alldailies.setModel(None) at the beginning of the method, then loading the new data into a new QStandardItemModel and setting the treeview to the new one after the new data is loaded with self.tree_alldailies.setModel(theNewItemModel), but that doesn't help at all; presumably because in the background, the old item model calls .clear() when it's destroyed or something.

    The delay also occurs when I close the main window of the application, presumably because of the same reason.

    Am I doing something wrong? How can I help .clear() execute faster?

    I understand I'm filling it with a lot of data, but every other operation (even loading the data in) goes literally orders of magnitude faster.



  • @mjiggidy
    I'm afraid it looks like you are not alone in this....
    https://forum.qt.io/topic/104647/it-s-extremely-slow-to-remove-all-rows-from-qtableview
    https://www.qtcentre.org/threads/65616-Slow-performance-Removing-huge-amount-of-rows-from-qstandarditemmodel

    Do make sure you are not getting multiple signals, or disconnect any signals, if that helps.

    All I can suggest is you profile the performance and see whether there is anything there you can do anything about. Maybe, for all I know, QStandardItemModel is a poor choice, and you would be better with your own model?



  • Thank you for your reply. I have tried setting blockSignals(True) on both the data model and the tree view before clearing rows (experimentally, with the understanding that it's not a great thing to do), and I see no difference. Also tried model.beginResetModel() and endResetModel() around the clear() method, and no difference there either.

    I have also tried this amalgamation just to try to dig into the problem:

    	self.model_dailies.blockSignals(True)
    	self.tree_alldailies.blockSignals(True)
    	for row in range(self.model_dailies.rowCount()):
    		print("Removing", row)
    		self.model_dailies.removeRow(row)
    	self.tree_alldailies.blockSignals(False)
    	self.model_dailies.blockSignals(False)
    

    From the print statement there, I can see that it starts out pretty quickly (removing approx 100 rows/second) then around 400-500 rows in, it slows down gradually to about 1 per second for the next ~500, and then fluctuates from there. This is true for any data that is being loaded/unloaded.

    What really gets me is that I'm re-writing this program from a version I wrote with tkinter, and in the tkinter program, switching between the exact same data sets in the treeview is instantaneous even with the REST call.

    I am admittedly very new to PySide2 and I'm sure I have much to learn (like implementing my own model from QAbstractItemModel), but I don't understand why merely adding text rows of data with Qt's own built-in QStandardItemModel yields such terrible performance!


Log in to reply