How do I debug excessive QAbstractItemModel::data() callbacks and general extreme inefficiency (PyQt5)
I'm working in PyQt5 with a QAbstractItemModel which I just tidied up a bit using the PyQt model tester. I'm still getting lots of data() calls against TextAlignmentRole. for example, switching to the window with my QTreeView with three rows and two columns and then switching back away from it results in just over 100 calls for TextAlignmentRole. Over 700 total data() calls. Is it just me or does this seem like there is something wrong with my model? Any ideas on how to debug?
Why do I think this as opposed to assuming this is normal? Because I would have expected that qt would not need to repeatedly re-request updates when I had not notified it of any changes.
The big picture is that my tree view is presenting realtime data off of a CAN bus with 10 to 100 messages being received at about 10Hz each and possibly several signals within each. So, a good bit of data. Other basic applications manage to present the information at roughly 1% CPU load while mine is taking at least 15% and often much more. With more devices on the CAN bus some PCs are unable to keep up. I could certainly be wrong but I am assuming that this is caused by an error in my usage of QAbstractItemModel.
Thanks for any help learning more about this issue.
One thing that can be done when getting lots of data is to handle them in batch and trigger the update only after a batch has been processed.
In a minimal case I had three values changing (two visible) at 10Hz each and the application was taking 8% CPU load on a high end workstation laptop I bought late last year (Lenovo W550s). I understand that long term I may need to throttle the data presentation on the display but this just level of inefficiency seems crazy. That's why I am looking for issues with how I am using the QTreeView.
I looked around a bit more and found that even though there were no changes in width, I still had a few columns set to autosize and turning that off dropped the CPU usage in half. I had previously turned that off for the columns that were continuously jittering back and forth and replaced it with a routine that calculate the max length string and set the width based on that.
Another specific scenario I've found is that if I call dataChanged() for a top level item it takes significantly longer (perhaps 4x or so) than calling dataChanged() on it's children even though both immediately trigger a visual update of the top level item. This again points me towards an error in the way I am using the QTreeView. For now I have just added a single dummy child to all top level items and call dataChanged() on it to alleviate the extra CPU load. :[
I will continue to dig on finding my misuses first and hope I don't have to do too much throttling/resampling/batching of data. But, I may well need some.
Thanks for your response.
Another specific scenario I've found is that if I call dataChanged() for a top level item it takes significantly longer (perhaps 4x or so) than calling dataChanged() on it's children
Apparently one of my other fixes alleviated this. There no longer seems to be a difference in load between calling dataChanged() for just the children vs. for a top level item.
What did you change ?
I corrected a few things found by the model test but I have not gone back to see which one affected this. While working through the changes for the model test I had my cover-up in place (pointless sub-item that I updated instead of the top level). If you think it would be useful I could go back and try to identify what particular change improved the situation. Generally speaking, it's the changes made in the st.mirror branch (the last 7 in the link).
With more research I've seen a few comments suggesting that a QTableView may be significantly faster than a QTreeView. My data is only two layers and I could probably 'fairly easily' make it add/remove the second layer to simulate the 'tree' expansion. Are you aware of the relative performance between tables and trees?
I'm constantly taunted by the simple application from the hardware supplier that does something fairly similar (albeit a pure table, no tree aspect) with less than a tenth the CPU usage of just the GUI part of my application.
Screenshot of the 'other'/'efficient' application.
Thanks for your continued interest.