Asking for help - what is wrong with my QAbstractItemModel and QTreeView?

DerReisende

Greetings everybody. What is wrong with my model - or with Qt? I apologize in advance for the length of the thread.

I created a windows app using QTreeView and a custom QAbstractItemModel when Qt 6.2 was released. It was based on the "Simple TreeModel example" from Qt.
The model stores around 92k items in memory and UI performance so far was more or less no problem. I implemented more and more features (QSortProxyModels etc), upgraded Qt and I was quite happy with the performance the way it was supposed to work.

Then I ported the app to macOS - and the problems began...expanding the tree was good enough on windows (around 2 seconds with all the sorting and filtering proxy models) but on macOS this was awfully slow (70s plus 10-20s UI refresh). Same code, no modifications.
I tried to figure out where I could have made something wrong - no luck. The app also uses QAbstractItemModelTester during runtime and never showed any problems.

First I thought that my proxy models were responsible for the performance trouble. I created a testcase app which only includes a stripped down version of the model I am using without any additional proxies. Although it was slightly faster it still exhibits bad performance on UI side. I produced the following measurements with the testcase app (all in relase mode):

Configuration	Expand	Collapse	Select all (expanded before)
Windows 11 amd64 Qt 6.2.4	0s	0s	43s
macOS Sonoma 14.3 M1 Qt 6.2.4	0s	0s	7s
Win11 amd64 Qt 6.6.1	0s	0s	41s
macOS Sonoma 14.3 M1 Qt 6.6.1	0s-47s	0s	7s

Environment:
Apple MacBook Air M1 16GB RAM, 1TB SSD, macOS Sonoma 14.3, Xcode 15.2
AMD Ryzen 4700H 16GB RAM, 2TB SSD, Windows 11 latest updates as of today, Visual Studio 2022 latest updates

Performance with Qt 6.2.4 is good on macOS. Even the selectAll() is okay.
The expandAll() behaviour with Qt 6.6.1 on macOS is horrible and totally unreliable. There are cases where it will expand almost immediately on first click while all following clicks will time around 47 seconds. Sometimes it does expand quickly several times after collapseAll() calls and then suddenly it takes 47 seconds. Or it will expand quickly but after a selectAll() it again slows down to 47 seconds. And sometimes it starts and even the first expand is already slow as hell.
I tested with Qt 6.7beta2 as well on windows and macOS and the behaviour is similar to 6.6.
I did some quick tests with Qt 6.5.3 on macOS as well and the performance seemed similar to 6.2.4 but after some time I was able to produce the same delays, but it took much longer than with 6.6.1.

The expandAll() performance is killing me, especially since the real app is using additional proxy models and it takes 73s to expand the complete model with 92k items where a filtered model with approx. 75k items takes around 31 seconds.

I created a testcase app (link to zip file) which uses a sqlite database to load sample data into the tree view. Note that the database and the loader code are just a quick hack to get data into the testcase and should be ignored. All model data is held in memory and during runtime there are no database calls. Due to the amount of data in the db the compressed testcase is around 3MB.

As I now have really no ideas why the performance is that bad I am asking for help - what is wrong with my model and can other members reproduce similar results with the testcase?

Christian Ehrlicher

The performace trace looks good - and t.b.h. I knew that there was a change wrt the accessibility in QTreeView but forgot about it until I saw the trace. It's 6a4afebc5ce8db69a6c9fb398cada31e6bad5e3c - remove the resetModelIfNeeded() , recompile Qt and see if it helps.

/edit: Looks like there was a similar problem with a11y on windows in Qt6.2.x: https://codereview.qt-project.org/c/qt/qtbase/+/403662

JonB

@DerReisende
I don't have Qt6 or Mac. It is probably a Mac issue of some kind (and probably on the UI side not the model side), so you will have to await someone who can test this out. But I did have a quick look at your code (unusual for me on external source, but your question is so polite and well laid out!). I notice that you have a couple of columns and do a lot of work on dates, converting them to & from strings.

Can you temporarily remove any date columns and date work from your sample? It was a while ago now and in Qt5, but at one point we discovered Qt code was incredibly slow at dealing with date conversions, really showing up when many values in a database/view. It would be good to eliminate dates completely from your code/tests to ensure this is not an issue.

DerReisende

@JonB Thank you for your reply and the suggestion. I modified the testcase without any date handling (all code and db data gone) - same bad performance :(
Here is a new testcase without QDate handling and conversion.
I was involved in the QDate discussion and AFAIR there was a problem constructing QDate from a string. This case was handled in the previous testcase by caching already created QDate objects in a map. I didn't think about going from QDate to QString but the new testcase shows in my case it is irrelevant unfortunately.

JonB

@DerReisende
The old Qt date slowness was during converting between UTC time and local time, but was fixed. There was just an off chance it might be relevant for your case. At least you have eliminated this possibility now.

Obviously the best would be to profile your code, but goodness knows how easy/difficult that is under Mac, it's not that easy under Linux! You might put a qDebug() in your model's data() method to see just how many times it gets called, especially if your have filtering/sorting proxies going on, and see if it seems "reasonable".

However, it would not surprise me if this is (somehow) an issue at the UI side, not the model, under whatever Mac & Qt versions.

DerReisende

@JonB said in Asking for help - what is wrong with my QAbstractItemModel and QTreeView?:

@DerReisende
The old Qt date slowness was during converting between UTC time and local time, but was fixed. There was just an off chance it might be relevant for your case. At least you have eliminated this possibility now.

Ah, I was remembering a discussion here regarding Qt6 QDate performance.

Obviously the best would be to profile your code, but goodness knows how easy/difficult that is under Mac, it's not that easy under Linux! You might put a qDebug() in your model's data() method to see just how many times it gets called, especially if your have filtering/sorting proxies going on, and see if it seems "reasonable".

I am going to try building a static debug version of Qt and maybe I can see using Apple tools where most of the performance is burnt. Unfortunately with the current libraries I can only see that 99% of the performance is burnt in Qt's event loop.

However, it would not surprise me if this is (somehow) an issue at the UI side, not the model, under whatever Mac & Qt versions.

I think I am going to file a bug for Qt as well. But I need some more time for investigation.
But still it would be great to here from others if they can reproduce similar results on their machines with Qt6.

DerReisende

@JonB I am going to file a bug report for it. Using a static debug build I can see that all of the energy is burnt within QTreeViewPrivate::layout:

Christian Ehrlicher

The performace trace looks good - and t.b.h. I knew that there was a change wrt the accessibility in QTreeView but forgot about it until I saw the trace. It's 6a4afebc5ce8db69a6c9fb398cada31e6bad5e3c - remove the resetModelIfNeeded() , recompile Qt and see if it helps.

/edit: Looks like there was a similar problem with a11y on windows in Qt6.2.x: https://codereview.qt-project.org/c/qt/qtbase/+/403662

DerReisende

@Christian-Ehrlicher I can confirm that the removal of resetModelIfNeeded fixes the performance problems. Testcase now measures 0 seconds for expand operation :D
Do you need a bugreport with reference to forum and the gerrit link? Or can you handle that internally?

DerReisende

@Christian-Ehrlicher Bug reported.