Delegate creation and multithreaded property updating

[felsi]

Fortunately, the problem appears every few runs...
By the way, there is of course no problem, when all are set before or after the delegate creations.
And also not everytime when in between.
After a few checks, i would say, the workarounds behave like expected.
For completeness:

With connectTrack() hackmack:
onPlaylistTrackChanged 1
connectTrack 1
updateTrack 1 : NULL
onCompleted 1
onPlaylistTrackChanged 2
connectTrack 2
updateTrack 2 : NULL
onCompleted 2
onPlaylistTrackChanged 3
connectTrack 3
updateTrack 3 : NULL
setTrack 1
onTrackChanged 1
updateTrack 1 : Track 1
setTrack 2
onTrackChanged 2
updateTrack 2 : Track 2
setTrack 3
onTrackChanged 3
updateTrack 3 : Track 3
setTrack 4
setTrack 5
onCompleted 3
onPlaylistTrackChanged 4
connectTrack 4
updateTrack 4 : Track 4
onCompleted 4
onPlaylistTrackChanged 5
connectTrack 5
updateTrack 5 : Track 5
onCompleted 5

With updateTrack() in onCompleted:
onPlaylistTrackChanged 1
updateTrack 1 : NULL
onCompleted 1
updateTrack 1 : NULL
onPlaylistTrackChanged 2
updateTrack 2 : NULL
onCompleted 2
updateTrack 2 : NULL
onPlaylistTrackChanged 3
updateTrack 3 : NULL
setTrack 1
onTrackChanged 1
updateTrack 1 : Track 1
setTrack 2
onTrackChanged 2
updateTrack 2 : Track 2
setTrack 3
setTrack 4
setTrack 5
onCompleted 3
updateTrack 3 : Track 3
onPlaylistTrackChanged 4
updateTrack 4 : Track 4
onCompleted 4
updateTrack 4 : Track 4
onPlaylistTrackChanged 5
updateTrack 5 : Track 5
onCompleted 5
updateTrack 5 : Track 5

[Asperamanca]

I wouldn't call it a workaround. The order of creation and update is not deterministic, so you have to take into account that components are created before or after you have already set some properties. For example, are some items visible and some not? This could trigger a kind of "priority of initialization" heuristic.

[felsi]

Yeah, but i wouldn't call it a patch either... ;)
Yes, i know, that's the reason, why i force the connection to be established before i update.
As far as i can see, that ensures, that there is at least one correct update.
And i have no other problems, everything's updating fine, if it does.
Without an explanation of the root cause i have no better solution...

I am very thankful for your time and effort to help me.
But honestly, if it's not a Qt bug, i will move on and live with the obscurity...

[Asperamanca]

QML scene are often loaded asynchronously. That's just necessary to prevent everything from blocking as you load a large scene.
When do you start calling the setters for Track? I don't believe I have seen your code that controls it.
Basically, if you don't wait for your QQuickView / QQuickWidget to signal it's status "Ready", you cannot depend that all components exist. Even after that, if you are using Loader or dynamic instantiation, you could be surprised. So the most robust code will not depend on the order of events. Component created first? Properties changed first? You should not (need to) care.

[felsi]

Uh, it is actually part of a larger element, which i load asynchronously with a loader! And it appears exactly then, at start-up.
I don't know, how and what i could show you. I have a PlaylistReader, running in a thread, connected to the setter, running in main thread. It loads the current playlist at start-up.
I absolutely see your point, it's damn ugly imperative...

If i disable asynchronously loading, setting and creating are not simultaneous anyway...
Uff, this opens up a lot of possible issues, damn.

Thank you very, very much!!
I wish you an extra sunny day!

[Asperamanca]

You are very welcome.
If you like, I can share my approach to cope with these issues.

[felsi]

I would highly appreciate to hear about that.

[Asperamanca]

My concept is this:

1. A QML component and C++ property class (i.e. the class with the necessary Q_PROPERTYs) always come in pairs
2. The property class instance ALWAYS exists before the QML component is created. It is passed as a "required property"
3. Complex sub-components come with their own property class, so...

C++:

class CInnerProperties : public QObject
{
// Some properties
};

class COuterProperties : public QObject
{
// Some properties
Q_PROPERTY(CInnerProperties* innerProperties [...])
};

QML (Inner component)

MyInnerQmlComponent
{
   required property CInnerProperties innerData
   //...
}

QML (Outer component)

MyOuterQmlComponent
{
   required property COuterProperties outerData
   //...
   MyInnerQmlComponent
   {
      innerData: outerData.innerProperties 
   }
}

4. Instantiation is controlled from the C++ side, by creating a QQmlComponent, and then calling createWithInitialProperties (passing the necessary property class instance to satisfy the "required" property
5. Pointer properties are almost always CONST. I change their content, not the pointer.
6. Data is passed by value, by copy. To improve performance and reduce memory usage, I make extensive use of implicit sharing.
7. Data is sent between threads via signals / slots

This approach has the following benefits:

• Since the property class is guaranteed to exist when the component is instantiated, I rarely have need to manually connect any signals. I can mostly rely on property bindings, which automatically work
• Since the property bindings are automatically there, it does not matter whether the properties are already initialized with their correct values when the QML component is created, or whether we start with dummy values, and update the properties
• By-value data management allows multiple threads to produce data, and the GUI thread to consume it. There is no locking necessary (besides the builtin atomic ref counts of implicit sharing). This also allows easy tracing of data changes.

[felsi]

Uh, wow, thank you!

Point 4 rocks!
So, i create the QQuickItem in data() of my model class and pass it as a property. Is this correct?
And in qml i do:

delegate: Item {
    id: delegateItem
    width: listView.width
    height: root_item.delegate_height

    property Button_Playlist_Info cppItem : model.cpp_item
    onCppItemChanged: {
        cppItem.parent = delegateItem;
        cppItem.anchors.fill = Qt.binding(function() {return cppItem.parent;});
        cppItem.track_nr = Qt.binding(function() {return model.track_nr;});
        cppItem.clicked.connect(root_item.clickEntry);
        // cppItem.clicked.connect(root_item.clickEntry(model.track_nr)); // not possible
        // delegateItem.destroyed.connect(cppItem.destroy()); // error
    }

I haven't even heard about creating qml stuff in c++ before!
Damn, that's an elegant way to make only this specific part not asynchronous, you are great!
And it looks pretty crazy, i like it!
I just haven't found a way to destroy properly yet...

This is ridiculous off-topic anyway, so maybe you find my use cases and features interesting (already implemented):

• Collections with 100k tracks are a realistic scenario.
• Adding all 100k tracks to a playlist is a common case.
• 5 loaded playlists for copy/paste/search is also common.
• 5 sorted collections, trees, but quite similar to playlists (CollectionTrack).
• The metadata of a track can easily contain 300 characters and more.
• Playlists up to 500k can be opened and are editable instantly and are fully loaded in 10s. (twice as fast as gedit, for whatever reason)
• The player starts instantly and the current track is instantly playable.
• Playlists can be loaded before the collection has been loaded/updated.
• Mediafiles can be cut and pasted freely or can temporarily be missing without invalidating playlists, as long as the metadata stays the same.
• The collection watches folders/drives and adds/removes tracks at runtime.

So, there is a lot, i have to consider.
E.g. there are easily up to 1M tracks present, but only 100k exist, avoiding to copy and lightweighted container classes are very important.
Even the difference between a shared and a blank pointer are several MB...
I really need to replace tracks, imagine the following: two files, same metadata, one get deleted...

My internal data structure fits my needs pretty well.
Everything is running like it should, i am pretty happy.
And i am mad enough to violate some qml principles. ;)

Thank you for all your hints and suggestions!
I am very, very, very glad, you helped me!
3 Times!!!

[Asperamanca]

It sounds like for your use case, and your number of items, using Model/View and ListView / TableView / TreeView would be the better approach. That way, the model/view infrastructure only creates as many visual items as necessary.
I use the creation in C++ only on a higher level in my scene. So e.g. I can create an item in C++ that in turn contains a TableView, and the TableView is connected to a model with some 10ks of lines.

I'm not sure I understand the QML example you posted. I'll try to elaborate more on how I do QML creation from C++ (but there are probably other ways to do it):

Let's say we have a component that should show a table. Since I only know at runtime whether I need it, I want to create it via C++ on demand.

We have a C++ class "CMyTableProperties". It is created once we know we'll need the table. Lifetime is controlled by C++. It must be exported as QML_ELEMENT. If you want to know the type in QML, you need to add it to a QML module, and import that module in the QML file.
CMyTableProperties exposes a "model" property, which is a QAbstractItemModel (or table model, or whatever)

Then we have a QML component (a separate file), e.g. "MyTable.qml", which looks something like this:

import MyProductName.MyQmlModuleName

Item
{
   required property CMyTableProperties pData
   implicitWidth: // some sensible value or calculation
   implicitHeight: // some sensible value or calculation
   TableView
   {
      anchors.fill: parent
      model: pData.model
      //...
      delegate: //...
   }
}

Your main QML scene can now have a placeholder item

Item
{
   // Main scene
   Item
   {
      objectName: "itemContainer"
      anchors.fill: parent
   }
}

When you want to add that table component, you

1. Create the property class
2. Find the "itemContainer" by traversing the tree of QQuickItems and looking for an object named "itemContainer"
3. Create the QML component "MyTable" in C++, pass your property class as initial property, and pass the itemContainer as parent item. You can also use the root item of a scene if you don't need any static QML.

[GrecKo]

@Asperamanca said in Delegate creation and multithreaded property updating:

I use the creation in C++ only on a higher level in my scene. So e.g. I can create an item in C++ that in turn contains a TableView

Why? That's adding some extra coupling between c++ and QML. Can't you just expose your model and access it when needed in QML?

@Asperamanca said in Delegate creation and multithreaded property updating:

When you want to add that table component, you

1. Create the property class
2. Find the "itemContainer" by traversing the tree of QQuickItems and looking for an object named "itemContainer"
3. Create the QML component "MyTable" in C++, pass your property class as initial property, and pass the itemContainer as parent item. You can also use the root item of a scene if you don't need any static QML.

That seems overly complicated to me.

[felsi]

@GrecKo
Just set the model after loading.
How boring...
I was so excited about item creation in c++.
Thank you! :)

@Asperamanca
Oh, i see. I just wrote some lines, i thought you meant something like that. As i said, i haven't noticed this topic before.
Thanks for helping me!

[GrecKo]

@felsi said in Delegate creation and multithreaded property updating:

I was so excited about item creation in c++.

You should not ;)

[Asperamanca]

@GrecKo said in Delegate creation and multithreaded property updating:

Why? That's adding some extra coupling between c++ and QML. Can't you just expose your model and access it when needed in QML?

Good question. Several reasons:

1. When I started the project, I was not aware of a QML way to just expose a model and have and item automatically created for each entry in the model. Something like a crossover between Loader and TableView. Does something like this exist now?
2. I need very precise control when items are created. For example, I might create data classes for 100k items (they are cheap to instantiate), and only create QML components for those that will be actually visible. The mythical components from item 1 might be able to do that, but I'd need to see it documented to understand whether it would be suitable for my use case.
3. I minimize JavaScript usage, because I have good tooling and static analyzers for C++, but not for JavaScript, and also we have developers trained in C++, and please don't tell me "everyone can write JavaScript". Well, yes. Bad, unsafe JavaScript.
4. This led to offloading pretty much all logic into C++, with the exception of layouting. QML is my layout language, and it is very good for that.
5. Item hit testing, most of the event handling....all done in C++ in a unified manner that I can roll out to 22 different types of component with ease. Did I mention I have written several bug reports about QML's handler components? Handling the QEvents yourself is much more reliable, I eliminated a whole layer of buggy code that way.
6. Finally, I am not writing a library. I am writing an application. Both the C++ and QML code are specific to the use case, so it's OK that C++ and QML code have to match. Even so, I fail to see "extra" coupling. The kind of coupling is different: Instead of using a Qt model to communicate between C++ and QML, I have defined my own API. Each component must provide both a C++ property class and a QML component identifier via factory. The rest of the code is generic. My dataset contains component FOO? Create a FOO property class up front. Once I have some data (and with every data change), I query my loader whether we need the UI for it. If we want it, I query the factory for the correct QML component for FOO, and load it.

It's rolled out in a product and easy to work with, and I haven't found any performance issues with it, either.

[Asperamanca]

@GrecKo But I do agree that for OP's use case, the model is the better approach.

[GrecKo]

1. & 2.
   I don't really understand the question. Models&Views does that.
   A ListView can do that with the added benefit of delegate pooling.
   If the ListView does not fit your use case (you might not need it to position the delegates). A Repeater (or Instantiator) plus a proxy model might do the trick.

2. This doesn't involve JS, just a couple more QML objects with basic bindings.

3. You still could do that in C++ if you want.

Your solution does work, I'm just saying it might not be the most straight forward one or necessary for everyone.

[Asperamanca]

@GrecKo said in Delegate creation and multithreaded property updating:

& 2.
I don't really understand the question. Models&Views does that.
A ListView can do that with the added benefit of delegate pooling.
If the ListView does not fit your use case (you might not need it to position the delegates). A Repeater (or Instantiator) plus a proxy model might do the trick.

Please walk me through it, if you would be so kind.
When I want to display a table of items, I use a TableView. The model can be huge, but only the delegates actually needed for display are created (and a few more to ensure smooth scrolling, maybe). So far, so good.

In my case, I do not want to arrange my components in a pre-defined way. The user has positioned them in a certain position, and maybe there is some (also user-specified) runtime logic that could move them around. Also, whether they are actually visible right now might depend on runtime logic.

A ListView is a Flickable. A flickable does not make sense for my use case (the components are geometrically unrelated to each other).

A Repeater (AFAIK) only instantiates the same type of item. I could use it, but I would need a wrapper item.
Then again "The Repeater type creates all of its delegate items when the repeater is first created.". That means, if I have 100k data classes, I will get 100k QML items of some kind. Now, I'd need to measure how expensive that would be, and it's certainly not free.

A Loader can only load a single component. I could probably pair it with Repeater to get what I want. I think it would pretty much do the same thing as my current code, at the cost of always loading the 100k Loaders. Also, how do you use a loader to load a component from a QML module directly? Eventually, I want my components to be pre-compiled.

Did I miss an option?

I admit, part of the reason I chose this approach is also that I'm not much of a ModelView guy, so I didn't consider using ModelView for anything else but where a ready-to-use View component already exists.

[felsi]

I don't want to bother you guys anymore, but i have some final thoughts...
Trying to avoid bindings as much as possible has become a bit of a habit and i use connections+js a lot...
It is clear now, but i did not expect, that connections+loader can have these side effects.
Maybe this is worth to be mentioned somewhere in the documentation with a small note.
And, by the way, is the following safe?

Button
{
    property PlaylistTrack playlistTrack
    onPlaylistTrackChanged: updateTrack()
    
    property Track track: playlistTrack.track
    onTrackChanged: updateTrack()
    
    function updateTrack()
    {
        ...
    }
        
}

[felsi]

Just a small simplification mistake, i don't call it twice, of course...

[Asperamanca]

@felsi Whether it's safe would mainly depend on what you do inside updateTrack() (do you check for null, etc.)
I can see no fault in calling updateTrack() in both change handlers.