Simplest way to write M-to-N (eg. aggregating) proxy models?

ssokolow

dataChanged is emitted anytime setData is called so while not being sufficient in general (adding an empty line will not trigger it) it is in practice (as soon as you populate that empty line dataChanged will be emitted) the other signals to keep an eye on are rowsRemoved, modelReset and possibly columnsRemoved

That makes no sense. I'm using QAbstractItemModel rather than QStandardItemModel, which means I don't just magically get insertRows for free, which means that the signals you listed can't be all there is to it.

I get the impression you're still be operating under the assumption that Qt's model holds the authoritative copy of the data in some sense of the word, rather than simply being a proxy that takes a non-Qt "model" as its source.

I have to get to bed but, once I get back tomorrow, why don't I just write the kind of summary I was hoping to find and then you can tell me if I got anything obviously wrong.

VRonin

That makes no sense. I'm using QAbstractItemModel rather than QStandardItemModel, which means I don't just magically get insertRows for free

If you don't, you are subclassing QAbstractItemModel the wrong way. http://doc.qt.io/qt-5/qabstractitemmodel.html#subclassing clearly specifies:

The dataChanged() and headerDataChanged() signals must be emitted explicitly when reimplementing the setData() and setHeaderData() functions, respectively.

An insertRows() implementation must call beginInsertRows() before inserting new rows into the data structure, and endInsertRows() immediately afterwards.

and endInsertRows takes care of emitting rowsInserted. below you find the implementation

void QAbstractItemModel::endInsertRows()
{
    Q_D(QAbstractItemModel);
    QAbstractItemModelPrivate::Change change = d->changes.pop();
    d->rowsInserted(change.parent, change.first, change.last);
    emit rowsInserted(change.parent, change.first, change.last, QPrivateSignal());
}

These mechanisms are relied upon by the views to work that's why they are so strict

ssokolow

@VRonin

Yes, but the docs for insertRows also say:

Note: The base class implementation of this function does nothing and returns false.

If you implement your own model, you can reimplement this function if you want to support insertions. Alternatively, you can provide your own API for altering the data. In either case, you will need to call beginInsertRows() and endInsertRows() to notify other components that the model has changed.

In other words, I have two choices for supporting unsolicited insertion events from the backend:

I can maintain the backend's internal model separate from the frontend's model (in which case, I might as well use QStandardItemModel) and write unappealing sync code to manually mirror changes back and forth.
I can write my QAbstractItemModel subclass such that the only state it keeps is cached results for expensive-to-derive fields and manually call beginInsertRows and endInsertRows when a new row arrives from the backend without warning.

To rephrase what I said before, we're obviously not on the same page because the only way your advice, as stated, makes sense, is if I'm taking the former approach, which I don't consider acceptable.

When I'm free tomorrow, I'll try to summarize my understanding of when to call which begin* and end* functions and you can tell me if I've missed or misunderstood any.

ssokolow

Things went bad and I only got about 3 or 4 hours of sleep. I'll summarize as soon as I've had a good night's sleep.

ssokolow

Sorry for the lack of response. Shortly after my last message, I got bombarded with obligations and it's taken me the last two weeks to get back to where I was before.

I may or may not have a proper response in the next couple of days. The rest of the family is just getting over a debilitating flu and I've started to show faint symptoms.

VRonin

That's often the result of meddling with model/view programming :)

Hope you get better

Eeli K

"That's often the result of meddling with model/view programming :)" Indeed!

I don't know if I can offer any real help, but I tell how I would begin. I would make a class inheriting QAbstractItemModel. It has the base model as a (pointer) member. The base model, if it behaves correctly, sends signals when changes are made to it. First I would create only one slot to which I would bind all the signals of the base model except "AboutTo" signals. This slot would simply emit modelAboutToBeReset and modelReset signals (in this order). Everything else would be implemented in the required public functions which are pure virtual in QAbstractItemModel (e.g. rowCount, index, data). Each of those functions would query the base data and return the required result dynamically. In this way every time something changes in the base model my new model would just inform the view that the model is reset. The view constructs itself automatically from scratch by calling the implemented required functions.

In this way I would see if the logic is correct. After that it's easy to cache some data and create new slots which catch e.g. dataChanged and rowsInserted signals which come from the base model. They must be translated to meaningful events in my own model, for example dataChanged of the base model could mean inserting a row in the new model. The new slot which catches the dataChanged signal of the base model queries the data and decides for example that a game was added, therefore new rows are inserted under several different top level nodes. It would emit rowsAboutToBeInserted and rowsInserted with proper arguments for each added row. The view catches those signals and adds the rows as it wants by calling the implemented required functions. As you can see, the QModelIndex is in a big role in emitting the signals and giving the data to the view.

There's nothing nice M-to-N in this, just a raw general way to create any kind of proxy. Also, because it reimplements QAbstractItemModel it's not as simple as one might wish. Especially I hate QModelIndex.

ssokolow

@Eeli-K

First I would create only one slot to which I would bind all the signals of the base model except "AboutTo" signals. This slot would simply emit modelAboutToBeReset and modelReset signals (in this order). Everything else would be implemented in the required public functions which are pure virtual in QAbstractItemModel (e.g. rowCount, index, data).

Thanks, but I already do that in the semi-placeholder implementation I used to take the screenshot. The part I'm uncertain about is how it all ties together. (eg. In which exact circumstances will and won't each signal get fired off? How does this all interact with lazy-loading models like QSqlQueryModel? etc. etc. etc.)

I have a vague theoretical understanding of how it all works, but I don't think I'm uniquely perceptive and Qt's Model/View system is a bit infamous for developers having implemented things incorrectly when writing custom components.

Anyway, once I'm no longer half-braindead from this cold, I'll try to compile my interpretation of how it all fits together into a document. Even if I can't get any "that's wrong" feedback for it, it should give me an idea where I could root out some ambiguities by writing automated tests.

Eeli K

@ssokolow said in Simplest way to write M-to-N (eg. aggregating) proxy models?:

I have a vague theoretical understanding of how it all works, but I don't think I'm uniquely perceptive and Qt's Model/View system is a bit infamous for developers having implemented things incorrectly when writing custom components.

No wonder. This is a learning experience for me, that's why I answered, I probably don't know more than you. The thread draw my attention exactly because the whole model/view system has felt frustrating and I would like to know more.

The part I'm uncertain about is how it all ties together. (eg. In which exact circumstances will and won't each signal get fired off? How does this all interact with lazy-loading models like QSqlQueryModel? etc. etc. etc.)

Good question. My understanding this far is something like this:

First, the model has to know when the underlying data is changed. For example, a database connection triggers an automatic message when something is changed. Or the data is queried periodically, or whatever suits best. The model (=programmer) has to decide which change-related signal it should trigger to inform views. Is it just one row or a set of continous rows inserted? If you don't cache anything, just call beginInsertRows and endInsertRows. They call the corresponding signals, which are handled by the views. Is it several rows, one here, another there? call begin... and end... for each row. If it looks like there are so many changes at the same time that it's cheaper for a view to reconstruct the whole thing, beginResetModel and endResetModel instead. If the amount of rows doesn't change but same data inside of a row is changed, naturally you emit dataChanged (there's no begin... and end... for data change).

If you cache data or have the data inside the model (e.g. in lists or maps), first call begin... , then change your data, then call end... As the doc says about beginInsertColumns:

Note: This function emits the columnsAboutToBeInserted() signal which connected views (or proxies) must handle before the data is inserted. Otherwise, the views may end up in an invalid state.

And about endInsertColumns:

When reimplementing insertColumns() in a subclass, you must call this function after inserting data into the model's underlying data store.

If you don't use or reimplement the standard data-changing API, you still call these functions when your underlying data is changed. But this triggers one question: should the view or other catcher of the begin... signal have access to the old state of the data? It would be impossible if you don't cache it and just receive a message about a change which happened earlier from the underlying data source.

In the case of the QStandardItemModel we have append*, insert*, set* etc. functions for manipulating data. It's logical to think that whenever you use one of these standard functions, the corresponding signals are triggered automatically. E.g. the default implementation of insertRow would first emit rowsAboutToBeInserted, then store the given QStandardItem, then emit rowsInserted.

ssokolow

Just a status update. I haven't given up and I've started to work my way through the relevant bits of the Qt docs to build a list of questions which I feel the docs don't answer properly.

Once I've finished that, I'll start putting together a plan for getting some answers.

ssokolow

In case anyone already knows the answers, here are the TODOs I've accumulated so far as I attempt to distill my way down the relevant bits of the Qt docs and fold in the appropriate cross-references.

How does API-based selection interact with "select by row/column, not by cell" restrictions on user selections?
Verify that list and table views essentially behave like tree views which hide the existence of child nodes (and verify my understanding of the effects of asking a list/table view to move, mutate, or delete nodes with children)
Test the actual user-visible result of not following the "common" convention that only items in the first column have children.
- ...what if multiple cells in the same row have different children?
How do ranges (eg. selection APIs) interact with tree-structured data? What about data where children have variable column counts? (Selecting all items in a model suggests children are skipped)
How do Qt's various views and proxy models behave when the number of columns is not "independent of the parent"? (eg. What if the proxy model is set
to expect a column number that not all subtrees have?)
What actual, implemented (not theoretical) purpose is served by having both "about to be" and "done" signals and slots for changes to the model?

For something so heavy on examples, Qt's manual just seems to say "you must do this" without explaining why.
Under what circumstances might I want to subclass QItemSelectionModel?
How does the lazy-loading (canFetchMore and fetchMore) interact with the signals and proxy models?
Make a list/table mapping common operations to signals which should be emitted. (eg. When you sort the model, emit layout[AboutToBe]Changed)
- The docs seem to be saying that, if I emit layout[AboutToBe]Changed rather than calling beginMoveRows/endMoveRows, I have to also call changePersistentIndex manually, which suggests that it should only be done that way if it really is a case like sorting where you're moving many rows in a non-contiguous and outwardly haphazard manner and don't want the overhead of a bunch of tiny operations.
- Audit which situations do and don't require changePersistentIndex or changePersistentIndexList to be manually called.
How does overriding match for non-column data map to QSortFilterProxyModel? Does it at all?
How exactly do submit and revert fit into things?
Investigate cache-related flags in EndEditHint
Does QAbstractItemModel::span have a default implementation?