QSqlQueryModel::data() return "null" vs "invalid" QVariant specification

[JonB]

I program in PyQt 5.7. I have to override http://doc.qt.io/qt-5/qsqlquerymodel.html#data. Because it returns a QVariant, as a result of my hugely complex thread https://forum.qt.io/topic/90363/inexplicable-qsqlquerymodel-handling-of-null-value (which you don't need to read to answer here) and the way PyQt handles that, suffice it to say that I need to know precisely when it will return a "null" QVariant versus an "invalid" QVariant.

Bearing in mind that I do not work with the Qt C++ sources, and that breakpoints on the overridden method are hit voluminously and are affected by up-fronting the window which makes it all very difficult, could some "expert" very kindly explain/detail, please?

My own limited investigations indicate that the method is called with a wide variety of ItemDataRoles, and returns either QVariant.isNull() or not QVariant.isValid() in different circumstances. Just for example, I have witnessed it return an isNull() against DisplayRole (for database NULL value), but a not isValid() for some other roles. But not enough for me to establish a robust pattern.

This distinction may be irrelevant to you in C++, but unfortunately because I am overriding the method in PyQt I have to know exactly when it will return each kind of null vs invalid QVariant, please... ?

[SGaist]

Hi,

Is it possible for you to provide a small sample application using e.g. SQLite that shows that behaviour you have ?

[JonB]

@SGaist
It's a question just about whatever the underlying Qt C++ code does.

Set a QSqlQueryModel to execute a query like SELECT col FROM table WHERE col IS NULL (where that column contains NULL in some rows), or even just a plain SELECT NULL AS come-column-name. Attach a QTableView to the model

Now override QVariant QSqlQueryModel::data(const QModelIndex &item, int role = Qt::DisplayRole) const to simply return the base method. Place a breakpoint on it and examine the returned value.

• If role==Qt::DisplayRole, the database NULL value will be returned as a "null" QVariant.
• If role==Qt::TextAlignmentRole, there is no existing text alignment role value, and the value returned will be an "invalid" QVariant.

From C++ there is no problem at all here, since "null" QVariant is different from "invalid" QVariant. You will be able to distinguish them.

From PyQt, however, the way it works is that QVariant return value is converted to a native Python type on return. In PyQt, both "null" & "invalid" QVariants are converted to Python None. The distinction is completely lost.

Worse, the same happens the other way round for bool QAbstractItemModel::setData(const QModelIndex &index, const QVariant &value, int role = Qt::EditRole). From PyQt, it is not possible for the QVariant &value to choose between passing either a "null" or an "invalid" QVariant. You can only pass Python None. And PyQt converts that to one of two of them, I don't even know which.

So..... I am asking, in the Qt C++ code for the various roles which can be passed to data() or setData() methods, is there, say, a particular role which can return both of "null" vs "invalid" QVariant from the data() method, or which from the setData() method acts differently according as the value passed in is "null" vs "invalid" QVariant? Because from PyQt I cannot see or maintain that distinction, and I'm stymied if it matters....

[SGaist]

Hence me asking you for an simple example so that we have exactly the same code to work on and find, if possible, a solution to your problem.

Roles and what is returned are two different things. There are roles that are expecting certain types, like for example a brush for BackgroundRole. Then for user specific roles, you are free to return whatever you want.

[JonB]

@SGaist
This is a question about the C++ source code/behaviour. I do not presently have a case which goes wrong, but I am concerned there may be such a case. I am not able to check every call in every combination, hence the question about the source code.

However, since you are kind enough to respond, here is outline Python code for the situation:

import sys
import typing
from PyQt5 import QtCore, QtWidgets, QtGui


class MySqlQueryModel(QtGui.QSqlQueryModel):
    def data(self, index: QtCore.QModelIndex, role: QtCore.Qt.ItemDataRole = QtCore.Qt.DisplayRole) -> typing.Any:
        return super().data(index, role)

    def setData(self, index: QtCore.QModelIndex, value: typing.Any, role: QtCore.Qt.ItemDataRole = QtCore.Qt.EditRole) -> bool:
        return super().setData(index, value, role)


class Main(QtWidgets.QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("Main")
        self.setGeometry(100, 100, 500, 500)

        self.centralWidget = QtWidgets.QWidget(self)
        self.setCentralWidget(self.centralWidget)
        self.centralLayout = QtWidgets.QHBoxLayout(self.centralWidget)

        # create a QTableView
        self.table = QtWidgets.QTableView()
        self.centralLayout.addWidget(self.table)

        # create a QSqlModel, and set it as view's model
        self.model = MySqlQueryModel()
        # set up a (MySQL) database connection
        # `my_table` has `col1` which is/can be NULL and `col2` which is/can be non-NULL
        self.model.setQuery("SELECT col1, col2 FROM my_table WHERE col1 IS NULL AND col2 IS NOT NULL")
        # or perhaps just the following:
        # self.model.setQuery("SELECT NULL AS col1, 'something-non-NULL' AS col2")
        self.table.setModel(self.model)

        self.show()


if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)

    main = Main()

    sys.exit(app.exec_())

Note the following:

• Here I am overriding QSqlQueryModel. If it makes a difference, I am doing exactly the same for QSqlTableModel & QSqlRelationalTableModel.
• The overridden data() & setData() methods are straight pass-throughs to the base C++ methods, but via Python/PyQt. If my code were C++ instead of Python/PyQt there would be no issue.

My question is about the behaviour of the C++ QVariant returned by, say, QSqlQueryModel::data() and the C++ QVariant passed from the caller into QSqlQueryModel::setData().

My question is for all possible in-built roles (enum Qt::ItemDataRole) which might be passed to these methods, from Qt::DisplayRole through Qt::InitialSortOrderRole, inclusive, i.e. all the Qt defined Qt::ItemDataRoles, and not any kind of Qt::UserRole I might define. Thus it includes both true "data" roles (e.g. Qt::DisplayRole) and purely "appearance" roles (e.g. Qt::TextAlignmentRole).

My question concerns 2 "slightly odd" QVariant types/values:

• The "invalid" QVariant. This is where QVariant::isValid() == false. I believe it can be created via plain QVariant().
• The "null" QVariant. This is where QVariant::isNull() == true. I am unsure how it is created in C++.

Now, unlike C++, PyQt "maps/unmaps" these two to/from the same Python None type/value. The QVariant type is simply hidden from me. The author of PyQt states that this relies on "[a particular call] not needing to treat these two as distinguishable", which he believes "is 'mostly' the case". But he states he does not claim to know exactly what individual C++ methods might do in the way of distinguishing them.

So, my questions are:

1. In the case of the return value from any of the data() overrides, for any given ItemDataRole passed in are there any cases where it might return both an "invalid" and a "null" QVariant under certain circumstances, to mean different things? We are only interested in one, single role being able to return either/both of these, not in different roles where some return one and some return the other. This is because PyQt returns None to me for either/both cases, so I cannot distinguish.

2. In the case of the value passed into any of the setData() overrides, for any possible ItemDataRole passed in are there any cases where it would behave differently depending on whether an "invalid" or a "null" QVariant is passed in? This might mean they treat those differently such that it matters, or it might mean that they only accept one of two and error on the other. This is because I can only pass None from PyQt for either/both cases [it maps that to one of "invalid"/"null" QVariant (always the same), and I don't even know which of the two], so I cannot distinguish.

I trust that the above is a very clear statement of the issue I face when writing my code. I cannot try out every combination of roles & values for each method in every circumstance, so I am asking someone with knowledge of the code/C++ behaviour to help.

Thank you very much.

[JonB]

@SGaist
Do you have any comment on my "loss-of-distinction" for "invalid" versus "null" QVariants causing any problems for either QAbstractItemModel::data() or QAbstractItemModel::setData(), in view of my clarification and small example code above?

Or, do I just have to go with it and hope/see whether I get away with it OK in practice?

[VRonin]

The only place it really matters is the editor delegate.

If your model data() returns a valid but null QVariant, the QItemEditorFactory will know what appropriate editor to show (QSpinBox for numbers, QDateEdit for dates, etc.) if you lose that distinction it will just get the default (a QLineEdit).

I'm not aware of any distinction mattering in setData on QSqlQueryModel

[JonB]

@VRonin
Thanks for comment.

Obviously I'm most concerned about

I'm not aware of any distinction mattering in setData on QSqlQueryModel

because getting that wrong will potentially end up putting the wrong data value back to the database!

In order set the data to what I will want passed onto MySQL as database NULL, I have to go QSqlQueryModel.setData(index, None) from PyQt. But I have no control over whether this ends up calling the C++ base method with the QVariant &value parameter set to a QVariant where either:

• QVariant::isValid() == false; or
• QVariant::isValid() == true && QVariant::isNull() == true

In all cases that None will map to (the same, regardless of the role) one and only one of these two, but I don't even know which!

If it does not matter which, or if it passes the "right" one (presumably the second case), it will be fine. Else I'll have problems... Let's hope for now that your "I'm not aware of any distinction mattering" is correct!

[VRonin]

The MySql driver only checks isNull (source) not isValid so since isValid==false implies isNull==true there is no distinction between them when using setData on QSqlQueryModel using that database

[JonB]

@VRonin
That sounds good.

so since isValid==true implies isNull==true

Umm, did you mean that? Any valid QVariant is also "null", doesn't sound right. Do you perhaps mean isValid==false implies isNull==true??

This is not helped for me because I'm at the Python side not the C++ side...

[VRonin]

@JonB said in QSqlQueryModel::data() return "null" vs "invalid" QVariant specification:

Do you perhaps mean isValid==false implies isNull==true??

Yep, exactly, fixed now

[JonB]

@VRonin
You did that deliberately just to confuse me, didn't you? ;-) I was scratching my head thinking that I really didn't understand what was going on!

[VRonin]

@JonB said in QSqlQueryModel::data() return "null" vs "invalid" QVariant specification:

You did that deliberately just to confuse me, didn't you?