Validation behaviour too confusing for end user

[JonB]

I have inherited a body of code which uses a validator on QLineEdits which allow a number with decimal point --- in fact specifically it's for a monetary amount. I know nothing about Qt's validation or how I'm intended to proceed. Here it is:

class DecimalValidator(QtGui.QRegExpValidator):
    def __init__(self, parent=None):
        super().__init__(parent)

        self.setRegExp(QtCore.QRegExp("\d+(\.\d{1,2})?"))

line = QLineEdit(self)
line.setValidator(DecimalValidator())
line.editingFinished.connect(lambda: ensureValidDecimal(self))

The problem lies with DecimalValidator. It is trying to enforce that a valid money amount is 1 or more digits, optionally followed by a decimal point with 1 or 2 digits after it. Now, it appears that QLineEdit.setValidator() applies the validator logic while the user is typing into the line (doubtless what it's supposed to do!).

1. The user first types in 3.21. All well & good.
2. But he realises he meant the amount to be 4.21 instead. What does he do?
3. Well, the natural thing to do is to move the caret to just before or after the leading 3, delete it, and type in 4.
4. But when he tries to delete the 3 it refuses to vanish (with no feedback), at which point he becomes confused because he has no idea why he can't delete it. The validator is failing because we have said that the number must have at least one leading digit (which I do want, but only when he has finished editing).
5. It does not occur to him to first type in the 4 he now wants and then delete the 3, and I don't want him to have to do that.

The problem is the strict enforcement of the validator as he is typing in. I am not used to this, I am used to enforcing validators at end of editing stage. I can see the code calls a function for this via QLineEdit.editingFinished.connect(), which I guess is what I am used to. 3 questions:

1. Can validation be delayed till end of editing rather than as you type, in some shape or form?

2. I can't see a way to write the validator regexp which still does the validation but allows the deletion of the leading digit if necessary. Can anyone else?

3. Assuming not, what do you think I should do here? I think I'm going to have open the validation at type-in time up to allow non-finally-acceptable input because of this issue. And then rely on QLineEdit.editingFinished.connect() to do definitive validation, right? In which case, I could either get rid of the regexp validator completely, or change it to "\d*(\.\d{1,2})?" which at least ensures only digits & decimal point typed in. Opinions?

Furthermore, looking at existing code for ensureValidDecimal() (which is a bit convoluted to include here), I don't see that it "errors" or "rejects" invalid input. If I have to use the editingFinished signal to do post-editing validation, what is it supposed to do to "reject" the input?

[aha_1980]

@JNBarchan said in Validation behaviour too confusing for end user:

Can validation be delayed till end of editing rather than as you type, in some shape or form?

In principle Qt's validator provide this: http://doc.qt.io/qt-4.8/qvalidator.html#State-enum

As long as your validator returns QValidator::Intermediate, you can continue editing it until the string becomes QValidator::Acceptable.

The problem you here run into, seems that QRegExpValidator always starts validating from the beginning: If a string is a prefix of an Acceptable string, it is considered Intermediate. For example, "" and "A" are Intermediate for the regexp [A-Z][0-9] (whereas "_" would be Invalid). [1]

I don't know if there is any workaround here. I'd suggest writing your own validator based on QValidator. Then return intermediate as long as the string can still become valid. Otherwise return QValidator::Invalid.

When the string is a valid input, return QValidator::Acceptable.

Hope that helps.

[1] http://doc.qt.io/qt-4.8/qregexpvalidator.html

[JonB]

@aha_1980
Thank you for this. I will investigate and report back.

However, may I ask regardless for an answer to: if, for whatever reason, I wish to do final validation in QLineEdit.editingFinished, since it's not like this returns True/False to prevent leaving the edit control, I'm guessing my handler would be responsible for raising an error if it didn't like what it sees?

Also, I don't suppose that setInputMask() unlike setValidator() only operates at editingFinished time rather than as the user types?

[aha_1980]

@JNBarchan said in Validation behaviour too confusing for end user:

However, may I ask regardless for an answer to: if, for whatever reason, I wish to do final validation in QLineEdit.editingFinished, since it's not like this returns True/False to prevent leaving the edit control, I'm guessing my handler would be responsible for raising an error if it didn't like what it sees?

You can call QLineEdit::hasAcceptableInput in that slot. I'm not sure if you should prohibit leaving the edit field, but at least you can disable the "Ok" button if the input is invalid. I have some dialogs acting like that (but I call hasAcceptableInput on every textChanged() signal to enable/disable the Ok button in a dialog.

[VRonin]

1. QRegExp is deprecated, use QRegularExpression
2. In C++ the solution is trivial, you just need to reimplement the validator to transform Invalid input into intermediate input:

class LenientRegExpValidator : public QRegularExpressionValidator{
Q_OBJECT
Q_DISABLE_COPY(LenientRegExpValidator)
public:
LenientRegExpValidator(QObject* parent = Q_NULLPTR) : QRegularExpressionValidator(parent){}
LenientRegExpValidator(const QRegularExpression &re, QObject *parent = Q_NULLPTR) : QRegularExpressionValidator(re,parent){}
QValidator::State validate(QString &input, int &pos) const Q_DECL_OVERRIDE{
const QValidator::State baseValidator = QRegularExpressionValidator::validate(input,pos);
if(baseValidator ==QValidator::Invalid)
return QValidator::Intermediate;
return baseValidator;
}
};

[JonB]

@VRonin said in Validation behaviour too confusing for end user:

QRegExp is deprecated, use QRegularExpression

Documentation observation:

In Qt docs, in http://doc.qt.io/qt-5/qregexp.html#details I see

Note: In Qt 5, the new QRegularExpression class provides a Perl compatible implementation of regular expressions and is recommended in place of QRegExp.

But in http://doc.qt.io/qt-5/qregexpvalidator.html#details I see no such admonition for using QRegularExpressionValidator instead.

[JonB]

@VRonin

In C++ the solution is trivial, you just need to reimplement the validator to transform Invalid input into intermediate input

OK, I have managed to implement this in PyQt for my Python (bit of a struggle, but done).

[As an aside, it's a bit more complicated than that. I presume I need 2 regular expressions in the QValidator, one for the reg exp which is as it is now to return Acceptable, and then a second one which allows no leading digit at all to return Intermediate, and then if it fails both (e.g. user types a letter, or two decimal points) return Invalid. Not sure how to do that as QValidator allows just one expression in setRegularExpression() which is used by the base validate()... Am I supposed to change the reg exp via setRegularExpression() twice and call base validate() twice each time in my validate() override??]

But ignoring that complication for the moment, what's the "point" of Intermediate? Once I have put in the code, instead of the user not being able to erase the single leading digit because it returns Invalid, he now can, just as if it returns Acceptable, as far as the Qt widget editing is concerned. So he can leave the field reading just .23, which is not acceptable as a "final answer" to me. Where is the difference between Acceptable & Intermediate "expressed"/"acted upon"?

I can only guess that the widget lets him type intermediate entry, and then it is up to me to go recheck for "truly" Acceptable, either on attempting to OK the Dialog or on attempting to exit editing the field. Is that right? The Qt validator is prepared to prevent me typing a truly illegal sequence into the widget as I go along, but it is not prepared to prevent me finishing editing with a still intermediate sequence? Then my problem is there are hundreds of such validators dotted all over the existing GUI code, with horrendousnesses to track them all down and think about changing behaviour, especially at the Dialog level but in general anyway....

In short, given my requirement --- a final "decimal" should be 1+ leading digit (optionally followed by 1 or 2 decimal places, but that's not the problem), but while editing the user should be able to delete the single leading digit to replace it with another one --- can you give me an outline (doesn't have to be specific code) as to how you would actually handle this? It seems to me I should not be the only person who requires a monetary input like this, without leaving the user lost as to how to replace the leading digit?

Thank you, people!

[VRonin]

@JNBarchan said in Validation behaviour too confusing for end user:

I can only guess that the widget lets him type intermediate entry, and then it is up to me to go recheck for "truly" Acceptable

You can easily do this on editingFinished() signal. This also applies to the default behaviour of QRegularExpressionValidator if you don't type anything after the decimal separator

---

let's try this:
(again, sorry for C++ but I have no idea how you would subclass in Python)

class LenientRegExpValidator : public QRegularExpressionValidator{
Q_OBJECT
Q_DISABLE_COPY(LenientRegExpValidator)
public:
LenientRegExpValidator(QObject* parent = Q_NULLPTR) : QRegularExpressionValidator(parent){}
LenientRegExpValidator(const QRegularExpression &re, QObject *parent = Q_NULLPTR) : QRegularExpressionValidator(re,parent){}
QValidator::State validate(QString &input, int &pos) const Q_DECL_OVERRIDE{
const auto regExpr = regularExpression();
if(regExpr.pattern().isEmpty())
return Acceptable;
const auto fullMatch = regExpr.match(input);
if (fullMatch.hasMatch())
return QValidator::Acceptable;
const auto partialMatch = regExpr.globalMatch(input, 0,QRegularExpression::PartialPreferFirstMatch);
if(partialMatch.hasNext())
return QValidator::Intermediate;
return QValidator::Invalid;
}
};

[JonB]

@VRonin

You can easily do this on editingFinished() signal.

But that's the bit I don't get! What do I do in the editingFinished handler? It's not like I can return false and then Qt would refuse leaving editing of the widget (which is exactly what I am used to in systems where a validator is evaluated on attempt to move out of editing instead of each character is typed).

On top of that, the docs state:

Note that if there is a validator() or inputMask() set on the line edit and enter/return is pressed, the editingFinished() signal will only be emitted if the input follows the inputMask() and the validator() returns QValidator::Acceptable.

So if I'm still returning Intermediate for what they've typed, that says I won't get the editingFinished signal anyway. Not that I see that matters, since I don't get what I would do in it even if I did receive it....

[aha_1980]

@JNBarchan said in Validation behaviour too confusing for end user:

@VRonin

You can easily do this on editingFinished() signal.

But that's the bit I don't get! What do I do in the editingFinished handler? It's not like I can return false and then Qt would refuse leaving editing of the widget (which is exactly what I am used to in systems where a validator is evaluated on attempt to move out of editing instead of each character is typed).

No, the QValidator works different. it forbids entering chars that lead to non-acceptable cases. for your example, it forbids entering a second . when there's already one. this is very convenient once you get used to.

On top of that, the docs state:

Note that if there is a validator() or inputMask() set on the line edit and enter/return is pressed, the editingFinished() signal will only be emitted if the input follows the inputMask() and the validator() returns QValidator::Acceptable.

So if I'm still returning Intermediate for what they've typed, that says I won't get the editingFinished signal anyway. Not that I see that matters, since I don't get what I would do in it even if I did receive it....

in editingFinished you could enable an Ok button, for example. I dont know why you would disallow moving out of the edit? you can always get focus out when the user switches to another window.

[VRonin]

@JNBarchan said in Validation behaviour too confusing for end user:

But that's the bit I don't get! What do I do in the editingFinished handler?

What I normally do is set the line edit background to red and maybe show a red QLabel to warn about the invalid output

that says I won't get the editingFinished signal anyway

Good spot. You can use textedited then

[JonB]

Thank you all for your input.

I have sat and thought about this carefully for a while. I am coming round to the conclusion that I cannot meet what I would desire.

I have no problem with the behaviours of QValidator::Acceptable & QValidator::Invalid. But I do not see how the behaviour of QValidator::Intermediate helps me/meets my requirements. (Please note: I do understand how it works, that's not the issue.)

Consider my example of a "decimal number": \d+(\.\d{1,2})?

• If user types a letter, it immediately returns Invalid and prevents the character being placed in the widget. Perfect, since that character can never be acceptable.

• What should I return if user deletes the single leading digit, in preparation for typing in a new one?

1. At present it returns Invalid, preventing the user from deleting. I have said I do not like this behaviour. So, I shall be changing over to Intermediate... but how/when?

2. If I return Intermediate instead of Invalid in all cases, this is simple. However, that allows the useless typing of a letter. It allows everything through, requiring me to check up more on completion.

3. I start to think of only returning Intermediate instead of Invalid in certain cases, of the kind outlines in @VRonin's last posted code example. That seems possible, till I consider what code I would need. I would require something like a regular expression capturing the difference between "what is potentially heading in the right direction" as opposed to "is simply unacceptable" (e..g typing a letter). @VRonin has suggested QRegularExpression::PartialPreferFirstMatch, but from what I can see in the documentation (not tested) of "partial matches" these only allow for an "incomplete match which could be satisfied by appending further characters". That will not help with the kind of situation I am thinking of, where the user perform edits "in the start/middle of the string" to get to what he wants. I would have to think out a regular expression or code for every situation I can imagine as my definition of "intermediate" to achieve this, a non-trivial task.

I can see that Qt's Intermediate may work in this sense assuming the user types linearly from left to right --- which is what it seems to be designed for --- but not in my sense. I also realise upon careful reflection how difficult it is to express just what should be Intermediate versus Invalid.

You people may be familiar & happy with how Qt validators work, but I (and my potential users) may not be attuned to its way of working. At least in my case of entering a "decimal number".

In light of the above I think I am left with "2.5" choices:

1. Make all Invalids return Intermediate instead (coding too hard to distinguish specific cases). Then I probably need to do color-marking of "intermediate-value" widgets (how would I even do this from within existing class derived from QValidator, I don't see a method to get at the widget which is being validated?). Then I definitely would need to do "final" validation on, say, Dialog "OK" or whatever, but there are 50 dialogs with say an average of 5 widgets to validate on each one, there's no central place for me to track them all down to alter code? This is why I'm thinking I'm not going to be able to use Intemediate.

2. a. Put up with exactly the current regular expression & behaviour. The user who attempted to use the widget and needed to delete the first digit to change it to another got completely stuck and had no idea what was wrong/what he needed to do :( But hey ho, you guys seem to like the behaviour :)
   b. Ask my stakeholder if I may change the validator to \d?(\.\d{1,2})? This is my preferred solution. In return for allowing the leading-delete, it will allow through .23, but provided the code can accept this it may be the simplest to resolve just this situation...

Those are my thoughts! I hope you're all fascinated :)

[VRonin]

I'm afraid there's not a generic way. In your specific case, probably a QDoubleValidator instead of a regexp one might work better.
For other cases you'd probably need to return intermediate always and reimplement QValidator::fixup to delete (or do something with it) the line if the input is invalid

[JonB]

@VRonin
Yup, thanks for your confirmation & patience.

I might have a look at QDoubleValidator, in case it internally handles my case better. Just to complicate things, I know it's not your area, but because I am Python/PyQt what I actually need to validate is that it satisfies an internal PyQt type named Decimal, which is not the same as double, and isn't even documented as to what format it parses...! :(

One final (honest!) question, inspired by something I mentioned above and your earlier comment:

What I normally do is set the line edit background to red and maybe show a red QLabel to warn about the invalid output

How (what code approach do you take) to achieve this in Qt? So far as I can see, one uses QLineEdit::setValidator() to set the validator, so a line edit can see its validator, but QValidator does not have a member to reference the QLineEdit it has been called from? And I suspect there cannot be one, as you could associate the same QValidator object with multiple QLineEdits, if you chose to do so. So when my overridden QValidator::validate() wants to return Intermediate, how can I have code there to know which widget to affect?

In the validator model I am accustomed to from another language/library, there is a one-to-one relationship between a control and its validator, so you can access one from the other. But I think that's not the case in Qt, so how do you manage it? I can only see creating a "lookup" table for each dialog so that I know the widget from the validator (assuming I stick to one-to-one), or maybe you sub-class QValidator and add a member for the associated QLineEdit, and that's going to be real messy for me....

[VRonin]

@JNBarchan said in Validation behaviour too confusing for end user:

I might have a look at QDoubleValidator, in case it internally handles my case better.

QDoubleValidator basically just uses QLocale::toDouble so should be more lenient.

How (what code approach do you take) to achieve this in Qt?

I normally change it to red when the user tries to submit a form

[JonB]

@VRonin said in Validation behaviour too confusing for end user:

How (what code approach do you take) to achieve this in Qt?

I normally change it to red when the user tries to submit a form

Sorry, this is not what I was asking. Ahh! Do you mean, you don't turn it red during QValidator::validate(), so don't access the QLineEdit from there; instead during QDialog:onOK() (or whatever it is, accept()) you enumerate each QLineEdit in the dialog and call its QLineEdit::hasAcceptableInput()? So you never try to map from validator to widget?

[VRonin]

@JNBarchan said in Validation behaviour too confusing for end user:

ou enumerate each QLineEdit in the dialog and call its QLineEdit::hasAcceptableInput()? So you never try to map from validator to widget?

Correct. The validator's job is not to take care of how an item is displayed. That's either the job of the linedit or its parent (either via connecting the textEdited() signal, using fixup() to clear invalid input and then using editingFinished() or uppon submit of form (e.g. when you press the Ok button at the end of a dialog)

[JonB]

@VRonin said in Validation behaviour too confusing for end user:

@JNBarchan said in Validation behaviour too confusing for end user:

I might have a look at QDoubleValidator, in case it internally handles my case better.

QDoubleValidator basically just uses QLocale::toDouble so should be more lenient.

Changed over to QDoubleValidator. Whether it implements via a different regular expression from the one I inherited or implements validation with dedicated code, either way it gives me/my user a more pleasant editing experience, allowing the original issue of being able to delete a lone leading digit. It also of course "feels" better.

So thank you, that will do nicely here after all this discussion (though that was worthwhile so that I now understand how Qt validators work).