What is the system under test when doing Qt tests?
-
Hello,
my question is less technical but more a conceptual one.
I am a Python developer using PyQt and experienced in unit testing, also aware of concepts like test driven development, london school, detroit school, etc.But when it comes to "GUI testing" I struggle to use these concepts. I know about Human-Machine-Interface testing like it could be done with Squish or OpenHMItester where you record live actions on a GUI and repeat them later.
I am aware that there is Qt Test and I wonder how or why to use it.
Let's assume a simple example: An "About" dialog having one OK/Close button, some text and multiple tabs (e.g. authors, licences, ...) in it. What should I test here? What is the "system under test"? What is the "unit" in the meaning of a "behavior" that should be tested? What outcome should I check for (e.g. via assertEqual() or similar methods)?
A test always need to have a benefit. What can be the benefit here?
Especially in Python (as a non-static language) there is a high need to execute each line or branch of code to find problems like typos (AttributeError exceptions because of missing names) for example. This can be usefull when migrating from PyQt5 to PyQt6 where most of the migration work is about renaming Qt elements that moved to other namespaces.
This are just my current thoughts about it. Does the "About" dialog example is useful for a further discussion about how to use the unit-test-approach to Qt-code?
-
Hello,
my question is less technical but more a conceptual one.
I am a Python developer using PyQt and experienced in unit testing, also aware of concepts like test driven development, london school, detroit school, etc.But when it comes to "GUI testing" I struggle to use these concepts. I know about Human-Machine-Interface testing like it could be done with Squish or OpenHMItester where you record live actions on a GUI and repeat them later.
I am aware that there is Qt Test and I wonder how or why to use it.
Let's assume a simple example: An "About" dialog having one OK/Close button, some text and multiple tabs (e.g. authors, licences, ...) in it. What should I test here? What is the "system under test"? What is the "unit" in the meaning of a "behavior" that should be tested? What outcome should I check for (e.g. via assertEqual() or similar methods)?
A test always need to have a benefit. What can be the benefit here?
Especially in Python (as a non-static language) there is a high need to execute each line or branch of code to find problems like typos (AttributeError exceptions because of missing names) for example. This can be usefull when migrating from PyQt5 to PyQt6 where most of the migration work is about renaming Qt elements that moved to other namespaces.
This are just my current thoughts about it. Does the "About" dialog example is useful for a further discussion about how to use the unit-test-approach to Qt-code?
@buhtz Well, you can for example test whether click on a button triggers the action it should trigger (https://doc.qt.io/qt-6/qtest.html#mouseClick). This is the benefit for UI testing.
"What should I test here?" - whatever functionality you have there.
-
Hi,
Well, the About dialog might not be the best example unless you have something that is for example dynamically generated and should present things differently or maybe with different content depending on some external factor.
The system under test is exactly that. Rather than an about box, let's take a file loading workflow as an example. You have mainly two outcomes:
- The user selected a file
- The user cancelled the operation
This means that in your application you should have two branches depending on the outcomes described above. The first one might trigger the creation of a new widget, a change in the state of your application, etc. That's what you want to check. Same with outcome two, you want to ensure that for example nothing has changed when cancel was hit or maybe that your application has reset its state, etc. You can then ensure that your GUI functionality matches what you expect it should do.
Just in case,
pytest-qtis a great plugin to test Qt application written with PySide or PyQt. -
Hi,
Well, the About dialog might not be the best example unless you have something that is for example dynamically generated and should present things differently or maybe with different content depending on some external factor.
The system under test is exactly that. Rather than an about box, let's take a file loading workflow as an example. You have mainly two outcomes:
- The user selected a file
- The user cancelled the operation
This means that in your application you should have two branches depending on the outcomes described above. The first one might trigger the creation of a new widget, a change in the state of your application, etc. That's what you want to check. Same with outcome two, you want to ensure that for example nothing has changed when cancel was hit or maybe that your application has reset its state, etc. You can then ensure that your GUI functionality matches what you expect it should do.
Just in case,
pytest-qtis a great plugin to test Qt application written with PySide or PyQt.Thanks for your example. This does bring up some ligth into my understanding.
I do understand that code coverage is not everything. ;) But lets assume I want to have the about-dialog code covered by tests. How can I achieve that? What should I test?
I will also have a look at pytest-qt.
-
Thanks for your example. This does bring up some ligth into my understanding.
I do understand that code coverage is not everything. ;) But lets assume I want to have the about-dialog code covered by tests. How can I achieve that? What should I test?
I will also have a look at pytest-qt.
Trigger the action, button (all of them if it you use all of these options) that is supposed to show the dialog and then check that the dialog has been shown.
-
For the about dialog you could also check if the text of labels is what you expect. This might be more relevant for dialogs with input controls. You could check if they have the correct default values.
Maybe this sort of GUI testing is more suitable for user stories. If you have a UX designer writing up stories or workflows you could program the workflow in the way the user does it (enter value here, click here, ...) and check the end result. With a good separation of the GUI from the backend you can easily write unit tests for the backend (which doesn't rely on the GUI). In that case it is probably closer to integration testing: Does the GUI integrate with the backend.
PS: Just thought about a test for the about dialog: A lot of times it contains a copyright with a year. If the year is baked in (and not generated, e.g. with CMake, on the fly during build) you could check if the year is the current year. Or you could check version information.
-
For the about dialog you could also check if the text of labels is what you expect. This might be more relevant for dialogs with input controls. You could check if they have the correct default values.
Maybe this sort of GUI testing is more suitable for user stories. If you have a UX designer writing up stories or workflows you could program the workflow in the way the user does it (enter value here, click here, ...) and check the end result. With a good separation of the GUI from the backend you can easily write unit tests for the backend (which doesn't rely on the GUI). In that case it is probably closer to integration testing: Does the GUI integrate with the backend.
PS: Just thought about a test for the about dialog: A lot of times it contains a copyright with a year. If the year is baked in (and not generated, e.g. with CMake, on the fly during build) you could check if the year is the current year. Or you could check version information.
-
Thanks a lot for your answers. This really helped me to look over my horizont. ;)
Mhm... So it seems it is not easy to reach complete code coverage for Qt code using tests.
@buhtz said in What is the system under test when doing Qt tests?:
So it seems it is not easy to reach complete code coverage for Qt code using tests.
If it's only your code which has full coverage I don't see why it should not be possible.
