Snapshot Testing


  • Qt Champions 2016

    Hello,

    How would you test your GUI application? From my experience, writing an automated GUI testing is very too time-consuming, and it may need a total rewrite when the requirement changes. Of coz, I won't allow my code is not tested at all. Therefore, I am always looking for a GUI testing method which is easy to write and update on requirement changes in Qt/QML.

    And I just found this method:

    Snapshot Testing · Jest

    Do you think it is a suitable testing method for Qt/QML GUI application? If this method is available in Qt, will you use it?

    I just want to brainstorm about this kind of problem. Please feel free to talk anything.


  • Qt Champions 2016

    Hi
    What do you want to test?

    We used this approach for testing configurations to be sure that the widgets would be shown and
    it works very well for that.

    However, comparing images won't tell you if a button calls wrong function or if it produce the right
    result.

    So what about the QML do you want to test for ? The backbone is tested with module tests
    so I assume its other types of test you are after ?


  • Qt Champions 2016

    @mrjj said in Snapshot Testing:

    Hi
    What do you want to test?

    Anything that may go wrong :)

    But I can't spent unlimited time to make every testable. So I usually focus on following topics:

    1. The logic and work flow behind UI
    2. Data model validation
    3. Individual pure Visual Component

    But that missed the interaction between visual components. (e.g Clicking a button , and verify the UI changes. )

    We used this approach for testing configurations to be sure that the widgets would be shown and
    it works very well for that.

    Cool. That is your own private framework or using 3rd party solution for this kind of test?

    However, comparing images won't tell you if a button calls wrong function or if it produce the right
    result.

    Exactly. But writing this kind of test is very time consuming. Just verify the side effect of clicking a button may be simple , but if it need to verify the rendered UI is another story. If that action involved multiple visual components across different pages, then the code size for simulating the environment (include mock object) will be increased too.

    It take a lot of time to write, and maintain is troublesome due to the requirement changes.

    So I usually only focus on the logic and work flow behind UI , but not integrate them . And design the visual components in a stateful way, so that the UI could refresh the data model correctly. And then I just only need to validate the data model after clicking a button.

    It could capture a lot of bug, but It is not ideal because it don't capture the bug in UI.

    So what about the QML do you want to test for ? The backbone is tested with module tests
    so I assume its other types of test you are after ?

    For example, is the UI changed correctly after clicking a button.


Log in to reply