Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Headless GUI testing with Qt?
Forum Updated to NodeBB v4.3 + New Features

Headless GUI testing with Qt?

Scheduled Pinned Locked Moved Unsolved General and Desktop
14 Posts 2 Posters 4.2k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • SGaistS Offline
    SGaistS Offline
    SGaist
    Lifetime Qt Champion
    wrote on last edited by
    #4

    Can you show an example of such a test failing ?

    I am using pytest-qt on both macOS and Linux with the offscreen backend but with widgets. There might be some different constraints for your situation.

    Interested in AI ? www.idiap.ch
    Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

    P 1 Reply Last reply
    2
    • SGaistS SGaist

      Can you show an example of such a test failing ?

      I am using pytest-qt on both macOS and Linux with the offscreen backend but with widgets. There might be some different constraints for your situation.

      P Offline
      P Offline
      patrickkidd
      wrote on last edited by
      #5

      @SGaist The common case is where an input widget like TextInput doesn’t receive focus unless the app is in the foreground so onTextChanged is never called. My own code therefore fails on an assertion when the item fails to receive focus. I’m not sure that pasting an example of that will add much to the conversation? This does not fail in all cases, but I haven’t been able to isolate the cases it does fail on.

      Are you saying that you successfully run tests using the offscreen plugin on macOS?

      https://alaskafamilysystems.com/

      1 Reply Last reply
      0
      • SGaistS Offline
        SGaistS Offline
        SGaist
        Lifetime Qt Champion
        wrote on last edited by
        #6

        Yes, that's what I am saying. But as I wrote, it's for a 100% widget application.

        An example of failing unit test would help use find what we are doing differently that may be explored to get your test more reliable.

        Interested in AI ? www.idiap.ch
        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

        P 1 Reply Last reply
        1
        • SGaistS SGaist

          Yes, that's what I am saying. But as I wrote, it's for a 100% widget application.

          An example of failing unit test would help use find what we are doing differently that may be explored to get your test more reliable.

          P Offline
          P Offline
          patrickkidd
          wrote on last edited by patrickkidd
          #7

          @SGaist Shoot, it is always so hard to pull out an example when it is an integration test. I'll see what I can do but chances are low.

          I am curious, do you just pass ['-platform, 'offscreen'] to your QApplication constructor in a fixture? I tried this for some of my QGraphicsScene|Item tests and it just froze.

          https://alaskafamilysystems.com/

          1 Reply Last reply
          0
          • SGaistS Offline
            SGaistS Offline
            SGaist
            Lifetime Qt Champion
            wrote on last edited by
            #8

            No, I am using the QT_QPA_PLATFORM environnement variable both in the CI and development machine.

            Interested in AI ? www.idiap.ch
            Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

            P 1 Reply Last reply
            2
            • SGaistS SGaist

              No, I am using the QT_QPA_PLATFORM environnement variable both in the CI and development machine.

              P Offline
              P Offline
              patrickkidd
              wrote on last edited by patrickkidd
              #9

              @SGaist said in Headless GUI testing with Qt?:

              QT_QPA_PLATFORM

              Good call with the env var. It appears that the following test passes with the default platform but fails at assert w.hasFocus() for QT_QPA_PLATFORM=offscreen.

              def test_active_window(qtbot):
                  w = QLineEdit()
                  w.show()
                  qtbot.keyClicks(w, 'here we are')
                  assert w.hasFocus()
                  assert w.text() == 'here we are'
              

              Thoughts?

              https://alaskafamilysystems.com/

              P 1 Reply Last reply
              0
              • P patrickkidd

                @SGaist said in Headless GUI testing with Qt?:

                QT_QPA_PLATFORM

                Good call with the env var. It appears that the following test passes with the default platform but fails at assert w.hasFocus() for QT_QPA_PLATFORM=offscreen.

                def test_active_window(qtbot):
                    w = QLineEdit()
                    w.show()
                    qtbot.keyClicks(w, 'here we are')
                    assert w.hasFocus()
                    assert w.text() == 'here we are'
                

                Thoughts?

                P Offline
                P Offline
                patrickkidd
                wrote on last edited by patrickkidd
                #10

                It also looks like the following test passes when QT_QPA_PLATFORM=offscreen, but fails when run together in a test suite with the previous QtWidgets-only test. The QtWidgets-only test fails in every case for QT_QPA_PLATFORM=offscreen. When QT_QPA_PLATFORM is blank, both tests pass when run both separately and when run together as a test suite. Hmmm...

                def test_qml_active_item(qtbot):
                
                    class QQW(QQuickWidget):
                        @pyqtSlot(str)
                        def textChanged(self, text):
                            print(text)
                            self._text = text
                    qml = QQW()
                    qml.rootContext().setContextProperty('qml', qml)
                    qml.setSource(QUrl.fromLocalFile('./stuff.qml'))
                    qtbot.mouseClick(qml, Qt.LeftButton, Qt.NoModifier)
                    qtbot.keyClicks(qml, 'here we are')
                    assert qml.rootObject().property('input').hasActiveFocus() == True
                    assert qml.rootObject().property('text') == 'here we are'
                    assert qml._text == 'here we are'
                

                stuff.qml:

                import QtQuick 2.12
                
                Item {
                    property var input: textInput
                    property string text: textInput.text
                
                    TextInput {
                	id: textInput
                        onTextChanged: qml.textChanged(text)
                    }
                }
                

                https://alaskafamilysystems.com/

                1 Reply Last reply
                0
                • SGaistS Offline
                  SGaistS Offline
                  SGaist
                  Lifetime Qt Champion
                  wrote on last edited by
                  #11

                  @patrickkidd said in Headless GUI testing with Qt?:

                  def test_active_window(qtbot):
                  w = QLineEdit()
                  w.show()
                  qtbot.keyClicks(w, 'here we are')
                  assert w.hasFocus()
                  assert w.text() == 'here we are'

                  if you add qtbot.waitUntil(lambda: w.hasFocus()) before your asserts, then it works as expected with both tests in a row.

                  Interested in AI ? www.idiap.ch
                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                  P 1 Reply Last reply
                  0
                  • SGaistS SGaist

                    @patrickkidd said in Headless GUI testing with Qt?:

                    def test_active_window(qtbot):
                    w = QLineEdit()
                    w.show()
                    qtbot.keyClicks(w, 'here we are')
                    assert w.hasFocus()
                    assert w.text() == 'here we are'

                    if you add qtbot.waitUntil(lambda: w.hasFocus()) before your asserts, then it works as expected with both tests in a row.

                    P Offline
                    P Offline
                    patrickkidd
                    wrote on last edited by
                    #12

                    @SGaist said in Headless GUI testing with Qt?:

                    if you add qtbot.waitUntil(lambda: w.hasFocus()) before your asserts, then it works as expected with both tests in a row.

                    Now that is very interesting. It works here too. I wonder why waiting for the first QtWidgets test to gain focus fixes the qml widget test when run as a suite?

                    https://alaskafamilysystems.com/

                    1 Reply Last reply
                    0
                    • SGaistS Offline
                      SGaistS Offline
                      SGaist
                      Lifetime Qt Champion
                      wrote on last edited by
                      #13

                      That's an pretty good question but I don't currently have any idea, sorry.

                      It might be some sort of internal state that is not updated properly.

                      Did you check by any chance if you have the same behaviour with PySide2 ?

                      Interested in AI ? www.idiap.ch
                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                      P 1 Reply Last reply
                      0
                      • SGaistS SGaist

                        That's an pretty good question but I don't currently have any idea, sorry.

                        It might be some sort of internal state that is not updated properly.

                        Did you check by any chance if you have the same behaviour with PySide2 ?

                        P Offline
                        P Offline
                        patrickkidd
                        wrote on last edited by
                        #14

                        @SGaist I’ve never used PySide2 but that would make sense. This has been a productive thread so far, I’d say. A lot of my tests work as is except my c++ engine was getting stuck waiting on a macOS queue that didn’t dispatch, and the same focus errors I originally complained about here. I have a feeling I’ll discover some problems fixing those and will leave the thread open for that.

                        https://alaskafamilysystems.com/

                        1 Reply Last reply
                        1

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved