How to test with QTest that slot is called?

adamhendry

Does anyone know how to assert that a slot is called by a signal? Specifically, I use PyQt5 and have a QToolButton. When it it clicked (i.e. its signal triggered is emitted), it calls a slot function my_func.

I know the function gets called because it works when I run the GUI, but I need to develop automated testing for CI/CD. I tried using pytest-qt (which is a simple python wrapper around QTest) and mocking the call to the function, but I cannot seem to get it to work.

Any help would be appreciated!

Link to my StackOverflow Question: SO question

Relevant Code

@patch.object(myproj.View, 'create_project', autospec=True, spec_set=True)
def test_make_project(create_project_mock: Any, app: MainApp, qtbot: QtBot):
    """Test when New button clicked that project is created if no project is open.

    Args:
        create_project_mock (Any): A ``MagicMock`` for ``View._create_project`` method
        app (MainApp): (fixture) The ``PyQt`` main application
        qtbot (QtBot): (fixture) A bot that imitates user interaction
    """
    # Arrange
    window = app.view

    toolbar = window.toolbar
    new_action = window.new_action
    new_button = toolbar.widgetForAction(new_action)

    qtbot.addWidget(toolbar)
    qtbot.addWidget(new_button)

    # Act
    qtbot.wait(10)  # In non-headless mode, give time for previous test to finish
    qtbot.mouseMove(new_button)
    qtbot.mousePress(new_button, QtCore.Qt.LeftButton)
    qtbot.waitSignal(new_button.triggered)

    # Assert
    assert create_project_mock.called

adamhendry

There is an important subtlety I overlooked. The python docs state that you must

patch where an object is looked up

but it should really be read

patch where YOU look up the object

I don't call create_project directly in my code (Qt does this under the hood). So, it isn't a good candidate for patching. The rule is:

only mock code you own/can change

"Growing Object-Oriented Software, Guided by Tests" by Steve Freeman, Nat Pryce

NB: You can mock a 3rd-party library method, but only when you call it in your code. Otherwise, the test will be brittle because it will break when the 3rd-party implementation changes.

Instead, we can use another test-double: a fake. This can be used to override create_project and we can exploit QtCore.QObject.sender() to get information about the caller and assert on it.

Lastly, it should be noted that it is easier to manually trigger the action in this test rather than use GUI automation tools like pytest-qt to trigger the action. Instead, you should create a separate test that uses pytest-qt to push the button and assert that the trigger signal is emitted.

def test_make_project(app: main.MainApp):
    """Test when ``New`` action is triggered that ``create_project`` is called.

    ``New`` can be triggered either from the menubar or the toolbar.

    Args:
        app (MainApp): (fixture) The ``PyQt`` main application
    """
    class ViewFake(view.View):
        def create_project(self):
            assert self.sender() is self.new_action

    app.view = ViewFake(controller=app)

    window = app.view
    new_action = window.new_action

    new_action.trigger()

SGaist

Hi,

Shouldn't you use mouseClick ? A press is not the same thing.

adamhendry

Hi @SGaist ,

Yes, it should say mouseClick. I've simplified the test by manually triggering new_action, but still have the same problem:

Test

@patch.object(myproj.View, 'create_project', autospec=True, spec_set=True)
def test_make_project(create_project_mock: Any, app: MainApp, qtbot: QtBot):
    """Test when ``New`` action is triggered that ``create_project`` is called.

    ``New`` can be triggered either from the menubar or the toolbar.

    Args:
        create_project_mock (Any): A ``MagicMock`` for ``View.create_project`` method
        app (MainApp): (fixture) The ``PyQt`` main application
        qtbot (QtBot): (fixture) A bot that imitates user interaction
    """
    # Arrange
    window = app.view

    new_action = window.new_action

    def check_create_project():
        assert create_project_mock.called

    # Act
    new_action.trigger()

    # Assert
    qtbot.waitUntil(check_create_project)

I know the function gets called because I added a toy print statement

myproj/view.py

def create_project(self):
        """Creates a new project."""
        print('hiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii')

and run pytest -s and see it in the output

stdout sanity check

===================================================================================================================== test session starts ======================================================================================================================
platform win32 -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
PyQt5 5.15.6 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
Using --randomly-seed=1234
rootdir: %USERPROFILE%\myproj, configfile: pyproject.toml, testpaths: tests
plugins: hypothesis-6.46.10, cov-3.0.0, doctestplus-0.12.0, env-0.6.2, forked-1.4.0, memprof-0.2.0, mock-3.7.0, qt-4.0.2, randomly-3.12.0, xdist-2.5.0, typeguard-2.13.3
collected 1 item
run-last-failure: rerun previous 1 failure (skipped 4 files)
                                                                                                                                                                                                                                                                     
tests\test_view.py hiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
F

but the mock still does not get called:

Error Message

    def check_create_project():
>       assert create_project_mock.create_project.called
E       AssertionError: assert False
E        +  where False = <MagicMock name='View.create_project' id='2173966999264'>.called
E        +    where <MagicMock name='View.create_project' id='2173966999264'> = <MagicMock name='View' id='2173917178032'>.create_project

SGaist

What does "create_project" do ?

How do you setup your mock ?

adamhendry

What does "create_project" do ?

I'm doing TDD, so right now nothing. It could do anything. I want to assert that the right function gets called when I trigger my action.

How do you setup your mock ?

Using python's built-in unittest.mock.patch method as a decorator:

@patch.object(myproj.View, 'create_project', autospec=True, spec_set=True)
def test_make_project(create_project_mock: Any, app: MainApp, qtbot: QtBot):
   ...

I know in python I am supposed to patch where the method is called, not where it is defined. However, I don't know who calls create_project since that seems to be handled by Qt.

SGaist

Then there's the need to have a bit of architecture done.

The name of the method suggests that something should be generated. Your test should then check that the stuff was generated correctly.

As for the mock part, that one depends on how your class is used in your application. Since you are testing not the class itself but its use, you should replace the object where it's created.

adamhendry

Here is the MRE:

Directory Structure

myproj/
├─myproj/
│ ├─main.py
│ ├─view.py
│ └──__init__.py
└─tests/
  ├─conftest.py
  ├─test_view.py
  └─__init__.py

The __init__.py files are empty.

`main.py`

"""Myproj main application (controller)."""
import myproj.view as view

class MainApp:
    def __init__(self) -> None:
        """Myproj GUI controller."""
        self.view = view.View(controller=self)

    def show(self) -> None:
        """Display the main window."""
        self.view.showMaximized()

`view.py`

"""Graphic front-end for Myproj GUI."""
from typing import TYPE_CHECKING

from pyvistaqt import MainWindow  # type: ignore
from qtpy import QtCore, QtGui, QtWidgets

if TYPE_CHECKING:
    from emmy.main import MainApp

class View(MainWindow):

    def __init__(
        self,
        controller: 'MainApp',
    ) -> None:
        """Display GUI main window.

        Args:
            controller (MainApp): The application controller, in the model-view-controller (MVC)
                framework sense
        """
        super().__init__()
        self.controller = controller
        self.setWindowTitle('Myproj')

        self.container = QtWidgets.QFrame()

        self.layout_ = QtWidgets.QVBoxLayout()

        self.container.setLayout(self.layout_)
        self.setCentralWidget(self.container)

        self._create_actions()
        self._create_menubar()
        self._create_toolbar()

    def _create_actions(self) -> None:
        """Create QAction items for menu- and toolbar."""
        self.new_action = QtWidgets.QAction(
            '&New Project...',
            self,
        )
        self.new_action.setShortcut('Ctrl+N')
        self.new_action.setStatusTip('Create a new project...')
        self.new_action.triggered.connect(self.create_project)

    def _create_menubar(self) -> None:
        """Create the main menubar."""
        self.menubar = self.menuBar()
        self.file_menu = self.menubar.addMenu('&File')
        self.file_menu.addAction(self.new_action)

    def _create_toolbar(self) -> None:
        """Create the main toolbar."""
        self.toolbar = QtWidgets.QToolBar('Main Toolbar')
        self.addToolBar(self.toolbar)
        self.toolbar.addAction(self.new_action)

    def create_project(self):
        """Creates a new project."""
        print('hiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii')

`conftest.py`

"""Configuration file for pytest tests."""
from typing import Generator, Sequence, Union

import pytest
from pytestqt.qtbot import QtBot  # type: ignore
from qtpy import QtCore

import myproj.main as main

# Register plugins to use in testing
pytest_plugins: Union[str, Sequence[str]] = [
    'pytestqt.qtbot',
]


@pytest.fixture(autouse=True)
def clear_settings() -> Generator[None, None, None]:
    """Fixture to clear ``Qt`` settings."""
    yield
    QtCore.QSettings().clear()

@pytest.fixture(name='app', scope='function')
def fixture_app(qtbot: QtBot) -> Generator[main.MainApp, None, None]:
    """``pytest`` fixture for ``Qt``.

    Args:
        qtbot (QtBot): pytest fixture for Qt

    Yields:
        Generator[QtBot, None, None]: Generator that yields QtBot fixtures
    """
    # Setup
    root = main.MainApp()
    # When in windows mode (non-headless), window must have focus. This is
    # important so tester does not click away from window and accidentally
    # fail test.
    root.view.setWindowFlags(
        QtCore.Qt.Window | QtCore.Qt.CustomizeWindowHint | QtCore.Qt.WindowStaysOnTopHint
    )
    root.show()

    qtbot.addWidget(root.view)

    # Run
    yield root

    # Teardown - None

`test_view.py`

"""Tests for ``myproj.view`` module."""
from typing import Any
from unittest.mock import patch

import pytest
from pytestqt.qtbot import QtBot
from qtpy import QtCore, QtWidgets

import myproj.main as main
import myproj.view as view

@patch.object(myproj.View, 'create_project', autospec=True, spec_set=True)
def test_make_project(create_project_mock: Any, app: MainApp, qtbot: QtBot):
    """Test when ``New`` action is triggered that ``create_project`` is called.

    ``New`` can be triggered either from the menubar or the toolbar.

    Args:
        create_project_mock (Any): A ``MagicMock`` for ``View.create_project`` method
        app (MainApp): (fixture) The ``PyQt`` main application
        qtbot (QtBot): (fixture) A bot that imitates user interaction
    """
    # Arrange
    window = app.view

    new_action = window.new_action

    def check_create_project():
        assert create_project_mock.called

    # Act
    new_action.trigger()

    # Assert
    qtbot.waitUntil(check_create_project)

adamhendry

@SGaist

you should replace the object where it's created.

I've provided the MRE in the post (I can only post 1x/10 minutes). I believe the object is created in my fixture app in conftest.py. Is that correct? If so, am I patching incorrectly?

The name of the method suggests that something should be generated

My question is more general because I want to test create_project separately. Let's assume, e.g., that it's called connect_to_database. I don't want to call that directly and there wouldn't be a GUI aspect to the function. I can test the functionality of connect_to_database in a separate test. The only aspect I want to test now is that the right function gets called when the action is triggered.

SGaist

Check this article about the integration of mock in pytest.

You'll see how to mock a method.

You can then set an object variable to true check that the method was called.

adamhendry

You can then set an object variable to true check that the method was called.

Forgive me, I have read the article, but I am lost. Can you show me how you would make the MRE work?

adamhendry

There is an important subtlety I overlooked. The python docs state that you must

patch where an object is looked up

but it should really be read

patch where YOU look up the object

I don't call create_project directly in my code (Qt does this under the hood). So, it isn't a good candidate for patching. The rule is:

only mock code you own/can change

"Growing Object-Oriented Software, Guided by Tests" by Steve Freeman, Nat Pryce

NB: You can mock a 3rd-party library method, but only when you call it in your code. Otherwise, the test will be brittle because it will break when the 3rd-party implementation changes.

Instead, we can use another test-double: a fake. This can be used to override create_project and we can exploit QtCore.QObject.sender() to get information about the caller and assert on it.

Lastly, it should be noted that it is easier to manually trigger the action in this test rather than use GUI automation tools like pytest-qt to trigger the action. Instead, you should create a separate test that uses pytest-qt to push the button and assert that the trigger signal is emitted.

def test_make_project(app: main.MainApp):
    """Test when ``New`` action is triggered that ``create_project`` is called.

    ``New`` can be triggered either from the menubar or the toolbar.

    Args:
        app (MainApp): (fixture) The ``PyQt`` main application
    """
    class ViewFake(view.View):
        def create_project(self):
            assert self.sender() is self.new_action

    app.view = ViewFake(controller=app)

    window = app.view
    new_action = window.new_action

    new_action.trigger()

adamhendry

This post is deleted!

adamhendry

@SGaist said in How to test with QTest that slot is called?:

Shouldn't you use mouseClick ? A press is not the same thing.

I also want to note that I originally did this for the following reason:

By default, QWidgets only receive mouse move events when at least one mouse button is pressed while the mouse is being moved. Using setMouseTracking to True should enable all events, but this does not seem to work in headless mode (i.e. with QT_QPA_PLATFORM=offscreen).

Hence, I tried to use mouseMove and mousePress instead.

See: https://doc.qt.io/qtforpython-5/PySide2/QtWidgets/QWidget.html#PySide2.QtWidgets.PySide2.QtWidgets.QWidget.setMouseTracking  # pylint: disable=line-too-long

Please also note that it appears mousePress and mouseClick in QTest are switched (i.e. mousePress behaves like mouseClick and vice-versa), even when setMouseTracking is True. I experience this behavior, but please reference this SO post for reference.

SGaist

I haven't had issues triggering slots through button click simulation even when using the offscreen backend. The most mocking I had to do was "answering" QInputMessage dialogs the way the test was validating either the yes or no code paths.