QApplication holds all threads for ~10s on intel macs

mliberato-pax

I'm experiencing problems with the QApplication exec function effectively locking out all threads running on all of our Intel-based macbooks. Interestingly I cannot replicate this problem on an M* computer (as far as I can tell). The problem seems very similar to https://forum.qt.io/topic/94657/small-example-showing-qtimer-slowing-to-a-halt/16, but is slightly different as it's not a QTimer but rather python's threading module that's held. I can provide more specific details on computers that fail if that's useful.

I'm using PyQt, and am currently using python 3.11.6, but have confirmed that using 3.9.18 also fails. I've tried and seen failures with both PyQt5 (5.15.10) and PyQt6 (6.6.1). Happy to try any configuration of PyQt/python version, but from what I've tested the problem seems pretty persistent across versions.

"""Simple test script to show problem"""
import logging
import sys
import threading
import time

from PyQt5 import QtWidgets

app = QtWidgets.QApplication(sys.argv)
logger = logging.getLogger(__name__)


def run() -> None:
    """Check the heartbeat, print hi"""
    _prev_time = time.time()
    while True:
        time.sleep(0.2)
        curr = time.time()
        logger.warning(curr - _prev_time)
        if curr - _prev_time > 0.5:
            raise RuntimeError('Heartbeat failure')

        _prev_time = curr


thread = threading.Thread(target=run)
thread.start()
widget = QtWidgets.QWidget()
widget.show()
app.exec()

The failure looks like...

> python test.py
0.32659006118774414
0.20141005516052246
...
0.20452380180358887
0.20130419731140137
10.20060110092163
Exception in thread Thread-1 (run):
Traceback (most recent call last):
  File ".../threading.py", line 1045, in _bootstrap_inner
    self.run()
  File ".../threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File ".../test.py", line 21, in run
    raise RuntimeError('Heartbeat failure')
RuntimeError: Heartbeat failure

Indicating the last loop had a 10s lockup, and then eventually resumed. This 10s lockup isn't feasible in our application, as we are driving a test with the background threads that changes state on the order of seconds. Originally thought this was a problem with QThreads/our handling of that, but was able to strip down to a single background thread still freezing with the main application execution, leading us to believe it's something within exec. I couldn't find what exec is doing, but a potential thought I had was moving that out of the application .exec execution, and into my own thread so I can dig into what locks.

I've tried a similar test of the python core threading module and haven't had issues, so it seems to be connected to this application layer locking up. Happy to try any thoughts, really just looking for ideas to unblock/understand why our program is locking

ChrisW67

@mliberato-pax said in QApplication holds all threads for ~10s on intel macs:

I've tried a similar test of the python core threading module and haven't had issues, so it seems to be connected to this application layer locking up.

I am confused. Your example code's do-nothing thread is using the Python threading and logging modules and you are simultaneously saying it does not work and it does.

You do not say what is happening during this 10 second period.

mliberato-pax

@ChrisW67 apologies, my explanation of the error wasn’t clear. The do nothing thread is intended to show the error —it runs correctly at the 200ms tick for 200-300 cycles (shown initially in the test output) and then is locked out and there’s a 10s period where nothing runs (shown as the last line of output). I am trying to understand what within the application could cause this 10s lockup, as my thread should be able to run continuously.

Discovering what is going on during this 10 second period and preventing it from impacting other threads is the goal.

ChrisW67

OK, your code runs perfectly fine for more than 1000 cycles through the thread loop for me: Ubuntu Linux, Python 3.10.12, PyQt 5, Qt 5.15.3.
Looks like a platform quirk. I do not have a Mac.

The log output is going to my console. Is that the case with you, or is there a logging daemon involved?
Could Mac OS be deciding to hibernate the application?

Does it behave differently if the GUI is active:

"""Simple test script to show problem"""
import logging
import sys
import threading
import time

from PyQt5 import QtCore, QtWidgets

app = QtWidgets.QApplication(sys.argv)
logger = logging.getLogger(__name__)

def run() -> None:
    """Check the heartbeat, print hi"""
    count = 0
    _prev_time = time.time()
    while True:
        time.sleep(0.2)
        curr = time.time()
        if count % 100 == 0 :
            logger.warning(count)
        logger.warning(curr - _prev_time)
        if curr - _prev_time > 0.5:
            raise RuntimeError('Heartbeat failure')

        _prev_time = curr
        count = count + 1


thread = threading.Thread(target=run)
thread.start()
widget = QtWidgets.QSpinBox()
widget.setValue(0)

timer = QtCore.QTimer()
timer.timeout.connect(lambda: widget.setValue(widget.value() + 1))
timer.setInterval(1000);
timer.start()

widget.show()
app.exec()

mliberato-pax

@ChrisW67 Seemed to have found the solution!! The key was found here -- macOS was throttling with App Nap. Solved by installing appnope, and including

import appnope
appnope.nope()

at the top of any PyQt application.