PySide2 - multi-threading cases Python crashs

wzzgdcn

The program is trying to do some work at a newly created qthread and send the data to GUI thread to show.
Refer this link manipulating-widget-in-pyside2-qthread-causes-python3-not-responding for some background.
Does anyone know this is the coding issue or an pyside2 bug?

Crash after a long run (100% reproduce, a few minutes or more, it depends) at env:
Python3.6.8 32bit， WIN10 X64, Pyside2 5.12.1

Fatal Python error: GC object already tracked

Current thread 0x000019a8 (most recent call first):
  File "C:\Python36-32\lib\ntpath.py", line 89 in join
  File "D:/project/demo.py", line 25 in list_folder
  File "D:/project/demo.py", line 29 in list_folder
  File "D:/project/demo.py", line 35 in run

Thread 0x000034b8 (most recent call first):
  File "D:/project/demo.py", line 74 in addTopItem
  File "D:/project/demo.py", line 105 in <module>

Process finished with exit code -1073740791 (0xC0000409)

Work with no problem (tested at least half an hour, try several times) at env:
Python3.6.8 32bit， WIN10 X64, PyQt5 5.12

Modify USE_PYSIDE2 = True/False to switch between pyside2 and pyqt5.

# -*- coding: utf-8 -*-
import os

USE_PYSIDE2 = True
if USE_PYSIDE2:
    from PySide2 import QtWidgets, QtGui, QtCore
    from PySide2.QtCore import Signal
else:
    from PyQt5 import QtWidgets, QtGui, QtCore
    from PyQt5.QtCore import pyqtSignal as Signal


class ShowFolderTreeThread(QtCore.QThread):
    addTopItem = Signal(str, str, tuple)

    def __init__(self, p, root_dir: str = "."):
        super().__init__(p)
        self.root_dir = root_dir

    def list_folder(self, parent_path: str, max_depth: int = 3, parent_item_ids=tuple()):
        if max_depth <= 0:
            return
        try:
            for row, content in enumerate(os.listdir(parent_path)):
                content_path = os.path.join(parent_path, content)
                is_dir: bool = os.path.isdir(content_path)
                self.addTopItem.emit(content, "Folder" if is_dir else os.path.splitext(content)[1], parent_item_ids)
                if is_dir:
                    self.list_folder(content_path, max_depth - 1, (*parent_item_ids, row))
        except Exception as e:
            print("list_folder", type(e), e)

    def run(self):
        try:
            self.list_folder(self.root_dir)
        except Exception as e:
            print(type(e), e)


class MainWindow(QtWidgets.QWidget):
    def __init__(self):
        super().__init__()
        self.treeWidget = QtWidgets.QTreeWidget(self)
        self.treeWidget.setColumnCount(2)
        self.treeWidget.setHeaderLabels(["name", "type", ])
        self.treeWidget.setColumnWidth(0, 250)

        self.pushButton = QtWidgets.QPushButton(self)
        self.pushButton.setText("&Refresh")

        layout = QtWidgets.QVBoxLayout(self)
        layout.addWidget(self.treeWidget)
        layout.addWidget(self.pushButton)
        self.setLayout(layout)

        self.resize(400, 600)

        home = os.path.expanduser('~')
        self.workThread = ShowFolderTreeThread(self, home)

        self.workThread.finished.connect(self.thread_finished)
        self.workThread.addTopItem.connect(self.addTopItem)
        self.pushButton.clicked.connect(self.start_thread)

        self.timer = QtCore.QTimer()
        self.timer.setInterval(10)
        self.timer.setSingleShot(True)
        self.timer.timeout.connect(self.pushButton.click)
        self.timer.start()

    def addTopItem(self, content: str, content_type: str, parent_item_ids: tuple):
        item = QtWidgets.QTreeWidgetItem()
        item.setText(0, content)
        item.setText(1, content_type)
        if parent_item_ids:
            index, *item_ids = parent_item_ids
            parent_item = self.treeWidget.topLevelItem(index)
            for i in item_ids:
                parent_item = parent_item.child(i)
            parent_item.addChild(item)
        else:
            self.treeWidget.addTopLevelItem(item)

    def start_thread(self):
        self.pushButton.setDisabled(True)
        self.treeWidget.clear()
        self.workThread.start()

    def thread_finished(self):
        self.pushButton.setEnabled(True)
        self.timer.start()

    def closeEvent(self, event: QtGui.QCloseEvent):
        self.timer.stop()
        super().closeEvent(event)


if __name__ == '__main__':
    import sys

    try:
        app = QtWidgets.QApplication(sys.argv)
        w = MainWindow()
        w.show()
        sys.exit(app.exec_())
    except Exception as e:
        print(type(e), e)
        raise e

Alfalfa

Just a guess, instead of subclassing QThread, perhaps it would work better with the moveToThread method.

https://gitlab.com/snippets/1732597

Godwin

You should use the default threading available as a standard python library for your threads. Its called "threading", just "threading". It's very, very simple.

walemark

The Python Global Interpreter Lock or GIL, in simple words, is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter. All the GIL does is make sure only one thread is executing Python code at a time; control still switches between threads. What the GIL prevents then, is making use of more than one CPU core or separate CPUs to run threads in parallel.

Python threading is great for creating a responsive GUI, or for handling multiple short web requests where I/O is the bottleneck more than the Python code. It is not suitable for parallelizing computationally intensive Python code, stick to the multiprocessing module for such tasks or delegate to a dedicated external library. For actual parallelization in Python, you should use the multiprocessing module to fork multiple processes that execute in parallel (due to the global interpreter lock, Python threads provide interleaving, but they are in fact executed serially, not in parallel, and are only useful when interleaving I/O operations). However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.

Holsty

Pyside2 currently does have a bug with QThread where your entire program will lag if you have a thread running. I haven't been able to make anything crash though, for me it will just not respond to the OS.
Here for the bug report, Pyside-803