Does Python's Global Interpreter Lock (GIL) affect QThreads?
-
Hello everyone,
As far as I understand, natively python's threads are limited by the GIL. (As far as CPU bound tasks go). Which effectively limits concurrency.I was wondering is this is also true for QThreads opened in python.
@Curtwagner1984
It gets complicated. If you want more in-detail discussion I suggest you Google for e.g.qthread python gil, where there are various hits to read through.A simple summary seems to me to come in https://stackoverflow.com/a/1595690/489865
QT threads operate with the GIL released. QT threads are able to execute QT library code (and other compiled module code that does not acquire the GIL) concurrently. However, the Python code executed within the context of a QT thread still acquires the GIL, and now you have to manage two sets of logic for locking your code.
My understanding is that while a
QThreadis executing non-Python C++ code it can run concurrently with e.g. your main thread. But as soon as it drops into executing Python code it is subject to GIL throttling against wherever the main/another thread is also executing Python code. Note that (I assume) where the main/UI thread is not doing anything but waiting for the next event/signal then it does not GIL-block another thread. -
Thank you, I guess the take away is that QThreads in python effectively are throttled by the GIL because any code you run in them except their internal implementation and communication with the Qt framework would be python code.
-
Thank you, I guess the take away is that QThreads in python effectively are throttled by the GIL because any code you run in them except their internal implementation and communication with the Qt framework would be python code.
@Curtwagner1984
I think that is my interpretation. When you do drop into Python code in each one they will be GIL-throttled.There are many ways to skin a cat. But there is only one cat to skin, and you still end up with one skinned cat.
[Note: I love pussy cats, and no cats were harmed in the writing of this answer.] -
Can one use ProcessPoolExecutor to divide the cat into smaller chunks and skin each of them at the same time?
I mean, to use QThreads to avoid blocking the main thread, but if there is CPU heavy load then use ProcessPoolExecutor inside a QThread to divide the labor and avoid being blocked by the GIL?
-
Can one use ProcessPoolExecutor to divide the cat into smaller chunks and skin each of them at the same time?
I mean, to use QThreads to avoid blocking the main thread, but if there is CPU heavy load then use ProcessPoolExecutor inside a QThread to divide the labor and avoid being blocked by the GIL?
@Curtwagner1984
I am not a Python expert. I would think out what it you are trying to parallelize, then see if the GIL is a limitation.
