How to use QThread to display videos
-
I am using PySide2 for interface development, and a part of it uses QLabel to display real-time videos. However, when there are other time-consuming program tasks, the real-time videos become stuttering and the latency increases. So I want to put the video display in a separate thread to avoid the influence of other programs in the main program. Can QLabel be redefined in the QThread thread and displayed in the main interface with real-time video updates?
The above method is not feasible. QLabel must be placed in the main interface. How can we solve the problem of high video frame delay? -
I am using PySide2 for interface development, and a part of it uses QLabel to display real-time videos. However, when there are other time-consuming program tasks, the real-time videos become stuttering and the latency increases. So I want to put the video display in a separate thread to avoid the influence of other programs in the main program. Can QLabel be redefined in the QThread thread and displayed in the main interface with real-time video updates?
The above method is not feasible. QLabel must be placed in the main interface. How can we solve the problem of high video frame delay?@muxing
You cannot/must not try to put aQLabel, or indeed any other UI item, in a secondary thread or attempt to access it (for read or write) from such a thread. Qt requires all UI accesses are done only from the main thread, else it might appear to work but will likely go wrong.You can use a secondary thread if you wish but only for computations, to offload them from the main/UI thread. But you must communicate between the main thread and secondary threads only via Qt signals & slots, and must be careful about sharing any data, e.g. proper locking mechanisms.
I cannot say about whatever "stuttering" you might experience. Like we said, if you have a lot of computations you could thread them, or (just so you know) you could do them in the main thread on a timer where during each timeout you only do a certain, limited number of computations and then exit to allow the main thread to do its UI updates and later pick up where you left off, so you only do "small chunks" of work at a time.
-
@muxing
You cannot/must not try to put aQLabel, or indeed any other UI item, in a secondary thread or attempt to access it (for read or write) from such a thread. Qt requires all UI accesses are done only from the main thread, else it might appear to work but will likely go wrong.You can use a secondary thread if you wish but only for computations, to offload them from the main/UI thread. But you must communicate between the main thread and secondary threads only via Qt signals & slots, and must be careful about sharing any data, e.g. proper locking mechanisms.
I cannot say about whatever "stuttering" you might experience. Like we said, if you have a lot of computations you could thread them, or (just so you know) you could do them in the main thread on a timer where during each timeout you only do a certain, limited number of computations and then exit to allow the main thread to do its UI updates and later pick up where you left off, so you only do "small chunks" of work at a time.
@JonB
Thank you for your timely and accurate response！In fact, I tried to read and process video stream information in the auxiliary thread, sending the required video frame signals (QImage) to the main thread through the connection mechanism between the signal and the slot at regular intervals, so that the main thread can update the video images that QLabel needs to display. However, there is a problem that the interval time mentioned above cannot be too small. For example, I can now send the video frame signals (QImage) every 200ms, and the images can be displayed on the main interface, but it is slow and slow, with poor effect. But if the interval time is adjusted to 160ms or less, the program will crash directly, and the main interface will display and exit after a few seconds. Therefore, this method is not feasible.
So, according to your suggestion, I can only display the video in the main program and put the time-consuming calculation tasks into the auxiliary thread. In addition, to ensure timely refreshing of the video, I should ensure that there are not too many other steps in the main program, so as not to exceed the time of video signal update for time polling. Will this also cause video lag and delay?
-
Hi,
Not to start a war on the languages but using Python for a multi threaded live video application with heavy processing might not be the best bet as currently you still have Python's GIL which might in part kill performances with regard to multi-threading.
Can you give more details about the heavy processing you are doing as well as the video characteristics ?
-
Hi,
Not to start a war on the languages but using Python for a multi threaded live video application with heavy processing might not be the best bet as currently you still have Python's GIL which might in part kill performances with regard to multi-threading.
Can you give more details about the heavy processing you are doing as well as the video characteristics ?
@SGaist Thanks for reminding me
Heavy processing involves real-time PIV analysis of images, which requires some time for acute calculations.If you wait for PIV analysis in the main thread for a few seconds, the main interface will appear unresponsive.Performing PIV analysis multiple times will reveal that the video display becomes stuttering and has high latency.The video signal was captured using a mobile phone camera and 1920X1080 in size.
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
-
Not knowing your current pipeline I can't really recommend anything.
Can you give more details ?
Are you showing the video plus some additional data ?
Did you benchmark your analysis code ?
Which framework are you using ? -
@SGaist Thanks for reminding me
Heavy processing involves real-time PIV analysis of images, which requires some time for acute calculations.If you wait for PIV analysis in the main thread for a few seconds, the main interface will appear unresponsive.Performing PIV analysis multiple times will reveal that the video display becomes stuttering and has high latency.The video signal was captured using a mobile phone camera and 1920X1080 in size.
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
@muxing The first thing to check if GPU is fully used. Another thing is to check if your display area has the same resolution as your camera 1920X1080. If not, reduce the resolution of your pipeline before the sink. For example, if the display area is 500*500, the resolution 1920X1080 of the stream can not be kept and your GPU will do something unuseful.
If PIV analysis of images takes some time, I guess you may need to cache your streaming data or buy a better GPU for quicker processing.
-
@muxing The first thing to check if GPU is fully used. Another thing is to check if your display area has the same resolution as your camera 1920X1080. If not, reduce the resolution of your pipeline before the sink. For example, if the display area is 500*500, the resolution 1920X1080 of the stream can not be kept and your GPU will do something unuseful.
If PIV analysis of images takes some time, I guess you may need to cache your streaming data or buy a better GPU for quicker processing.
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
-
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
-
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
@muxing if you've done the video processing properly, it should be an easy enough task to create multiple instances of it and process multiple frames in parallel.
As long as you do not exceed the number of your cpu/gpu cores it should improve the performance. Just make sure you pipeline the frames correctly so you can assemble them in the correct order after processing.
Are you actually using CUDA or OpenCL? if not all processing is done on the CPU anyway. So keep that in mind.
-
@muxing
If you do computations which take longer to perform than the rate at which you need to display the result you will have trouble keeping up and no delay or frame skipping, won't you?@JonB
There is not much direct relationship between calculation and video display.Because video updates are still in the main interface, what we have observed now is that even without complex calculations, there will still be delays over time. After running the program for more than two hours, the delay has reached about 8 seconds.
According to online reports, using OpenCV to read network camera videos generally results in delays. However, I think the time spent on other running parts of the program will also affect the update speed of the video screen in the main interface, and as time accumulates, the delay becomes more and more obvious.
Now I have written a main interface that only displays videos without any other functions. I will test and see how much video delay can accumulate over time to determine the impact of my other functions on video display.
-
(Side note: a lot of thread-related issues were improved in PySide6, so if possible try to use that instead of PySide2)
-
@SGaist Thanks for reminding me
Heavy processing involves real-time PIV analysis of images, which requires some time for acute calculations.If you wait for PIV analysis in the main thread for a few seconds, the main interface will appear unresponsive.Performing PIV analysis multiple times will reveal that the video display becomes stuttering and has high latency.The video signal was captured using a mobile phone camera and 1920X1080 in size.
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
-
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
@SimonSchroeder said in How to use QThread to display videos:
Python's GIL means that only a single thread is executed at a time
When I looked this up a long time ago, my limited understanding was: Python script can have multiple threads executing like other (e.g. C++) programs. But when a thread comes to execute a Python instruction (e.g. as opposed to running just Qt code) then it blocks (becomes single-threaded) against any other thread executing a Python instruction? It is the Python instruction interpreter which is effectively single-threaded, not what those threads are doing (if executing some non-Python instruction code).
-
(Side note: a lot of thread-related issues were improved in PySide6, so if possible try to use that instead of PySide2)
@CristianMaureira
I initially installed PySide6, but when using QtDesigner, it froze and I had no choice but to install PySide2 as the second best option. -
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
@SimonSchroeder
So Python is currently not very suitable for developing overly complex programs.Thank you for recommending Mojo language to me. This is my first time hearing about this new language and I look forward to it getting better and better.
-
Python is suitable, however it needs some care depending on what you do.
-
@SimonSchroeder said in How to use QThread to display videos:
Python's GIL means that only a single thread is executed at a time
When I looked this up a long time ago, my limited understanding was: Python script can have multiple threads executing like other (e.g. C++) programs. But when a thread comes to execute a Python instruction (e.g. as opposed to running just Qt code) then it blocks (becomes single-threaded) against any other thread executing a Python instruction? It is the Python instruction interpreter which is effectively single-threaded, not what those threads are doing (if executing some non-Python instruction code).
-
@muxing It's useful for a large range of applications whether on the command line or using a GUI.
It's used for automation, scientific computation, application extension, etc.
As I wrote above, you have to be careful when you code for it (as for any other language, all have limits).
As for the multi-threading part, check Python 3.13. The GIL is being removed.
