How to use QThread to display videos
-
Not knowing your current pipeline I can't really recommend anything.
Can you give more details ?
Are you showing the video plus some additional data ?
Did you benchmark your analysis code ?
Which framework are you using ? -
@SGaist Thanks for reminding me
Heavy processing involves real-time PIV analysis of images, which requires some time for acute calculations.If you wait for PIV analysis in the main thread for a few seconds, the main interface will appear unresponsive.Performing PIV analysis multiple times will reveal that the video display becomes stuttering and has high latency.The video signal was captured using a mobile phone camera and 1920X1080 in size.
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
@muxing The first thing to check if GPU is fully used. Another thing is to check if your display area has the same resolution as your camera 1920X1080. If not, reduce the resolution of your pipeline before the sink. For example, if the display area is 500*500, the resolution 1920X1080 of the stream can not be kept and your GPU will do something unuseful.
If PIV analysis of images takes some time, I guess you may need to cache your streaming data or buy a better GPU for quicker processing.
-
@muxing The first thing to check if GPU is fully used. Another thing is to check if your display area has the same resolution as your camera 1920X1080. If not, reduce the resolution of your pipeline before the sink. For example, if the display area is 500*500, the resolution 1920X1080 of the stream can not be kept and your GPU will do something unuseful.
If PIV analysis of images takes some time, I guess you may need to cache your streaming data or buy a better GPU for quicker processing.
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
-
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
-
@JoeCFD
I have implemented real-time video display using the OpenCV module in the main program. However, later on, the PIV lamp calculation and analysis function was added, and it was found that performing PIV analysis and calculation would affect the display of the video. Therefore, I wanted to put the video in a separate thread.The good news is that I have successfully used threads to display the video today. The bad news is that I have also put the PIV analysis and calculation function into another separate thread, but when running the PIV calculation and analysis, there will still be some delay in the video, but the video display is continuous and not lagging.
So when there are time-consuming tasks, will video inevitably experience delays?
How to solve the problem of video delay?
@muxing if you've done the video processing properly, it should be an easy enough task to create multiple instances of it and process multiple frames in parallel.
As long as you do not exceed the number of your cpu/gpu cores it should improve the performance. Just make sure you pipeline the frames correctly so you can assemble them in the correct order after processing.
Are you actually using CUDA or OpenCL? if not all processing is done on the CPU anyway. So keep that in mind.
-
@muxing
If you do computations which take longer to perform than the rate at which you need to display the result you will have trouble keeping up and no delay or frame skipping, won't you?@JonB
There is not much direct relationship between calculation and video display.Because video updates are still in the main interface, what we have observed now is that even without complex calculations, there will still be delays over time. After running the program for more than two hours, the delay has reached about 8 seconds.
According to online reports, using OpenCV to read network camera videos generally results in delays. However, I think the time spent on other running parts of the program will also affect the update speed of the video screen in the main interface, and as time accumulates, the delay becomes more and more obvious.
Now I have written a main interface that only displays videos without any other functions. I will test and see how much video delay can accumulate over time to determine the impact of my other functions on video display.
-
(Side note: a lot of thread-related issues were improved in PySide6, so if possible try to use that instead of PySide2)
-
@SGaist Thanks for reminding me
Heavy processing involves real-time PIV analysis of images, which requires some time for acute calculations.If you wait for PIV analysis in the main thread for a few seconds, the main interface will appear unresponsive.Performing PIV analysis multiple times will reveal that the video display becomes stuttering and has high latency.The video signal was captured using a mobile phone camera and 1920X1080 in size.
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
-
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
@SimonSchroeder said in How to use QThread to display videos:
Python's GIL means that only a single thread is executed at a time
When I looked this up a long time ago, my limited understanding was: Python script can have multiple threads executing like other (e.g. C++) programs. But when a thread comes to execute a Python instruction (e.g. as opposed to running just Qt code) then it blocks (becomes single-threaded) against any other thread executing a Python instruction? It is the Python instruction interpreter which is effectively single-threaded, not what those threads are doing (if executing some non-Python instruction code).
-
(Side note: a lot of thread-related issues were improved in PySide6, so if possible try to use that instead of PySide2)
@CristianMaureira
I initially installed PySide6, but when using QtDesigner, it froze and I had no choice but to install PySide2 as the second best option. -
@muxing said in How to use QThread to display videos:
Due to the Python‘s GIL, even if I put complex calculation parts into auxiliary threads, the video displayed on the main interface may still be delayed and stuttered? If I use C++language for development, can auxiliary threads solve this problem?
Python's GIL means that only a single thread is executed at a time (there's currently an effort to remove the GIL, but you have to compile Python yourself to get it). You don't get real multithreading. This means that C++ would solve your problem. Almost any other language will solve your problem (I am not aware of any other language having something comparable to the GIL). In the future you might be able to use Mojo which aims to be fully compatible with Python syntax (eventually).
@SimonSchroeder
So Python is currently not very suitable for developing overly complex programs.Thank you for recommending Mojo language to me. This is my first time hearing about this new language and I look forward to it getting better and better.
-
Python is suitable, however it needs some care depending on what you do.
-
@SimonSchroeder said in How to use QThread to display videos:
Python's GIL means that only a single thread is executed at a time
When I looked this up a long time ago, my limited understanding was: Python script can have multiple threads executing like other (e.g. C++) programs. But when a thread comes to execute a Python instruction (e.g. as opposed to running just Qt code) then it blocks (becomes single-threaded) against any other thread executing a Python instruction? It is the Python instruction interpreter which is effectively single-threaded, not what those threads are doing (if executing some non-Python instruction code).
-
@muxing It's useful for a large range of applications whether on the command line or using a GUI.
It's used for automation, scientific computation, application extension, etc.
As I wrote above, you have to be careful when you code for it (as for any other language, all have limits).
As for the multi-threading part, check Python 3.13. The GIL is being removed.
