Call computationally intensive code inside QVideoFilterRunnable? Alternatives?
-
Hello everybody!
I'm new to Qt, so I'm still trying to understand which classes are the best to use in order to obtain the outcome I need.
What I need to do is the following:- grab the frames in real-time from the webcam
- call on (possibly a subset of) them a computationally intensive AI-based algorithm
- collect a bunch of 2D coordinates from the AI output
- use these coordinates to plot some lines on the frame and visualize it
Now, here's what I was thinking of doing:
- use a VideoOutput object in QML and apply my implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter I will then call my computationally intensive algorithm
- collect its output and
- use it to draw the lines on the frame
(To write on the QVideoFrame I'd extract the QVideoFrame raw data with the bits() method, use it to build a cv::mat, draw on it, and re-convert the cv:mat to QVideoFrame).
Now, reading through the QVideoFilterRunnable class documentation, I have stumbled upon this note:
Note: Avoid time consuming operations in this function as they block the entire rendering of the application.
This sounds like a showstopper to me... am I wrong?
Do you have any suggestions about what to try next?
Thanks a lot for your time!
Paolo -
Hello everybody!
I'm new to Qt, so I'm still trying to understand which classes are the best to use in order to obtain the outcome I need.
What I need to do is the following:- grab the frames in real-time from the webcam
- call on (possibly a subset of) them a computationally intensive AI-based algorithm
- collect a bunch of 2D coordinates from the AI output
- use these coordinates to plot some lines on the frame and visualize it
Now, here's what I was thinking of doing:
- use a VideoOutput object in QML and apply my implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter I will then call my computationally intensive algorithm
- collect its output and
- use it to draw the lines on the frame
(To write on the QVideoFrame I'd extract the QVideoFrame raw data with the bits() method, use it to build a cv::mat, draw on it, and re-convert the cv:mat to QVideoFrame).
Now, reading through the QVideoFilterRunnable class documentation, I have stumbled upon this note:
Note: Avoid time consuming operations in this function as they block the entire rendering of the application.
This sounds like a showstopper to me... am I wrong?
Do you have any suggestions about what to try next?
Thanks a lot for your time!
Paolo@paolo_lazer said in Call computationally intensive code inside QVideoFilterRunnable? Alternatives?:
Now, here's what I was thinking of doing:
- use a VideoOutput object in QML and apply my implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter I will then call my computationally intensive algorithm
- collect its output and
- use it to draw the lines on the frame
(To write on the QVideoFrame I'd extract the QVideoFrame raw data with the bits() method, use it to build a cv::mat, draw on it, and re-convert the cv:mat to QVideoFrame).
Instead of doing that I would do it this way:
- use a VideoOutput object in QML and apply your implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter, call asynchronously your computationally intensive algorithm
- collect its output and expose it to QML
- draw it in Qt Quick as an overlay to your image.
-
@paolo_lazer said in Call computationally intensive code inside QVideoFilterRunnable? Alternatives?:
Now, here's what I was thinking of doing:
- use a VideoOutput object in QML and apply my implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter I will then call my computationally intensive algorithm
- collect its output and
- use it to draw the lines on the frame
(To write on the QVideoFrame I'd extract the QVideoFrame raw data with the bits() method, use it to build a cv::mat, draw on it, and re-convert the cv:mat to QVideoFrame).
Instead of doing that I would do it this way:
- use a VideoOutput object in QML and apply your implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter, call asynchronously your computationally intensive algorithm
- collect its output and expose it to QML
- draw it in Qt Quick as an overlay to your image.
@GrecKo said in Call computationally intensive code inside QVideoFilterRunnable? Alternatives?:
@paolo_lazer said in Call computationally intensive code inside QVideoFilterRunnable? Alternatives?:
Now, here's what I was thinking of doing:
- use a VideoOutput object in QML and apply my implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter I will then call my computationally intensive algorithm
- collect its output and
- use it to draw the lines on the frame
(To write on the QVideoFrame I'd extract the QVideoFrame raw data with the bits() method, use it to build a cv::mat, draw on it, and re-convert the cv:mat to QVideoFrame).
Instead of doing that I would do it this way:
- use a VideoOutput object in QML and apply your implementation of the QAbstractVideoFilter interface to it
- In the run method of the filter, call asynchronously your computationally intensive algorithm
- collect its output and expose it to QML
- draw it in Qt Quick as an overlay to your image.
Hi @GrecKo , thanks a lot for your valuable suggestions, I'd definitely try to follow them!
For what concerns asynchrounous call, can I use QtConcurrent::run for it?
Thanks again for your time!
Paolo
