Efficient way to display videos

[yuvalg1987]

Hi,

I’m working with OpenCV for video processing and would like to display the processed image in a Qt based GUI (video running in real-time and a full screen display mode is an option). So far, I’ve encountered several methods that enable the conversion between OpenCV’s data structure and Qt’s. Currently, I’m using the method that seemed the simplest according to the following pseusdo-code:

cv::Mat rgbImage;
QPixmap p = QPixmap::fromImage(QImage(rgbImage.data, rgbImage.cols, rgbImage.rows, rgbImage.step, QImage::Format_RGB888));
lbl->setPixmap(p.scaled(lbl_width,lbl_height));

Looking at the code, it is quite clear that we need to allocate a QPixmap, scale it (another allocation) and then set the label (not sure what’s happening under the hood). This process seems pretty wasteful and I can also experience significant slowness when displaying the image on the entire screen (using a simple timer to trigger the display function that executes the mentioned code). I know of at least two other ways of displaying the image:

1. Using QGraphicsScene and a QGraphicsView, overwriting the draw_background method and using QPainter.drawImage in order to set the image directly from a QImage.
2. Using a standard QWidget, overwriting its paintEvent method and again using QPainter.drawImage on the QImage.

I curious to know what is the most computationally efficient way to do so. What makes a QGraphicScene different than a simple Widget? Any methods that I’ve missed?

Thanks
Yuval

[SGaist]

Hi,

Several possibilities:

• use an image format that is closer to what QPixmap used by default
• allocate a QPixmap and update its content directly from OpenCV
• let QLabel handle the resizing
• request an image of a smaller size from OpenCV
• move to OpenGL

[Kent-Dorfman]

@yuvalg1987 said in Efficient way to display videos:

Any methods that I’ve missed?

If you are on Linux/X11 then you could bypass Qt altogether and use one of the X11 visual extensions: xv, vdpau, or opengl. Anything that goes thru the Qt pipeline will be less optimized. Depends on what you need: compatibility with Qt, or direct visual access.

[yuvalg1987]

Thanks for the suggestions!

1. It is quite unclear what is exactly the format used by QPixmap, and whether the transition from QImage to QPixmap requires copying the data. In addition, is the data stored on the GPU/CPU?
2. What do you mean by move to OpenGL? I know the framework but not sure about how to use it in the context of Qt.
3. Is there a major difference between displaying using a QLabel Vs. a QSceneGraph?

Thanks
Yuval

[SGaist]

1. you can check it once it has been created. AFAIR, CPU.
2. Qt has a dedicated module for that
3. in the OP you mention QGraphicsScene and now the scene graph, which one are you looking for ?

[SimonSchroeder]

@yuvalg1987 said in Efficient way to display videos:

What do you mean by move to OpenGL?

With OpenGL you could provide the unscaled image as a texture to the GPU. When you display it in the correct size the GPU with its many cores and optimized texture interpolation hardware will handle the scaling very efficiently.

[yuvalg1987]

Hi.

By QGrphicScene I meant using the following pseudo-code :

self.graphics_view = QGraphicsView(self)
self.scene = QGraphicsScene()
self.graphics_view.setScene(self.scene)
self.scene.drawBackground = self.draw_background

then the method draw_background updates the image using QPainter.drawImage(QImage) after being called from self.scene.update(). I'm not sure what's happening in the background so I can't tell whether this is better or worse than my original implementation.

It seems that the updated OpenGL Module supports using the QPainter. did anybody tried displaying OpenCV images ? any examples?