Display a videoframe with graphic overlay

makopo

Hi everbody,

just a conceptual beginner question: I want to create an application that shows a frame of video data in a widget. In this widget there should be something like a graphical overlay that visualizes defined areas of the frame (for example a special line or area) by mousehover. With the video data I got I created a QImage. Now my idea is to use QGraphicScene to show this frame into GUI.

Is this a good idea, also for creating overlays like line and rectangles? Or is it better to use the QOpenGLWidget for this?

Kind regards.

SGaist

Hi,

Are you going to show your video frame by frame or as a continuous stream ?

makopo

@SGaist

I want to show a single frame of the stream. For normal video playback I think the QOpenGLWidget should be the best.

SGaist

Well for the single frame you can paint on top of it before setting it on your QLabel.

That said, you could also implement that through OpenGL.

makopo

I have a further question. I create a QImage with given Data from an API. The class where I construct the image is derived from QOpenGLWidget and I override paintEvent(QPaintEvent*) to draw the function. There QPainter calls painter->drawImage(this->rect(), *m_image). But all I get is an empty widget. I have a Data class where I get the data of a current frame and where I emit a signal that send this data to Render class. As I can see on the console, transmitting the data works fine.

void RenderClass::receiveData(uchar* bufData, int outWidth, int outHeight, int bytesPerRow)
{
	m_image = new QImage(bufData, outWidth, outHeight, bytesPerRow, QImage::Format_RGB444);
}


void RenderClass::paintEvent(QPaintEvent*)
{
	QPainter* painter = new QPainter(this);
	if (m_image == nullptr) {
		qDebug() << "Image not loaded.";
	}
	else {
		painter->drawImage(this->rect(), *m_image);
		qDebug() << "paintEvent called.";
		QWidget::update();
	}
	painter->end();
}

It is possible to draw primitives with QPainter into the widget. Also loading images from disk works. I'am not shure if I construct the QImage correct.
Did someone have an advice for me?

Christian Ehrlicher

@makopo said in Display a videoframe with graphic overlay:

m_image = new QImage(bufData, outWidth, outHeight, bytesPerRow, QImage::Format_RGB444);

Please read the docs: "The buffer must remain valid throughout the life of the QImage and all copies that have not been modified or otherwise detached from the original buffer. The image does not delete the buffer at destruction. You can provide a function pointer cleanupFunction along with an extra pointer cleanupInfo that will be called when the last copy is destroyed."

QPainter* painter = new QPainter(this);#

Thsi creates a memleak for no reason - create the QPainter on the stack (and while you're at it also don't create ther QImage on the heap since there is also a memleak)

makopo

@Christian-Ehrlicher

Thanks for your advice. I found that the value of m_image in paintEvent() method is null. m_image is the initalized in the constructor. For me it look like that paintEvent can only work with local variables and not with class member variables.

void RenderClass::paintEvent(QPaintEvent*)
{
	QImage image(m_bufferData, m_frameWidth, m_frameHeight, m_bytesPerRow, QImage::Format_RGB444);
	QPainter painter(this);
	painter.drawImage(this->rect(), image);
	painter.end();

	qDebug() << "paintEvent called." << image;
	QWidget::update();
}

As decribed in the code I paste member variables to the QImage. That members get there values in in the receiveData function

Christian Ehrlicher

@makopo said in Display a videoframe with graphic overlay:

For me it look like that paintEvent can only work with local variables and not with class member variables.

For sure not except you use some custom c++ variant :D

When it's a nullptr then receiveData() is not called.

makopo

@Christian-Ehrlicher

Actually I go back to my older code where I construct a QImage in the receiveData() method.

void RenderClass::receiveData(uchar* pData, int outWidth, int outHeight, int bytesPerRow)
{
	m_image = QImage(pData, outWidth, outHeight, bytesPerRow, QImage::Format_RGB444);

	//qDebug() << "Receive Video Data" << bufData << "-" << outWidth << "-" << outHeight << "-" << bytesPerRow;
	qDebug() << "QImage" << m_image;
}


void RenderClass::paintEvent(QPaintEvent*)
{
	QPainter painter(this);
	painter.drawImage(this->rect(), m_image);
	painter.end();
	
	qDebug() << "paintEvent called." << m_image;
	QWidget::update();
}

If I print the value of m_image to the console I got the following:

paintEvent called. QImage(null)
QImage QImage(QSize(1280, 720),format=QImage::Format_RGB444,depth=16,devicePixelRatio=1,bytesPerLine=3200,sizeInBytes=2304000)
paintEvent called. QImage(null)
paintEvent called. QImage(null)
QImage QImage(QSize(1280, 720),format=QImage::Format_RGB444,depth=16,devicePixelRatio=1,bytesPerLine=3200,sizeInBytes=2304000)
paintEvent called. QImage(null)
paintEvent called. QImage(null)
paintEvent called. QImage(null)
QImage QImage(QSize(1280, 720),format=QImage::Format_RGB444,depth=16,devicePixelRatio=1,bytesPerLine=3200,sizeInBytes=2304000)
paintEvent called. QImage(null)
....
// and so on

If the program starts the paint event is called and the value is null. Thats ok because I have a button in my program that do an API-Call and that start a callback function to get the data of the video. If the button is clicked the data is received by the slot, m_image get the value of QImage. As you can see the value of m_image in paintEvent() remains QImage(null).

Christian Ehrlicher

@makopo said in Display a videoframe with graphic overlay:

Actually I go back to my older code where I construct a QImage in the receiveData() method.

Which is still wrong as stated in my first post...

Please read the docs: "The buffer must remain valid throughout the life of the QImage and all copies that have not been modified or otherwise detached from the original buffer. The image does not delete the buffer at destruction. You can provide a function pointer cleanupFunction along with an extra pointer cleanupInfo that will be called when the last copy is destroyed."

SGaist

Beside the point of @Christian-Ehrlicher about the lifetime of the data you use to build your QImage. It would be simpler to create on QImage of the right size and then memcopy the data to it.

That said, the wait you do it, you are modifying your m_image object from two different threads without any protection.