How to speed up video refreshing on QT UI?

SGaist

If in it's not from a file, where does your video come from ?

Brooks

@SGaist
The video is from a live stream over ethnet, orginating from another device. Here is the more detailed story, I am developping a video + audio commnication device, which is to send and receive video data in H.264 and audio in AAC to/from each other. My problem about video presentation performance occurs after receiving and decoding H.264 video data from the peer. Right now, if I disabled this part, the CPU usage is only 20% or less, in other words, the performance pre displaying has been tuned well enough. But when I added this part, the CPU usage would rush to 80%, which is not satisfying, for I have not added audio sending,receiving, encoding, decoding and etc.
I am thinking about underlying driver, about LCD or HDMI, maybe, just maybe, the breakthough could be there.

SGaist

What are you currently using to decode that stream ?

Brooks

@SGaist S5PV210 itself ofcuase. It has MFC( multi format codec ) support.

SGaist

I meant software wise, what are you using to decode the images ?

Brooks

@SGaist
The proccess is below,
(H.264 stream Data)-->(S5PV210's built-in decoder)--->(NV12 data)-->(ColorConversion)-->(RGB888 Data)-->QPixelMap->QLabel
First, stream data is decoded by hardware decoder built in S5PV210, which is manipulated by software APIs, demonstrated in samsung examples.( They are public on the net) The output data is NV12 data from built-in decoder.
Second, I change those NV12 data to RGB888, which is supported by Qt presentation UI.
At last, just put RGB into QPixlMap and let QLabel to refresh content by calling mylabel->setPixmap(QPixmap::fromImage(image, Qt::AutoColor));
That's it.

SGaist

Then why not build a QtMultimedia backend ?

Brooks

@SGaist
Maybe you are right, but in my dev env, this module, QtMultimedia, is not available. Now, I managed to get RGB data being written to FrameBuffer directly, with QDirectPainter and vaguely see the video refreshing. However, the video area is flickering. I guess it's caused by QT GUI thread's overwriting on my video. The following is code snippet,

void MainUiWdt::paintVideo ( )
{
.....
QDirectPainter painter( this, QDirectPainter::Reserved);
....
painter.setRegion(reservRgn);
painter.startPainting();
....
unsigned char* pBuf = (unsigned char*)painter.frameBuffer();
unsigned char* pSrc = image.bits();
for( ... )
{
for(....)
{
//B
pBuf[ dest++ ] = pSrc[ src+2 ];//
//G
pBuf[ dest++ ] = pSrc[ src+1 ];
//R
pBuf[ dest++ ] = pSrc[ src ];
//A
pBuf[ dest++ ] = 0;
}
}
}

How could reserve the target region totally, making Qt UT thread ignoring that part?
Please note, I've already used reserved mode to draw that part of frame buffer.

LuGRU

In Qt 4.x QtMultimedia is available as separate module that You can compile Yourself.
Few ideas:

why QLable? Isn't it better to do painting Yourself? I.e. QWidget::paintEvent().
acceleration will speed up, for maximising compatibility I would use OGL with GLS and have OpenGL shader (version 1.0 - for displaying it's just fine) do the hard work - I was using this approach for image correction of images of size 5k and it did work in real time on vintage laptop with IGM'a (old model from 2006 or 2007).

Brooks

@LuGRU
Probably you are right, OpenGL could be another feasible solution. Now, I solved the flickering problem by removing the last call, 'painter.endPainting' (which is not in my code snippet above), i.e, keeping that area reserved all the time and directly writting the RGB data to framebuffer. However, I encountered new problem that the picture resolution is 640x480, but the target area is 600x350, which means I need scale the original image to new dimension. Since you just mentioned open GL, may I ask, could open GL help me to scale image in such a kind of acceleration mode that cpu would not be choked?