Play H.264 Elementary Stream

HeresJonny

I am receiving an H.264 elementary stream from a camera and would like to create a Qt application to play it back. I can find plenty of examples of playing back files with a standard container (i.e. MP4), but is there any way to pass in individual H.264 frames to some Qt class for it to decode and render?

If someone with knowledge of the Qt multimedia APIs could help me, I'd like to know:

If this is even possible
Which multimedia API I could use to do this
Which classes I should look at to get started
Which platforms this would actually be supported on

SGaist

Hi and welcome to devnet,

I'd recommend to also try on the interest mailing list. You'll find there Qt's developers/maintainers (this forum is more user oriented)

Vincent007

use QVideoFrame, QVideoProbe and QMediaPlayer.

HeresJonny

The documentation for QVideoFrame seems to indicate that it is used to hold the pixels from a decoded frame, whereas I would like to feed in H.264-encoded frames. I could put an external decoder in front of the QVideoFrame, but I am curious whether there is a solution for this purely within Qt.

andreyc

Qt does not know how to decode h.264.
It uses an "OS backend":http://doc.qt.io/qt-5/multimediaoverview.html#limitations to decode video.

Take a look "here":http://doc.qt.io/qt-5/videooverview.html#working-with-low-level-video-frames for low level example within Qt

[edit: updated the links SGaist]

Vincent007

QMediaPlayer cannot help you? QMediaPlayer can decode video files.

Bongani

@HeresJonny and @SGaist , Did you finally find a solution for this problem ?

SGaist

The links provided by @andreyc are a good starting point.

Bongani

@SGaist i am receiving raw h264 frames from a QTCP socket, i need a function or library where i will pass the h264 data (QByteArray) from a http payload, and decode it to pass it to a video player. The QT Multimedia library doesn't work with encoded data. i already tried using the QTAV library but there's is no luck in Reaching what i need. Please help.

SGaist

What OS are you targeting ?

Bongani

@SGaist Windows, Preferably 7 to 10.

SGaist

Do you have any h264 codec installed ?

Bongani

@SGaist Yes, unfortunately VLC-QT from https://vlc-qt.tano.si doesn't support the particular functionality i require. i also tried using the QTAV library from http://www.qtav.org, which uses the ffmpeg library. none of these are of help because they don't support raw h264 bit streaming. you can check my conversation with on of the repository admins here https://github.com/wang-bin/QtAV/issues/510. I was wondering if QT has a library i could pass my QByteArray data (raw h264 data) from a QTCPSocket and decodes the bit-stream and stream it live.

SGaist

Are you sure that FFMPEG doesn't help ? See this

Bongani

@SGaist

AVPacket avpkt; int err, frame_decoded = 0;
AVCodec *codec = avcodec_find_decoder ( AV_CODEC_ID_H264 );
AVCodecContext *codecCtx = avcodec_alloc_context3 ( codec );
avcodec_open2 ( codecCtx, codec, NULL );
// Set avpkt data and size here
err = avcodec_decode_video2 ( codecCtx, avframe, &frame_decoded, &avpkt );

avframe being the QByteArray data from the socket ?

SGaist

You'll likely have build your avframe with the data contained by the QByteArray