Play H.264 Elementary Stream

[HeresJonny]

I am receiving an H.264 elementary stream from a camera and would like to create a Qt application to play it back. I can find plenty of examples of playing back files with a standard container (i.e. MP4), but is there any way to pass in individual H.264 frames to some Qt class for it to decode and render?

If someone with knowledge of the Qt multimedia APIs could help me, I'd like to know:

1. If this is even possible
2. Which multimedia API I could use to do this
3. Which classes I should look at to get started
4. Which platforms this would actually be supported on

[SGaist]

Hi and welcome to devnet,

I'd recommend to also try on the interest mailing list. You'll find there Qt's developers/maintainers (this forum is more user oriented)

[Vincent007]

use QVideoFrame, QVideoProbe and QMediaPlayer.

[HeresJonny]

The documentation for QVideoFrame seems to indicate that it is used to hold the pixels from a decoded frame, whereas I would like to feed in H.264-encoded frames. I could put an external decoder in front of the QVideoFrame, but I am curious whether there is a solution for this purely within Qt.

[andreyc]

Qt does not know how to decode h.264.
It uses an "OS backend":http://doc.qt.io/qt-5/multimediaoverview.html#limitations to decode video.

Take a look "here":http://doc.qt.io/qt-5/videooverview.html#working-with-low-level-video-frames for low level example within Qt

[edit: updated the links SGaist]

[Vincent007]

QMediaPlayer cannot help you? QMediaPlayer can decode video files.

[Bongani]

@HeresJonny and @SGaist , Did you finally find a solution for this problem ?

[SGaist]

The links provided by @andreyc are a good starting point.

[Bongani]

@SGaist i am receiving raw h264 frames from a QTCP socket, i need a function or library where i will pass the h264 data (QByteArray) from a http payload, and decode it to pass it to a video player. The QT Multimedia library doesn't work with encoded data. i already tried using the QTAV library but there's is no luck in Reaching what i need. Please help.

[SGaist]

What OS are you targeting ?

[Bongani]

@SGaist Windows, Preferably 7 to 10.

[SGaist]

Do you have any h264 codec installed ?

[Bongani]

@SGaist Yes, unfortunately VLC-QT from https://vlc-qt.tano.si doesn't support the particular functionality i require. i also tried using the QTAV library from http://www.qtav.org, which uses the ffmpeg library. none of these are of help because they don't support raw h264 bit streaming. you can check my conversation with on of the repository admins here https://github.com/wang-bin/QtAV/issues/510. I was wondering if QT has a library i could pass my QByteArray data (raw h264 data) from a QTCPSocket and decodes the bit-stream and stream it live.

[SGaist]

Are you sure that FFMPEG doesn't help ? See this

[Bongani]

@SGaist

AVPacket avpkt; int err, frame_decoded = 0;
AVCodec *codec = avcodec_find_decoder ( AV_CODEC_ID_H264 );
AVCodecContext *codecCtx = avcodec_alloc_context3 ( codec );
avcodec_open2 ( codecCtx, codec, NULL );
// Set avpkt data and size here
err = avcodec_decode_video2 ( codecCtx, avframe, &frame_decoded, &avpkt );

avframe being the QByteArray data from the socket ?

[SGaist]

You'll likely have build your avframe with the data contained by the QByteArray