Use ffmpeg with Qt6.4+

[TheEnigmist]

I want to try ffmpeg and as doc says from 6.4 ffmpeg is one of multimedia backend.
I've installed qt 6.4.0 with online installer but if I open "Media Player Example" and setting env var QT_MEDIA_BACKEND=ffmpeg as said here I get this error: could not load multimedia backend "ffmpeg".
So online installer doesn't add ffmpeg support.
As I read it is currently available as a technology preview, so my question is: how can I get this preview? How can I properly install ffmpeg with Qt to use as multimedia backend?

[Christian Ehrlicher]

@TheEnigmist said in Use ffmpeg with Qt6.4+:

So online installer doesn't add ffmpeg support.

It does provide the plugin but not all needed ffmpeg libs I would guess. What os do you use?

[TheEnigmist]

@Christian-Ehrlicher said in Use ffmpeg with Qt6.4+:

@TheEnigmist said in Use ffmpeg with Qt6.4+:

So online installer doesn't add ffmpeg support.

It does provide the plugin but not all needed ffmpeg libs I would guess. What os do you use?

Win 11 22H2, so what settings should I set to properly set ffmpeg as backend?

[Christian Ehrlicher]

"With Qt 6.4, we are adding a new, more platform independent backend based on the FFmpeg framework. The FFmpeg backend is currently available as a technology preview. This backend has the advantage that we can offer proper cross-platform support for all features of Qt Multimedia, going beyond the limitations that exist with many of the native frameworks."

Are you sure the ffmpeg backend is provided with the windows binaries?

[TheEnigmist]

@Christian-Ehrlicher said in Use ffmpeg with Qt6.4+:

"With Qt 6.4, we are adding a new, more platform independent backend based on the FFmpeg framework. The FFmpeg backend is currently available as a technology preview. This backend has the advantage that we can offer proper cross-platform support for all features of Qt Multimedia, going beyond the limitations that exist with many of the native frameworks."

Are you sure the ffmpeg backend is provided with the windows binaries?

I really don't know. I'm searching all over internet to see how get access to that "technology preview". Should I fill a form somewhere? Should I contect support?
Inside the installer there is nothing that point to FFmpeg, so I think it is not shipped with windows binaries.
I searched inside Qt installed folder for "ffmpeg", there was no result.

[Christian Ehrlicher]

@TheEnigmist said in Use ffmpeg with Qt6.4+:

I searched inside Qt installed folder for "ffmpeg", there was no result.

So you have to either use another backend or compile it by yourself.

[JonB]

@TheEnigmist
Not sure about this. I believe you showed in a previous thread that your Qt is compiled to use ffmpeg, because you set that environment variable to ffmpeg and it "worked" but errorred. Are you maybe supposed to install ffmpeg yourself rather than expecting Qt to ship with it?

[TheEnigmist]

@Christian-Ehrlicher said in Use ffmpeg with Qt6.4+:

So you have to either use another backend or compile it by yourself.

I don't know, but at least they should add in the docs how to point to ffmpeg so Qt can use it.

@JonB said in Use ffmpeg with Qt6.4+:

@TheEnigmist
Not sure about this. I believe you showed in a previous thread that your Qt is compiled to use ffmpeg, because you set that environment variable to ffmpeg and it "worked" but errorred. Are you maybe supposed to install ffmpeg yourself rather than expecting Qt to ship with it?

Well in the doc they wrote "The FFmpeg backend shipped with the Qt binary packages is configured to not contain any of the components that are available under GPLv2 only" so there is a ffmpeg inside a Qt binary, but not now. Maybe in the future?
I think we need to manually install ffmpeg and set that env var to ffmpeg so it can load properly since I think that env var is general, not specific to 6.4.

[Bonnie]

The installer definitely doesn't have any ffmpeg plugin with 6.4.0.
I've checked the 6.5.0 snapshot, there is libffmpegmediaplugin.so/.dylib in the Linux and MacOS binaries, but not in Windows.
So I think it is something just like the gstreamer plugin, it is supported in the code, but not enabled in all platforms' prebuilt binaries.

[TheEnigmist]

@Bonnie said in Use ffmpeg with Qt6.4+:

The installer definitely doesn't have any ffmpeg plugin with 6.4.0.
I've checked the 6.5.0 snapshot, there is libffmpegmediaplugin.so/.dylib in the Linux and MacOS binaries, but not in Windows.
So I think it is something just like the gstreamer plugin, it is supported in the code, but not enabled in all platforms' prebuilt binaries.

I was trying to manually build QtMultimedia, with 6.4.0 there is a lot of code about FFMpeg and there are env vars in Cmake script, you can configure and generate a Windows .sln, but it fails to build.
Btw I took a look at the code, there is something weird. Actually even if you force ffmpeg it is hardcoded to select specific platform backend "untill ffmpeg become more mature" as written in a comment LOL

I'm wondering if in future they let you access some low-level function of ffmpeg like adding a filter or choose a codec, I need only these two feature in my project, nothing more (FFmpeg + filtering + choose manually a codec if needed)

[Bonnie]

@TheEnigmist
I've successfully built QtMultimedia of 6.4.0 by MinGW, but unfortunately got crashes on runtime.
FYI, my building commands are

*Add building tools, Perl and cmake to PATH
cd build_dir
qt-cmake -G "MinGW Makefiles" PATH_TO_Qt\6.4.0\Src\qtmultimedia -DCMAKE_INCLUDE_PATH="PATH_TO_FFmpeg\include;PATH_TO_Vulkan-Headers\include" -DCMAKE_LIBRARY_PATH="PATH_TO_FFmpeg\lib"
cmake --build .

Also currently I haven't seen any low-level function supported.

[TheEnigmist]

@Bonnie Yes, I think is still in a embryonic state. Still they hardcoded to use platform specific backend even if you set ffmpeg with macro. Maybe in 6.5 we will be able to use core functionality. I will open a bug in bugreports about videos with alpha channel. Maybe they will share some info about ffmpeg (since it is the only way to have video with alpha channel in cross-platform environment)

EDIT: As I see in QtMultimedia github dev branch, there is a QFFmpegMediaPlayer, so in the beginning we will use a totally different media player based entirly on ffmpeg

[Bonnie]

I did use ffmpeg backend if setting QT_MEDIA_BACKEND=ffmpeg, just got exception so it crashed, there were ffmpeg logs in the output.
And I think QFFmpegMediaPlayer is a private class for QMediaPlayer so we are not supposed to use it directly.
Actually there are also QGstreamerMediaPlayer and QAndroidMediaPlayer, etc...

[TheEnigmist]

@Bonnie said in Use ffmpeg with Qt6.4+:

I did use ffmpeg backend if setting QT_MEDIA_BACKEND=ffmpeg, just got exception so it crashed, there were ffmpeg logs in the output.
And I think QFFmpegMediaPlayer is a private class for QMediaPlayer so we are not supposed to use it directly.
Actually there are also QGstreamerMediaPlayer and QAndroidMediaPlayer, etc...

That is a good news. I twill try to rebuild QtMultimedia, I was missing Vulkan lib. What exception did you get?
I alredsy know that some video will have a problem with alpha (ffmpeg will use vp9 instead of libvp9) and they still don't give access to low level backend so I won't be able to fix it on demand

[Bonnie]

Oops, my bad, it was not an exception but segmentation fault, caused by a null pointer in the AudioRenderer part.
All the videos I tried are only playable when not setting audio output, otherwise it will crash or freeze on the first frame.
Also when I try a hls online stream, it is not smooth.
*Edit: I've tried building with 6.5.0 snapshot today, it seems that the audio issue has already been fixed. But I was not able to play hls streams

[TheEnigmist]

@Bonnie said in Use ffmpeg with Qt6.4+:

Oops, my bad, it was not an exception but segmentation fault, caused by a null pointer in the AudioRenderer part.
All the videos I tried are only playable when not setting audio output, otherwise it will crash or freeze on the first frame.
Also when I try a hls online stream, it is not smooth.
*Edit: I've tried building with 6.5.0 snapshot today, it seems that the audio issue has already been fixed. But I was not able to play hls streams

Indeed I was writing that I was able to configure, generate and build v6.4.0 with CMake and MSVC. I edited some FFmpeg related src and reebuilt it and it reflected to my execution so it works.
What it is unexpected is that even with ffpemg videos with alpha channel has black background instead of transparency. I changed VP9 with libvpx-vp9 (online I read that VP9 doesn't handle alpha while libvpx-vp9 yes).
I don't know if it an internall bug or I should convert QVideoFrame in another format to show image as expected.

Btw since I'm now able to use ffmpeg I close this thread and mark as expected. I may open a new thread for video / audio bugs that has different topic.

For those who want to access preview of FFmpeg I used this to configure and build QtMultimedia in Win11:

• Installed Qt 6.4.0 within online installer
• Downloaded QtMultimedia src (from github)
• With CMake fill info about FFmpeg libs (avcodec, avfilter, etc.)
• With Cmkae fill info about WMF (download Windows SDK)
• I added Vulkan lib info too (don't know if could be a problem)
• Configure -> Generate -> Open .sln -> Compile ALL_BUILD
• Enojoy preview of FFmpeg in Qt