Can't build WebEngine on linux with proprietary codecs

Kobid

Hi,

I'm searching how to build QWebEngine on linux. What I'm doing so far:

In command line go to /qt/5.14.1/Src/qtwebengine
Run:

qmake WEBENGINE_CONFIG+=use_proprietary_codecs

Result

/home/prog/qt/5.14.1/Src/qtwebengine/qtwebengine.pro:1: Unknown test function: qtHaveModule
WARNING: /home/prog/qt/5.14.1/Src/qtwebengine/qtwebengine.pro:2: Unable to find file for inclusion qt_parts
WARNING: /home/prog/qt/5.14.1/Src/qtwebengine/qtwebengine.pro:3: Unable to find file for inclusion functions

Tried also open qtwebengine.pro and comment these lines:

requires(qtHaveModule(gui))
load(qt_parts)
load(functions)

Now, "qmake WEBENGINE_CONFIG+=use_proprietary_codecs" seems to work fine (this commandline just not return anything). And after make:

/home/prog/qt/5.14.1/Src/qtwebengine$ qmake WEBENGINE_CONFIG+=use_proprietary_codecs
/home/prog/qt/5.14.1/Src/qtwebengine$ make
g++ -m64 -Wl,-O1 -o qtwebengine     -L/usr/lib/x86_64-linux-gnu -lQtGui -lQtCore -lpthread 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
Makefile:102: recipe for target 'qtwebengine' failed
make: *** [qtwebengine] Error 1

Could someone help me? I have official Qt 5.14.1 installed by Qt Maintenance Tool

UPDATE
Just read somewhere that I have to use qmake from Qt 5.14.1, not qmake from my OS. This command:

../../gcc_64/bin/qmake WEBENGINE_CONFIG+=use_proprietary_codecs

Result with:

Info: creating cache file /home/prog/qt/5.14.1/Src/qtwebengine/.qmake.cache

Running configuration tests...
Checking for architecture supported... yes
Checking for bison... no
Checking for python2... /usr/bin/python2
Checking for gperf... no
Checking for flex... no
Checking for host pkg-config... /usr/bin/pkg-config
Checking for jumbo build merge limit... 8
Checking for platform supported... 
Checking for build path without whitespace... yes
Checking for x11... yes
Checking for libdrm... yes
Checking for xcomposite... no
Checking for submodule initialized... yes
Checking for d-bus... no
Checking for fontconfig... yes
Checking for freetype >= 2.4.2... yes
Checking for glib-2.0 >= 2.32.0... yes
Checking for glibc > 2.16... yes
Checking for jsoncpp... no
Checking for khr... yes
Checking for lcms2... no
Checking for libevent... no
Checking for libvpx... no
Checking for libwebp, libwebpmux and libwebpdemux... no
Checking for compatible libxml2 and libxslt... no
Checking for minizip... no
Checking for system ninja... no
Checking for nss >= 3.26... no
Checking for opus... no
Checking for protobuf... no
Checking for re2... no
Checking for snappy... no
Checking for xcursor... no
Checking for xi... no
Checking for xtst... no
Checking for zlib... yes
Done running configuration tests.

Configure summary:

Qt WebEngine Build Tools:
  Use System Ninja ....................... no
  Use System Gn .......................... no
  Jumbo Build Merge Limit ................ 8
  Developer build ........................ no
  QtWebEngine required system libraries:
    fontconfig ........................... yes
    dbus ................................. no
    nss .................................. no
    khr .................................. yes
    glibc ................................ yes
  QtWebEngine required system libraries for qpa-xcb:
    x11 .................................. yes
    libdrm ............................... yes
    xcomposite ........................... no
    xcursor .............................. no
    xi ................................... no
    xtst ................................. no
  Optional system libraries used:
    re2 .................................. no
    icu .................................. no
    libwebp, libwebpmux and libwebpdemux . no
    opus ................................. no
    ffmpeg ............................... no
    libvpx ............................... no
    snappy ............................... no
    glib ................................. yes
    zlib ................................. yes
    minizip .............................. no
    libevent ............................. no
    jsoncpp .............................. no
    protobuf ............................. no
    libxml2 and libxslt .................. no
    lcms2 ................................ no
    png .................................. no
    JPEG ................................. no
    harfbuzz ............................. no
    freetype ............................. yes

Note: The following modules are not being compiled in this configuration:
    webenginecore
    webengine
    webenginewidgets

WARNING: Tool gperf is required to build QtWebEngine.

WARNING: QtWebEngine will not be built.

WARNING: QtPdf will not be built.

Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.
Qt will be installed into '/home/dibo/programowanie/qt/5.14.1/gcc_64'.

Prior to reconfiguration, make sure you remove any leftovers from
the previous build

That is looking promising but something is missing in my prereqs. Investigating....

UPDATE 2
I have installed GPerf tool but why qmake doesn't recognize it? It is still saying that "WARNING: Tool gperf is required to build QtWebEngine." and output is smaller. It doesn't contain all "Checking for..." lines

jsulm

@Kobid said in Can't build WebEngine on linux with proprietary codecs:

I have installed GPerf

Did you install GPerf dev package also?
You can get more information about exact problem looking into configure.log file.

Kobid

Yes. Removed and installed Qt again and now it is fine. Seems like something wrong with qmake cache. But even now after fresh installation, first qmake is showing:

Running configuration tests...
Checking for architecture supported... yes
Checking for bison... no
Checking for python2... /usr/bin/python2
....

But second qmake call is without above. It show only configure summary

Kobid

@jsulm I was finally able to turn on proprietary codecs. Seems that this command doesn't work in qtwebengine directory :
../../gcc_64/bin/qmake WEBENGINE_CONFIG+=use_proprietary_codecs
With this command:
../../gcc_64/bin/qmake -- -webengine-proprietary-codecs
... I finally got expected result:

Running configuration tests...
Checking for architecture supported... yes
Checking for bison... yes
Checking for python2... /usr/bin/python2
Checking for gperf... yes
Checking for flex... yes
Checking for submodule initialized... yes
Checking for build path without whitespace... yes
Checking for platform supported... 
Checking for x11... yes
Checking for libdrm... yes
Checking for xcomposite... yes
Checking for xcursor... yes
Checking for xi... yes
Checking for xtst... yes
Checking for host pkg-config... /usr/bin/pkg-config
Checking for jumbo build merge limit... 8
Checking for d-bus... yes
Checking for fontconfig... yes
Checking for freetype >= 2.4.2... yes
Checking for glib-2.0 >= 2.32.0... yes
Checking for glibc > 2.16... yes
Checking for jsoncpp... no
Checking for khr... yes
Checking for lcms2... no
Checking for libevent... no
Checking for libvpx... no
Checking for libwebp, libwebpmux and libwebpdemux... no
Checking for compatible libxml2 and libxslt... no
Checking for minizip... no
Checking for system ninja... yes
Checking for nss >= 3.26... yes
Checking for opus... no
Checking for protobuf... no
Checking for re2... no
Checking for snappy... no
Checking for zlib... yes
Checking for alsa... no
Checking for embedded build... no
Checking for linker supports -z noexecstack... yes
Checking for poppler-cpp... no
Checking for pulseaudio >= 0.9.10... yes
Done running configuration tests.

Configure summary:

Qt WebEngine Build Tools:
  Use System Ninja ....................... yes
  Use System Gn .......................... no
  Jumbo Build Merge Limit ................ 8
  Developer build ........................ no
  QtWebEngine required system libraries:
    fontconfig ........................... yes
    dbus ................................. yes
    nss .................................. yes
    khr .................................. yes
    glibc ................................ yes
  QtWebEngine required system libraries for qpa-xcb:
    x11 .................................. yes
    libdrm ............................... yes
    xcomposite ........................... yes
    xcursor .............................. yes
    xi ................................... yes
    xtst ................................. yes
  Optional system libraries used:
    re2 .................................. no
    icu .................................. no
    libwebp, libwebpmux and libwebpdemux . no
    opus ................................. no
    ffmpeg ............................... no
    libvpx ............................... no
    snappy ............................... no
    glib ................................. yes
    zlib ................................. yes
    minizip .............................. no
    libevent ............................. no
    jsoncpp .............................. no
    protobuf ............................. no
    libxml2 and libxslt .................. no
    lcms2 ................................ no
    png .................................. no
    JPEG ................................. no
    harfbuzz ............................. no
    freetype ............................. yes
Qt WebEngineCore:
  Embedded build ......................... no
  Full debug information ................. no
  Pepper Plugins ......................... yes
  Printing and PDF ....................... yes
  Proprietary Codecs ..................... yes
  Spellchecker ........................... yes
  Native Spellchecker .................... no
  WebRTC ................................. yes
  Geolocation ............................ yes
  WebChannel support ..................... yes
  Use v8 snapshot ........................ yes
  Kerberos Authentication ................ no
  Extensions ............................. yes
  Support qpa-xcb ........................ yes
  Use ALSA ............................... no
  Use PulseAudio ......................... yes
Qt WebEngineQml:
  Support Qt WebEngine Qml ............... yes
  UI Delegates ........................... yes
  Test Support ........................... no
Qt WebEngineWidgets:
  Support Qt WebEngine Widgets ........... yes

Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.

But after make :

ninja: build stopped: subcommand failed.
Makefile.gn_run:568: recipe for target 'run_ninja' failed
make[3]: *** [run_ninja] Error 1
make[3]: Opuszczenie katalogu '/home/prog/qt/5.14.1/Src/qtwebengine/src/core'
Makefile:82: recipe for target 'sub-gn_run-pro-make_first' failed
make[2]: *** [sub-gn_run-pro-make_first] Error 2
make[2]: Opuszczenie katalogu '/home/prog/qt/5.14.1/Src/qtwebengine/src/core'
Makefile:77: recipe for target 'sub-core-make_first' failed
make[1]: *** [sub-core-make_first] Error 2
make[1]: Opuszczenie katalogu '/home/prog/qt/5.14.1/Src/qtwebengine/src'
Makefile:48: recipe for target 'sub-src-make_first' failed
make: *** [sub-src-make_first] Error 2

That was a big disappointment after almost one hour of compiling :(
I don't understand few things. On what basis it detected that Proprietary Codecs = yes? Did it found H264 in my Linux KDE Neon? For example from Chrome / Brave / Opera browser? I planned to use openh264.so which I have compiled in my download folder, nothing officialy installed in linux libs dir. Only libwidevinecdmadapter.so and libwidevinecdm.so is in /opt/google/chrome. I don't understand what must be statically linked and what can be loaded dynamically. I don't have license for H264. I just need QWebEngine working with DRM for commercial use

Kobid

Finally solved my problem. Compilation error and also I'm able to play Spotify and Netflix. Here is solution if someone has same problems.

First. Solution for ninja: build stopped: subcommand failed. Not sure what exactly helped here. Googled for answers and found 3 suggestions. I used all at once because compiling webengine is taking hours and I didn't have such time to test each solution:

First I installed all dependencies described here (even additional dependencies):
https://wiki.qt.io/QtWebEngine/How_to_Try
I found that someone has similar error and problem was with RAM and not enough of swap space. Note that compiling webengine consume a lot of RAM and CPU. I have Intel i7, 16GB RAM and 512GB SSD, I thought that problem doesn't apply to me. However, when I bought my laptop, I tweaked my system to use swapiness as low as possible. My value was cat /proc/sys/vm/swappiness => 1. I changed it to 60 which is default for linux. When compiling I didn't notice swap usage but RAM usage was between 9-12GB at the end.
Someone also suggested use MAKEFLAGS="-j2". So in my home .profile file I added this lines:

MAKEFLAGS="-j2"
export MAKE="make $MAKEFLAGS"
alias make="make $MAKEFLAGS"

After such changes I run make -j2 in qtwebengine and after 5 hours of compilation I finally got my own libwQt5Webengine*.so libs
Now, after run your browser you can go to https://html5test.com and see if H.264 and DRM is enabled. Even if I got "Yes" for both, I still wasn't able to play Spotify even if I had libwidevinecdmadapter.so and libwidevinecdm.so in /opt/google/chrome. I read somewhere that widevine plugin must be for specific chromium version. QtWebengine in Qt 5.14.1 is based on chromium 77.0.3865.129. So I started to looking for it. Could not found exactly version but downloaded 76.0.3809.100 from here:
https://www.slimjet.com/chrome/google-chrome-old-version.php
When you unpack .deb package, you find libwidevinecdm.so. Copy it into /opt/google/chrome didn't work for me. I found that correct dir is /usr/lib/chromium. I created this dir and copied libwidevinecdm.so. Also run sudo chmod 644 /usr/lib/chromium/libwidevinecdm.so.
Rebuild your app (better clean up it well) and run. You should be able to play Spotify and Netflix.

Final notes:

Seems that libwidevinecdmadapter.so is not needed. I read somewhere that it is for pepper plugin only but don't know if it is true
Don't know which H.264 codec is used, probably the patented one installed in my linux system because libopenh264.so is not installed anywhere. I have it compiled in my download folder so I'm pretty sure that webengine didn't find it there. What I want to test is make sure that my browser is using libopenh264.so instead of licensed H.264. I hope that it doesn't require rebuild QtWebEngine again.
/usr/lib/chromium/ is not an option for me. I have to find way to link this lib in my app dir so it will be easy to deploy