Qt 5.7: QtWebEngineWidgets not working on Linux

Stefan Scherfke

Hi all,

I am currently trying to build Qt5 + QtWebEngine for PyQt5 in
a conda environment.

Most components (e.g., QtWidgets) work as expected but I have problems
getting QtWebEngine(Widgets) to work.

I will first show the errors I got and then how I built Qt5.7.

My issues

The first issue I had was that the file icudtl.dat was missing. It should
have been installed into share/qt5/resources/icudtl.dat but was not. I got
this error and then the app crashed:

Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine ICU data not found at /home/stefan/pydist/envs/qttest/bin. Trying fallback directory... The application MAY NOT work.
Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/bin/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
[0822/102244:ERROR:icu_util.cc(162)] Invalid file descriptor to ICU data received.
[0822/102244:FATAL:content_main_runner.cc(714)] Check failed: base::i18n::InitializeICU().

After I copied the file from
qt-everywhere-opensource-src-5.7.0/qtwebengine/src/3rdparty/chromium/third_party/icu/source/data/in/icudtl.dat,
I got a new errors. The app did not crash but instead of the rendered HTML
I more or less get the raw HTML (see this screenshot:
https://imgur.com/a/CKRpQ):

Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/bin/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/bin. Trying fallback directory... The application MAY NOT work.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources_100p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_resources_200p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.qttest.py/qtwebengine_devtools_resources.pak
Some features may not be available.
[0822/104459:WARNING:resource_bundle.cc(311)] locale_file_path.empty() for locale
[0822/104459:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
Installed Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/share/qt5/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /home/stefan/pydist/envs/qttest/lib/qt5/libexec/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5/resources. Trying parent directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/share/qt5. Trying application directory...
Qt WebEngine resources not found at /home/stefan/pydist/envs/qttest/lib/qt5/libexec. Trying fallback directory... The application MAY NOT work.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources_100p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_resources_200p.pak
Some features may not be available.
[0822/104459:ERROR:resource_bundle.cc(720)] Failed to load /home/stefan/.QtWebEngineProcess/qtwebengine_devtools_resources.pak
Some features may not be available.
[0822/104459:WARNING:resource_bundle.cc(311)] locale_file_path.empty() for locale

It looks again as if some required data files did not get installed, although
this time, the errors don’t lead to a crash. There’s also the error
[0822/104459:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md
for more information on developing with the sandbox on.

So my two main issues I could not yet find a solution for:

1. How to I get the required resources to be installed into share/qt5?
2. How to I fix the sandboxing issue with Chromium?

How I built Qt5

My base build system is Centos 7.

I installed the following requirements system system wide:

$ yum groupinstall 'Development Tools'
$ yum install mesa-libGL-devel libX11-devel libXext-devel libXfixes-devel libXi-devel libXrender-devel libxcb-devel
$ yum install bison flex gperf

The remaining requirements were installed as Conda packages:

fontconfig
freetype
libicu (I tried it with and without this lib as it is also bundled with the
Chromium that is bundles with Qt 5.7)
libjpeg
libmysqlclient
libpng
openssl
xz
zlib

I applied two patches to Qt5 to fix an issue with libjpeg9 and glibc:

The Conda build.sh (the build script) looks like this (it is largely based on
this recipe):

# Main variables
# --------------
BIN=$PREFIX/lib/qt5/bin
QTCONF=$BIN/qt.conf
VER=$PKG_VERSION
CXXFLAGS=-fno-delete-null-pointer-checks


# Compile
# -------
chmod +x configure

# This might only work on Linux.  Check
# https://github.com/conda/conda-recipes/tree/master/qt5 for OS X support

MAKE_JOBS=$CPU_COUNT

./configure -prefix $PREFIX \
            -libdir $PREFIX/lib \
            -bindir $PREFIX/lib/qt5/bin \
            -headerdir $PREFIX/include/qt5 \
            -archdatadir $PREFIX/lib/qt5 \
            -datadir $PREFIX/share/qt5 \
            -L $PREFIX/lib \
            -I $PREFIX/include \
            -verbose \
            -release \
            -opensource \
            -confirm-license \
            -shared \
            -nomake examples \
            -nomake tests \
            -qt-pcre \
            -qt-xcb \
            -qt-xkbcommon \
            -dbus \
            -optimized-qmake

LD_LIBRARY_PATH=$PREFIX/lib make -j $MAKE_JOBS
make install


# Post build setup
# ----------------

# Remove unneeded files
rm -rf $PREFIX/share/qt5

# Make symlinks of binaries in $BIN to $PREFIX/bin
for file in $BIN/*
do
    ln -sfv ../lib/qt5/bin/$(basename $file) $PREFIX/bin/$(basename $file)-qt5
done

# Remove static libs
rm -rf $PREFIX/lib/*.a

# Add qt.conf file to the package to make it fully relocatable
cp $RECIPE_DIR/qt.conf $BIN/

As I said before, this mostly works quite well except that the ICU data for
Qt5 / QtWebEngine is not packaged/installed and that Chromium has these
sandboxing issues.

Summary

I did a lot of research during the last two weaks and re-compiled qt5 many
times. I made some progress and got it mostly working, but two issues still
remain:

1. How to I get the required resources to be installed into share/qt5?
2. How to I fix the sandboxing issue with Chromium?

I am stucked and have not yet found any solutions for them.

Stefan Scherfke

I finally found the solution:

When QtWebEngine is compiled, it generates the required resources in

qtwebengine/src/core/Release/icudtl.dat
qtwebengine/src/core/Release/gen/repack/*.pak
qtwebengine/src/core/Release/gen/repack/qtwebengine_locales/*

The icudtl.dat and *.pak files have to be copied into share/qt5/resources/ and the qtwebengine_locales folter into share/qt5/translations/.

Stefan Scherfke

Hi,

just made some progress: The binary PyQt5-package from PyPI comes with a bundled Qt5.
If I copy the translations and resource folders from it to share/qt5/ of my
self-compiled qt5, it works as expected.

So the question is: How can I create all necessary files in share/qt5 automatically when
I build Qt5?

Stefan Scherfke

I finally found the solution:

When QtWebEngine is compiled, it generates the required resources in

qtwebengine/src/core/Release/icudtl.dat
qtwebengine/src/core/Release/gen/repack/*.pak
qtwebengine/src/core/Release/gen/repack/qtwebengine_locales/*

The icudtl.dat and *.pak files have to be copied into share/qt5/resources/ and the qtwebengine_locales folter into share/qt5/translations/.

zhouyy

mark
qtwebengine_resources.pak + qtwebengine_resources_100p.pak+ qtwebengine_resources_200p.pak