Cannot cross-compile Qt 5.10.0 for Raspberry Pi

Lurch

I have a working setup to cross-compile Qt 5.9.3 on my PC for the Pi 2/3 without X11 on the Pi. It builds all modules, including webengine. I can cross-build all examples from the Qt sources just fine, and when I copy them to the Pi they execute without warning or error.

I tried the same setup with Qt 5.10.0, there is one bug that I can work around (bad linker command line in <build>/qtwebengine/src/core/release/host/obj/net/tools/transport_security_state_generator/transport_security_state_generator.ninja), but later (when linking libQtWebEngine) I get this:

/usr/local/src/qt5/build/qtwebengine/src/core/release/obj/ui/ozone/common/common/gl_ozone_egl.o: In function `ui::GLOzoneEGL::InitializeGLOneOffPlatform()':
gl_ozone_egl.cc:(.text._ZN2ui10GLOzoneEGL26InitializeGLOneOffPlatformEv+0x16): undefined reference to `gl::GLSurfaceEGL::InitializeOneOff(int)'
collect2: error: ld returned 1 exit status
Makefile.core_module:74: die Regel für Ziel „../../lib/libQt5WebEngineCore.so.5.10.0“ scheiterte
make[4]: *** [../../lib/libQt5WebEngineCore.so.5.10.0] Fehler 1
make[4]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src/core“ wird verlassen
Makefile:122: die Regel für Ziel „sub-core_module-pro-make_first“ scheiterte
make[3]: *** [sub-core_module-pro-make_first] Fehler 2
make[3]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src/core“ wird verlassen
Makefile:77: die Regel für Ziel „sub-core-make_first“ scheiterte
make[2]: *** [sub-core-make_first] Fehler 2
make[2]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src“ wird verlassen
Makefile:46: die Regel für Ziel „sub-src-make_first“ scheiterte
make[1]: *** [sub-src-make_first] Fehler 2
make[1]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine“ wird verlassen
Makefile:946: die Regel für Ziel „module-qtwebengine-make_first“ scheiterte
make: *** [module-qtwebengine-make_first] Fehler 2
make: Verzeichnis „/usr/local/src/qt5/build“ wird verlassen

So gl_ozone_egl.cc is using a static function gl::GLSurfaceEGL::InitializeOneOff(int) which is visible to the preprocessor but not to the linker. This looks bad...

Has Qt 5.10.0 been tested on the Pi? Where there any changes in the build requirements from Qt 5.9.3 to Qt 5.10.0?

I am using a Raspberry Pi 3 with Raspbian Stretch Lite (2017-11-29-raspbian-stretch-lite.img) and a corrected /opt/vc (by running rpi-update once). My host setup is a debian-9.3.0-i386-netinst.iso, cross-compiler is Debian's crossbuild-essential-armhf. I am building Qt for a Raspberry Pi 2 (configure option -device linux-rasp-pi2-g++).

Thanks to anyone who can help me here.

Ondrejicek

Hi,
QTBUG-65079 might be related.
I have similar issue.
Could you please provide the workaround for the bad linker?
Thank you.

sneubert

Maybe we should raise a bug report.
With our setup, same rpi and also stretch lite but cross-compiler gcc-linaro-4.9-2016.02-x86_64_arm-linux-gnueabihf and -device linux-rasp-pi3-g++ we get exactly the same linker error.

Lurch

@Ondrejicek said in Cannot cross-compile Qt 5.10.0 for Raspberry Pi:

Hi,
QTBUG-65079 might be related.
I have similar issue.
Could you please provide the workaround for the bad linker?
Thank you.

Yes, the first bug is QTBUG-65079, though on a different architecture. What I get is however very similar:

FAILED: host/transport_security_state_generator
/usr/bin/python2 "../../../../../../../../../home/cs/qt-src.git/qtwebengine/src/3rdparty/chromium/build/toolchain/gcc_link_wrapper.py" --output="host/transport_security_state_generator" -- /usr/bin/g++ -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--no-as-needed -lpthread -Wl,--as-needed -m32 -Wl,-O1 -Wl,--gc-sections -Wl,-rpath-link=host -Wl,--disable-new-dtags -L/usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf -o "host/transport_security_state_generator" -Wl,--start-group @"host/transport_security_state_generator.rsp"  -Wl,--end-group   -ldl -lpthread -lrt -lnss3 -lnssutil3 -lsmime3 -lplds4 -lplc4 -lnspr4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libpthread.so when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libpthread.a when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libdl.so when searching for -ldl
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libdl.a when searching for -ldl
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libpthread.so when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libpthread.a when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/librt.so when searching for -lrt
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/librt.a when searching for -lrt
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libnss3.so when searching for -lnss3
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libnssutil3.so when searching for -lnssutil3
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libsmime3.so when searching for -lsmime3
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libplds4.so when searching for -lplds4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libplds4.a when searching for -lplds4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libplc4.so when searching for -lplc4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libplc4.a when searching for -lplc4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libnspr4.so when searching for -lnspr4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libnspr4.a when searching for -lnspr4
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libm.so when searching for -lm
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libm.a when searching for -lm
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible /usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf/libc.a when searching for -lc
collect2: error: ld returned 1 exit status
[7/6347] CXX host/obj/third_party/protobuf/protobuf_full/type.pb.o
ninja: build stopped: subcommand failed.
Makefile.gn_run:989: die Regel für Ziel „run_ninja“ scheiterte
make[4]: *** [run_ninja] Fehler 1
make[4]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src/core“ wird verlassen
Makefile:80: die Regel für Ziel „sub-gn_run-pro-make_first“ scheiterte
make[3]: *** [sub-gn_run-pro-make_first] Fehler 2
make[3]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src/core“ wird verlassen
Makefile:77: die Regel für Ziel „sub-core-make_first“ scheiterte
make[2]: *** [sub-core-make_first] Fehler 2
make[2]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine/src“ wird verlassen
Makefile:46: die Regel für Ziel „sub-src-make_first“ scheiterte
make[1]: *** [sub-src-make_first] Fehler 2
make[1]: Verzeichnis „/usr/local/src/qt5/build/qtwebengine“ wird verlassen
Makefile:946: die Regel für Ziel „module-qtwebengine-make_first“ scheiterte
make: *** [module-qtwebengine-make_first] Fehler 2
make: Verzeichnis „/usr/local/src/qt5/build“ wird verlassen

So in order to build transport_security_state_generator, make is calling Python script gcc_link_wrapper.py with /usr/bin/g++ which is the host's "native" compiler (the one that targets it's own architecture, i386 in my case). But later in that same command line there's -L/usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf which effectively adds /usr/lib/arm-linux-gnueabihf from my armhf sysroot to the linker's search path which seems a bit silly.

Since it's obviously a host tool we're trying to build here I simply deleted that nonsensical linker option -L/usr/local/src/qt5/sysroot/usr/lib/arm-linux-gnueabihf from <build>/qtwebengine/src/core/release/host/obj/net/tools/transport_security_state_generator/transport_security_state_generator.ninja, which is the auto-generated Makefile of our build target transport_security_state_generator.

Note that transport_security_state_generator.ninja does not exist until make enters Qt's qtwebengine branch, it is auto-generated there in the beginning.

So far I didn't search for the real cause of it (somewhere buried in qtwebengine's build system), so what I do is:

run configure, then make and let it fail
edit transport_security_state_generator.ninja
run make again to continue the build

It's an admittedly ugly workaround, but it works. If I get 5.10 to build completely I might look for a better solution.

Lurch

@sneubert said in Cannot cross-compile Qt 5.10.0 for Raspberry Pi:

Maybe we should raise a bug report.
With our setup, same rpi and also stretch lite but cross-compiler gcc-linaro-4.9-2016.02-x86_64_arm-linux-gnueabihf and -device linux-rasp-pi3-g++ we get exactly the same linker error.

I just published my builder script (see https://forum.qt.io/topic/86027) which makes this clearer to reproduce.

I have quite a different setup than you (my -device is linux-rasp-pi2-g++ and my cross-toolchain is Debian's crossbuild-essential-armhf) but we get the same results which I find somewhat encouraging when it comes to fixing this, so I agree, this should be filed as a bug.

Lurch

I found a workaround for the 2nd linker bug.

At first I thought the linker is missing a function external to Qt, but it turned out that Qt cannot find its own functions.

The compact explanation: gl_ozone_egl.cc uses class gl::GLSurfaceEGL declared in gl_surface_egl.h, but the corresponding gl_surface_egl.cc is never compiled or linked into libgl_wrapper.a which goes into libQt5WebEngineCore.so.5.10.0 and thus the linker fails.

This can be shown by editing the two methods GLOzoneEGL::InitializeGLOneOffPlatform() and GLOzoneEGL::ShutdownGL() defined in in qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/ui/ozone/common/gl_ozone_egl.cc.

Open the file in your editor and comment out the uses of gl::GLSurfaceEGL::InitializeOneOff() and gl::GLSurfaceEGL::ShutdownOneOff(). Just return false in InitializeGLOneOffPlatform() for the whole function.

Now we have removed all linker dependencies to gl::GLSurfaceEGL. Run make to continue your build and it should complete now.

I cross-compiled qtwebview's minibrowser and successfully ran it on the Pi, scored 511 on http://html5test.com and played a youtoube video via HTTPS which looked totally perfect. So this workaround doesn't kill webengine!

But this is not a fix, just a first isolation of the issue. Maybe it's good enough to file this as a bug report.

So I tried to dig a bit deeper.

The class which we just commented out is declared in qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/ui/gl/gl_surface_egl.h and implemented in gl_surface_egl.cc in the same folder.

The ninja Makefile of libgl_wrapper.a is <build>/qtwebengine/src/core/release/obj/ui/gl/gl.ninja, search in it for gl_surface and see that all except for gl_surface_egl are compiled and linked here.

I addded gl_surface_egl.cc to gl.ninja and it compiled but then the linker exploded, so I do not know enough about the linker dependencies here, is this documented somewhere?

Christian

Lurch

@sneubert said in Cannot cross-compile Qt 5.10.0 for Raspberry Pi:

Maybe we should raise a bug report.

Ok, I've posted my findings regarding the first issue to QTBUG-65079, and created QTBUG-65256 for the second one after I couldn't find any issue like it.

Lurch

From what I understood, the reason for QTBUG-65079 seems to go back to missing host libraries and how pkg-config works, the tool to find compiler- and linker-command line options for installed libraries. Qtwebengine builds several tools (like gn, ninja) on the host before cross-compiling the actual library sources using these tools and your cross-toolchain.

So, qtwebengine's build pipeline uses pkg-config to search for host libraries required for these host tools. If pkg-config cannot find a library in the host's standard library paths, it uses the environment variables PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR and will pick it up from there when it finds it there, and that leads to the invalid architecture linker error.

So there's this Gerrit patch attached to QTBUG-65079, it basically separates calls to pkg-config in the two different contexts (host and target) by using the proper paths in each case (or so). Install the patch (and libpng-dev on your host, from what I understood it's required), and that error goes away.

Honestly, there are a few things beyond me here, but that patch fixes it, also the symptom that I observed (the -L in the ninja file) goes away.

So I don't know why it compiled after I applied my hotfix in that ninja file (because I never installed that missing host library until after I read about it), also how this is supposed to work without me setting the environment variables PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR to my target's sysroot before cross-compiling the Qt SDK, but maybe someone from the community can shed a bit more light on this issue.

However, there's a fix for the first bug.

Lurch

This post is deleted!