Cross-compile on WSL without building Qt from source

AnttiK

Hi all,

What I am trying to achieve is cross-compiling a simple QtWidget-application on an x86-64 platform (the host) for an emulated aarch64 target. Note that the target is not a Raspberry Pi, or any other single board computer. I am simply trying to get the cross-compile process to work, and run it via emulation.

I am aware of the various guides floating in the Internet, and on the Qt Group website regarding how to cross-compile the entire Qt from sources to a given target platform. I would like to avoid that since in my case both the host and target architecture libraries and tools are readily available from the package manager. There doesn't seem to be any rational reason for building everything from source.

What I have done so far:

Windows 11 host system, running x86-64
WSL (ver 2.2.4.0, WSLg 1.0.61, Kernel version 5.15.153.1-2)
Ubuntu 22.04 distribution installed on WSL

I first installed the necessary packages for creating "native" Qt applications:

qtdeclarative5-dev:amd64
g++-10:amd64
make:amd64

After this I created the symlinks /usr/bin/gcc and /usr/bin/g++. I was then able to create a simple Qt application, invoke qmake -project to create a project file and qmake and make to build it. The binary runs fine, and shows a simple main window with a label widget inside it (hooray to the insidious RDP-wiring in WSL).

I then made the Ubuntu system "arm64 aware" with dpkg --add-architecture arm64 and made the necessary modifications to the apt sources list. After this I was able to install the next set of tools for cross-compilation:

qtdeclarative5-dev:arm64
g++-10-aarch64-linux-gnu:amd64

I then created the necessary symlinks /usr/bin/aarch64-linux-gnu-gcc and /usr/bin/aarch64-linux-gnu-g++ and attempted to invoke qmake with the undocumented xspec parameter: qmake -xspec linux-aarch64-gnu-g++.

qmake generated a Makefile for me which uses the correct cross-compiler but the INCPATH and LIBS variables in the Makefile point to /usr/include/x86_64-linux-gnu/qt5 and /usr/lib/x86_64-linux-gnu. Needless to say if I invoke make the source file is cross-compiled properly but the linker stage falls flat on its face, trying to combine an aarch64 binary with an amd64 library.

I tried a brute-force hack by replacing the INCPATH and LIBS variables in the Makefile, and saving it but apparently on each invocation of make the original qmake command is re-invoked, and overwrites my hand-made changes in the Makefile. Logical, yes, but also quite frustrating.

How can I coax qmake to generating a Makefile that'd use INCPATH /usr/include/aarch64-linux-gnu/qt5/ and LIBS /usr/lib/aarch64-linux-gnu/?

My final goal in this little exercise is to run the cross-compiled executable using qemu-user.

Or is there potentially a flaw in my approach? Can there be a conflict between the C/C++ standard library that the cross-compiler uses and the standard (or other) libraries that the arm64 Qt5 libraries have originally been compiled against?

AnttiK

Thanks @SimonSchroeder, your hint led me down the right track. The rabbit hole was deep. Very, very deep.

It seems when I install qtdeclarative5-dev:arm64 I get not only the include and library directories mentioned in my first post but also a shell script into /usr/bin folder with the name aarch64-linux-gnu-qmake.

By default my PATH environment variable has /usr/bin folder and in there the qmake executable points to qtchooser which, in turn, points to the /usr/lib/x86_64-linux-gnu/qt5/bin folder which, again in turn has a symlink file qmake which points to /usr/bin/x86_64-linux-gnu-qmake which is a shell script that contains just the right incantations to produce x86-64 binaries.

Based on this discovery I checked if a file such as /usr/bin/aarch64-linux-gnu-qmake exists and, much to my surprise, it did.

Long story short, to cross-compile with ready-made packages you invoke /usr/bin/aarch64-linux-gnu-qmake -spec linux-aarch64-gnu-g++ <project file> directly. This results in a Makefile which uses not only the right cross-compiler but also sets the INCPATH and LIBS variables correctly.

Tested it, and checked the final executable with readelf. It was an aarch64 binary alright.

AnttiK

As a side-track I also attempted the cross-compilation using cmake and had much, much better success.

After creating an initial CMakeLists.txt and an aarch64-linux-gnu.toolchain.cmake file I was able to create a Makefile with cmake -DCMAKE_TOOLCHAIN_FILE=aarch64-linux-gnu.toolchain.cmake and build it with make.

The resulting executable ran correctly with qemu-aarch64.

I continued then on this journey to run the same executable on real aarch64 hardware. After copying the executable over via SCP I ran it, only to face the error

./qt-test: /lib/aarch64-linux-gnu/libc.so.6: version 'GLIBC_2.34' not found (required by ./qt-test)

I found out that the target platform only had libc version 2.31 so the error itself seemed clear. I solved the situation by installing the previous version of Ubuntu (20.04) to WSL, and retracing my steps. The new executable built with Ubuntu 20.04 ran correctly on the target hardware.

For reference, here's the final "Bill of Materials" for cross-compiling a simple Qt application using Ubuntu 20.04 on x86-64 host for Debian Bullseye on aarch64:

Install and update WSL to latest version
Install Ubuntu-20.04 on WSL, and update its packages to latest versions
Install apt install g++-10-aarch64-linux-gnu cmake g++- (cross-compile tools)
Enable arm64 architecture (dpkg --add-architecture arm64) and specify apt sources (see attachment)
Install apt install qtdeclarative5-dev:arm64
Create CMakeLists.txt and CMake toolchain file (see attachment)
Invoke cmake with the toolchain file, and finally make

The original question still remains unanswered, though. How to coax qmake to do my bidding?

SimonSchroeder

@AnttiK said in Cross-compile on WSL without building Qt from source:

How can I coax qmake to generating a Makefile

Even though you have found an answer using CMake, I'd like to give a hint what to do when using qmake. You just have to use the correct qmake. Each Qt installation has its own qmake and will use the includes and libs from the same path. So, you'd have to explicitly call the qmake located in the installation path for arm64 cross compilation.

AnttiK