Definitive guide to building source from downloaded .run file?

Tom asso

Thanks @KH-219Design - that qmake was built during the configure step (thanks for the explanation of how 'qmake' should be used - I am still learning!)
I can see from the output of 'make' that many datavisualization and quick modules were compiled, so I don't understand why qmake is having problems...
Strange, I also see that 'make' apparently did not build qtcreator that I can find...

Thanks!

KH-219Design

Ok. I still have a slight inkling that you might be using a qmake that is other than the "ideal" one.

If you can decide what is the "outermost" enclosing directory (folder) that holds ALL you recent experimentation, then in a terminal window go to that outermost directory and run this:

find . -type f -name qmake

List back here all the qmake instances that were found. If there are more than one, then you make just be using the "wrong" one to build your project.

Tom asso

Now qmake runs without error on my application pro file. Not only do I have to directly execute the qmake that was built as part of the source download, but my $PATH must include other binary executables that were also built. So I ran 'make install' on the Qt source, which installed things in /usr/local/Qt5.14.2/bin, added that directory to PATH, and now qmake generates a Makefile.

KH-219Design

@Tom-asso said in Definitive guide to building source from downloaded .run file?:

Now qmake runs without error on my application pro file. Not only do I have to directly execute the qmake that was built as part of the source download, but my $PATH must include other binary executables that were also built. So I ran 'make install' on the Qt source, which installed things in /usr/local/Qt5.14.2/bin, added that directory to PATH, and now qmake generates a Makefile.

Hooray! Yes, that all sounds correct. Good on you for solving it, and thanks for posting the whole solution!

Tom asso

But now a new problem. After running qmake on my project .pro file, generating a Makefile, then running 'make' to build my application, my application executable ("mbgrdviz-2") does not run:

% ./mbgrdviz-2
qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, webgl.

Aborted (core dumped)

Anyone know why? I built the Qt sources with configure, make, make install. Then built my app with qmake then make. What am I missing?

KH-219Design

I think you now need this next tip (referring back again to the Linux getting started notes from my own team):

If the app fails with a message such as qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/daniel/Qt/5.14.0/gcc_64/plugins" even though it was found., you need to set QT_QPA_PLATFORM_PLUGIN_PATH to the correct location. And when you set that, you must ensure that LD_LIBRARY_PATH also points to the same Qt build.

 export QT_QPA_PLATFORM_PLUGIN_PATH="$(pwd)/build_qt_binaries/qt5_opt_install/plugins/platforms
 export LD_LIBRARY_PATH="$(pwd)/build_qt_binaries/qt5_opt_install/lib:$LD_LIBRARY_PATH"

^^ obviously you need to adjust the paths a bit, since on your setup the plugins and libs are not under "build_qt_binaries/qt5_opt_install"

Tom asso

@KH-219Design - I did 'make install' after building the source, which put stuff in /usr/local/Qt5.14.2, including in /usr/local/Qt5.14.2/ilbs and /usr/local/Qt5.14.2/plugins/platforms.

So:

% export LD_LIBRARY_PATH=/usr/local/Qt-5.14.2/lib
% export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/local/Qt-5.14.2/plugins/platforms

But my app still fails to find the xcb plugin, and I can see why because 'make' of the Qt source code didn't build an xcb plugin and it wasn't installed:

ls /usr/local/Qt-5.14.2/plugins/platforms
libqeglfs.so       libqminimal.so    libqwayland-egl.so
libqlinuxfb.so     libqoffscreen.so  libqwayland-generic.so
libqminimalegl.so  libqvnc.so        libqwebgl.so

I.e. there is no "libxcb.so" or anything like that in there, although I see there are many source files with names containing "xcb". Per this page I tried running 'configure' with the -qt-xcb option:

./configure -qt-xcb
[...]
ERROR: Feature 'xcb' was enabled, but the pre-condition 'features.thread && features.xkbcommon && libs.xcb' failed.

Not sure what that means...
I tried to manually build libxcb within the Qt source directories - that built libxcb_static.a, but then failed:

% cd qtbase/src/plugins/platforms/xcb
% qmake   # Build Makefile from xcb.pro
% make
[...]
ar cqs ../../../../../lib/libxcb-static.a .obj/xfixes.o .obj/randr.o .obj/shm.o .obj/sync.o .obj/render.o .obj/shape.o .obj/xkb.o .obj/xinerama.o .obj/xinput.o .obj/xcb_aux.o .obj/atoms.o .obj/event.o .obj/xcb_image.o .obj/keysyms.o .obj/util.o .obj/icccm.o
make[1]: Leaving directory '/home/oreilly/qt-download/qt-everywhere-src-5.14.2/qtbase/src/plugins/platforms/xcb/xcb-static'
( test -e Makefile.xcb_qpa_lib || /home/oreilly/qt-download/qt-everywhere-src-5.14.2/qtbase/bin/qmake -o Makefile.xcb_qpa_lib /home/oreilly/qt-download/qt-everywhere-src-5.14.2/qtbase/src/plugins/platforms/xcb/xcb_qpa_lib.pro ) && make -f Makefile.xcb_qpa_lib 
Project ERROR: Could not find feature xcb-xinput.
Makefile:90: recipe for target 'sub-xcb_qpa_lib-pro-make_first-ordered' failed
make: *** [sub-xcb_qpa_lib-pro-make_first-ordered] Error 3

Man, I have no idea how you guys are able to get this to work ;-)

KH-219Design

Man, I have no idea how you guys are able to get this to work ;-)

Heh. I think it is like eating spicy food. First you develop a tolerance for it gradually, then you actually start to crave and enjoy it ;)

Hmmm....

I've got 2 thoughts now.

The most direct attack would be to get further into this:

./configure -qt-xcb
[...]
ERROR: Feature 'xcb' was enabled, but the pre-condition 'features.thread && features.xkbcommon && libs.xcb' failed.

That seems to me like a good approach, and there must be more info "somewhere" about that failure. Isn't there a config.log file somewhere, as an output from the configure? Its contents should look roughly like:

... truncated ...

executing config test linuxfb
+ cd /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/linuxfb && PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /opt/repositories/fc-build-01/qt5_dbg_configuration/qtbase/bin/qmake "CONFIG -= qt debug_and_release app_bundle lib_bundle" "CONFIG += shared warn_off console single_arch" "QMAKE_CFLAGS += --sysroot=/" "QMAKE_CXXFLAGS += --sysroot=/" "QMAKE_LFLAGS += --sysroot=/" -early "CONFIG += cross_compile" /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/linuxfb
+ cd /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/linuxfb && MAKEFLAGS= /usr/bin/make
> g++ -c -pipe --sysroot=/ --sysroot=/ -g -w -fPIC  -I. -I/opt/repositories/qt5_sources/qtbase/mkspecs/linux-g++ -o main.o main.cpp
> g++ --sysroot=/ --sysroot=/ -o linuxfb main.o      
test config.qtbase_gui.tests.linuxfb succeeded
looking for library mtdev
Trying source 0 (type pkgConfig) of library mtdev ...
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --exists --silence-errors mtdev
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --modversion mtdev
> 1.1.5
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --libs-only-L mtdev
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --libs-only-l mtdev
> -lmtdev
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --cflags mtdev
+ cd /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/mtdev && PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /opt/repositories/fc-build-01/qt5_dbg_configuration/qtbase/bin/qmake "CONFIG -= qt debug_and_release app_bundle lib_bundle" "CONFIG += shared warn_off console single_arch" "QMAKE_CFLAGS += --sysroot=/" "QMAKE_CXXFLAGS += --sysroot=/" "QMAKE_LFLAGS += --sysroot=/" -early "CONFIG += cross_compile" 'QMAKE_USE += mtdev' 'QMAKE_LIBS_MTDEV = -lmtdev' /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/mtdev
+ cd /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/mtdev && MAKEFLAGS= /usr/bin/make
> g++ -c -pipe --sysroot=/ --sysroot=/ -g -w -fPIC  -I. -I/opt/repositories/qt5_sources/qtbase/mkspecs/linux-g++ -o main.o main.cpp
> g++ --sysroot=/ --sysroot=/ -o mtdev main.o   -lmtdev   
 => source accepted.
test config.qtbase_gui.libraries.mtdev succeeded
looking for library openvg
Trying source 0 (type pkgConfig) of library openvg ...
+ PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /usr/bin/pkg-config --exists --silence-errors vg
pkg-config did not find package.
  => source produced no result.
Trying source 1 (type makeSpec) of library openvg ...
VG/openvg.h not found in [] and global paths.
  => source produced no result.
test config.qtbase_gui.libraries.openvg FAILED
executing config test qpa_default_platform
test config.qtbase_gui.tests.qpa_default_platform gave result xcb
looking for library harfbuzz
Trying source 0 (type inline) of library harfbuzz ...
+ cd /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/harfbuzz && PKG_CONFIG_SYSROOT_DIR=/ PKG_CONFIG_LIBDIR=//usr/lib/pkgconfig://usr/share/pkgconfig://usr/lib/x86_64-linux-gnu/pkgconfig /opt/repositories/fc-build-01/qt5_dbg_configuration/qtbase/bin/qmake "CONFIG -= qt debug_and_release app_bundle lib_bundle" "CONFIG += shared warn_off console single_arch" "QMAKE_CFLAGS += --sysroot=/" "QMAKE_CXXFLAGS += --sysroot=/" "QMAKE_LFLAGS += --sysroot=/" -early "CONFIG += cross_compile" 'QMAKE_USE += harfbuzz' 'QMAKE_LIBS_HARFBUZZ = -lharfbuzz' /opt/repositories/fc-build-01/qt5_dbg_configuration/config.tests/harfbuzz

... truncated ...

Hopefully there would be a more precise answer in there as to exactly what steps caused "features.xkbcommon" (and the others) to fail.

A less direct attack would be:

... if we assume the configure failed because you simply need more dev headers of more cascading dependencies to be available on your machine for the compiler to use...

Then if you try this and things get installed:

sudo sed -i -- 's/#deb-src/deb-src/g' /etc/apt/sources.list
sudo sed -i -- 's/# deb-src/deb-src/g' /etc/apt/sources.list
sudo apt-get update

sudo apt-get build-dep qt5-default

Then just try ./configure -qt-xcb again (or even a bare ./configure), and this time configure should be able to detect more build dependencies on the system.

Tom asso

@KH-219Design - I will try that. But in the meantime I actually built shared library libxcb.so "by hand" (since 'make' failed) and copied it into the plugins/platforms directory:

% g++ -shared -o libxcb.so *.o
% sudo cp libxcb.so /usr/shared/Qt5.14.2/plugins/platforms/.

I then rebuilt my app and tried to run it:

% qmake
% make clean
% make
% ./mbgrdviz-2
qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in "/usr/local/Qt-5.14.2/plugins/platforms"
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs (from /usr/local/Qt-5.14.2/plugins/platforms), linuxfb (from /usr/local/Qt-5.14.2/plugins/platforms), minimal (from /usr/local/Qt-5.14.2/plugins/platforms), minimalegl (from /usr/local/Qt-5.14.2/plugins/platforms), offscreen (from /usr/local/Qt-5.14.2/plugins/platforms), vnc (from /usr/local/Qt-5.14.2/plugins/platforms), wayland-egl (from /usr/local/Qt-5.14.2/plugins/platforms), wayland (from /usr/local/Qt-5.14.2/plugins/platforms), webgl (from /usr/local/Qt-5.14.2/plugins/platforms), eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, webgl.

So although the library I built, libxcb.so, is in /usr/shared/Qt5.14.2/plugsins/platforms, it is not detected as a "plugin" - why might that be? Is there some special step to "register" a plugin?
Thanks!

Tom asso

@KH-219Design - taking the "direct attack" by looking into the config.log file... I just installed the ubuntu package for xkbcommon (apt install libxkbcommon-dev), verified presence of /usr/include/xkbcommon/xkbcommon.h. Then re-ran "./configure -qt-xcb" which again gives "ERROR: Feature 'xcb' was enabled, but the pre-condition 'features.thread && features.xkbcommon && libs.xcb' failed." I look in config.logs and see these lines:

loaded result for library config.qtbase_gui.libraries.xkbcommon
Trying source 0 (type pkgConfig) of library xkbcommon ...
+ /usr/bin/pkg-config --exists --silence-errors xkbcommon '>=' 0.5.0
pkg-config did not find package.
  => source produced no result.
Trying source 1 (type inline) of library xkbcommon ...
xkbcommon/xkbcommon.h not found in [] and global paths.
  => source produced no result.
test config.qtbase_gui.libraries.xkbcommon FAILED

So configure cannot find /usr/include/xkbcommon/xkbcommon.h - even though it exists?

KH-219Design

regarding the question about not finding a plugin... I'm not really sure, but you might try running ldd on the so you built.

if ldd produces anything like either of these, then you would be missing load-time dependencies for your so:

bad outcome #1:

ldd libQt5Gui.so.5.14.0
# this message about the version not found is bad:
./libQt5Gui.so.5.14.0: /usr/lib/x86_64-linux-gnu/libQt5Core.so.5: version `Qt_5.14' not found (required by ./libQt5Gui.so.5.14.0)
	linux-vdso.so.1 (0x00007ffe7e3e5000)
	libQt5Core.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 (0x00007f30cdac6000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f30cd8a7000)

bad outcome #2 ("not found"):

ldd libQt5Gui.so.5.14.0
	linux-vdso.so.1 (0x00007ffc16ba5000)
	libQt5Core.so.5 => not found                  # <-- anything "not found" is a problem
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f624ce6a000)
	libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1 (0x00007f624cbde000)
	libharfbuzz.so.0 => /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f624c940000)

(Note: I ran ldd on libQt5Gui only for the sake of generating sample output. I am not suggesting you ldd libQt5Gui, but rather that you ldd libxcb.so )

Regarding your results from the direct attack on configure failures:

+ /usr/bin/pkg-config --exists --silence-errors xkbcommon '>=' 0.5.0
pkg-config did not find package.

You could run /usr/bin/pkg-config --libs xkbcommon and see what you get.

You could also run apt-cache show libxkbcommon-dev and make sure the version is greater than 0.5.0

Racking my brain trying to think what else...

I guess just make sure any "left over cruft" from previous configure attempts are interfering... (but I could not enumerate what kinds of cruft you should be looking for).

Tom asso

Running pkg-config from the command line:

% /usr/bin/pkg-config --libs xkbcommon
-lxkbcommon

And "apt-cache show libxkbcommon-dev" shows version 0.8.0-1.
Thanks for all your help @KH-219Design !
I've been evaluating Qt for a seafloor-mapping project for a few months now - Qt has its share of bugs, there are many many moving pieces to it, many of those pieces barely documented or not documented at all - a very steep learning curve, at least for me. I'm grateful that you and others in the community provide invaluable help!

SGaist

What Ubuntu version are you running ?

KH-219Design

@Tom-asso said in Definitive guide to building source from downloaded .run file?:

xkbcommon.h not found in [] and global paths

I'm reading through this right now... seems like the same "xkbcommon" problem: https://forum.qt.io/topic/100908/qt-5-12-2-raspberry-pi-x11/16

KH-219Design

@KH-219Design said in Definitive guide to building source from downloaded .run file?:

seems like the same "xkbcommon" problem: https://forum.qt.io/topic/100908/qt-5-12-2-raspberry-pi-x11/16

Maybe I spoke too soon. Although the same error is mentioned, that thread seems entirely RPi specific in the end. Arggg...

KH-219Design

I also found this: https://github.com/RudolfCardinal/camcops/issues/3

... which seems to circle back to:

sudo sed -i -- 's/#deb-src/deb-src/g' /etc/apt/sources.list
sudo sed -i -- 's/# deb-src/deb-src/g' /etc/apt/sources.list
sudo apt-get update

sudo apt-get build-dep qt5-default

Has that been tried yet? I've lost track :)

Tom asso

@SGaist - I am running Ubuntu 18.04.4

Tom asso

Thanks @KH-219Design - I haven't tried the "sed" yet - what exactly will those commands do?

sudo sed -i -- 's/#deb-src/deb-src/g' /etc/apt/sources.list
sudo sed -i -- 's/# deb-src/deb-src/g' /etc/apt/sources.list
sudo apt-get update

sudo apt-get build-dep qt5-default

KH-219Design

@Tom-asso said in Definitive guide to building source from downloaded .run file?:

I haven't tried the "sed" yet - what exactly will those commands do?

The sed commands are to work around:

E: You must put some 'source' URIs in your sources.list

Described here: https://wiki.qt.io/Building_Qt_5_from_Git#Linux.2FX11

Tom asso

I'm reluctant to install the ubuntu qt5-default at this point - in the past I've run into problems when mixing ubuntu qt-related packages with source/software downloaded from download.qt.io. And I don't think I've actually seen the message "E: You must put some 'source' URIs in your sources.list".