Qt libraries not found is the executable is copied from another machine

Mark81

I'm facing this odd behavior.

Machine 1
Ubuntu 22.04, Qt 6.4.0, QtCreator 8.0.1
I create an empty widget project, name Untitled1.

Machine 2
Ubuntu 22.04, Qt 6.4.0, QtCreator 8.0.1
I create an empty widget project, name Untitled2.

Both files are identically in size (1078792 bytes).

I copy Untitled1 on Machine 2.
I get:

$ ldd untitled1
	linux-vdso.so.1 (0x00007ffe7a0ff000)
	libQt6Widgets.so.6 => not found
	libQt6Core.so.6 => not found
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9af49ed000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9af49cd000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9af47a5000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9af46bc000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9af4c36000)

$ ldd untitled2
	linux-vdso.so.1 (0x00007fffda7da000)
	libQt6Widgets.so.6 => /home/user/Qt/6.4.0/gcc_64/lib/libQt6Widgets.so.6 (0x00007f2fe2050000)
	libQt6Core.so.6 => /home/user/Qt/6.4.0/gcc_64/lib/libQt6Core.so.6 (0x00007f2fe19d2000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f2fe1794000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f2fe1774000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2fe154c000)
	libQt6Gui.so.6 => /home/user/Qt/6.4.0/gcc_64/lib/libQt6Gui.so.6 (0x00007f2fe0bac000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007f2fe0b25000)
	libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007f2fe0ade000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2fe0ad9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f2fe09f2000)
	libicui18n.so.56 => /home/user/Qt/6.4.0/gcc_64/lib/libicui18n.so.56 (0x00007f2fe0400000)
	libicuuc.so.56 => /home/user/Qt/6.4.0/gcc_64/lib/libicuuc.so.56 (0x00007f2fe0000000)
	libicudata.so.56 => /home/user/Qt/6.4.0/gcc_64/lib/libicudata.so.56 (0x00007f2fde600000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2fe09eb000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f2fe08b1000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2fe03e4000)
	libgthread-2.0.so.0 => /lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007f2fe08ac000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f2fe08a5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2fe2788000)
	libEGL.so.1 => /lib/x86_64-linux-gnu/libEGL.so.1 (0x00007f2fe03d1000)
	libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f2fde5b6000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f2fde476000)
	libQt6DBus.so.6 => /home/user/Qt/6.4.0/gcc_64/lib/libQt6DBus.so.6 (0x00007f2fde3b2000)
	libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f2fde2ea000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f2fde232000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f2fde1fe000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f2fde188000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f2fde157000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f2fe03c8000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f2fde12d000)
	libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007f2fde0df000)
	libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f2fde0a4000)
	libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f2fe03ba000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f2fe0899000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f2fdfff8000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007f2fddfdd000)
	libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f2fddfba000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f2fddfa2000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f2fddf77000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f2fddea8000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f2fdde88000)
	libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f2fdffed000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f2fddd4a000)
	libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x00007f2fddd3d000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f2fddd17000)

In other words, the binary file compiled on the same Machine (i.e. untitled2 on Machine 2) finds the libraries, but the binary built on a twin machine with the very same tools does not.
And it's true also in the opposite way.

I don't understand why this happens. If the paths are configured it should find the libraries in both cases!
What am I missing here?

jsulm

@Mark81 If you take a closer look at what you posted you will see the difference.
In short: on one machine you're using Qt provided by the distribution on another one you use Qt installed using Qt Online Installer...

JonB

@Mark81
One quick thought to eliminate: is untitled2 being run from the same directory as when you run untitled1? In particular, you aren't sitting in /home/user/Qt/6.4.0/gcc_64/lib when you run untitled2 but not for untitled1, are you?

JonB

@jsulm
Your answer crossed with mine, you are doubtless correct.

There is one thing I have never understood: how does the runtime know to look in /home/user/Qt/6.4.0 or a system directory or whatever? There isn't an environment variable for this, is there? Soft links? What?

jsulm

@JonB said in Qt libraries not found is the executable is copied from another machine:

how does the runtime know to look in /home/user/Qt/6.4.0 or a system directory or whatever?

This is done during build/linking, the binary contains this information. This is also the reason why you can't simply move your Qt installation to a different location (installed via online installer).

Mark81

@jsulm said in Qt libraries not found is the executable is copied from another machine:

@Mark81 If you take a closer look at what you posted you will see the difference.
In short: on one machine you're using Qt provided by the distribution on another one you use Qt installed using Qt Online Installer...

I'm sorry but I don't get this.
In both machines I have installed Qt only with the online installed. To check:

$ dpkg -l | grep qt
ii  libqt5core5a:amd64                         5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 core module
ii  libqt5dbus5:amd64                          5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 D-Bus module
ii  libqt5gui5:amd64                           5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 GUI module
ii  libqt5network5:amd64                       5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 network module
ii  libqt5opengl5:amd64                        5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 OpenGL module
ii  libqt5printsupport5:amd64                  5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 print support module
ii  libqt5svg5:amd64                           5.15.3-1                                          amd64        Qt 5 SVG module
ii  libqt5widgets5:amd64                       5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 widgets module
ii  libqt5x11extras5:amd64                     5.15.3-1                                          amd64        Qt 5 X11 extras
ii  qt5-gtk-platformtheme:amd64                5.15.3+dfsg-2ubuntu0.2                            amd64        Qt 5 GTK+ 3 platform theme
ii  qttranslations5-l10n                       5.15.3-1                                          all          translations for Qt 5
ii  vlc-plugin-qt:amd64                        3.0.16-1build7                                    amd64        multimedia player and streamer (Qt plugin)

There are no Qt6 libs installed.
What I can see above is one executable finds the libraries in the correct place (the installation dir) the other finds nothing. I don't understand how you can say it finds the libraries from the distribution.

Mark81

@jsulm said in Qt libraries not found is the executable is copied from another machine:

This is done during build/linking, the binary contains this information. This is also the reason why you can't simply move your Qt installation to a different location (installed via online installer).

Argh! Why it's inside the binary!?
And why exporting the LD_LIBRARY_PATH it works? If inside the binary it should not matter...

Mark81

@JonB said in Qt libraries not found is the executable is copied from another machine:

@Mark81
One quick thought to eliminate: is untitled2 being run from the same directory as when you run untitled1? In particular, you aren't sitting in /home/user/Qt/6.4.0/gcc_64/lib when you run untitled2 but not for untitled1, are you?

No matter if the binaries is in the same or different folder. The behavior is the same.
Furthermore, no one is inside the lib dir.

JonB

@jsulm said in Qt libraries not found is the executable is copied from another machine:

This is done during build/linking, the binary contains this information.

I'm sorry, I know @Mark81 has marked this up, but I simply cannot believe this is the case?!

[EDIT See later on for ensuing discussion, you were absolutely correct!!]

I elect to install my Qt into /path1 under my development Ubuntu. I build an application.
I send you the application.
But you have elected to install your Qt into /path2.
According to you, the binary states to find Qt libraries in /path1, and therefore won't work.

This "cannot" be correct, surely? This would mean every product/executable would rely on Qt being installed into the same system path, and I cannot believe that is really the case...??

UPDATE
I am now reading up about chrpath command (or patchelf). Maybe you are right! :) Is this "RPATH" what it is all about, and that is indeed stamped into the executable at link time? I didn't know about that!

RPATH is a mechanism to include a list of directories in a binary where required shared libraries may be available. These locations are considered by the dynamic loader ( ld*. so ) to locate the libraries that are required by a particular binary.

Hmmmm! Wazzis then? It didn't exist last millenium under UNIX System V.0 ;-)

OK. My Ubuntu 20.04 has just the system-supplied Qt installed, never anything else. For any Qt executable I have built gives:

chrpath -l executable
executable: no rpath or runpath tag found.

so at least by default it does not seem to be stamped into my executables...?? Could you tell me under what circumstances my Qt executables should have this, assuming this is what you are speaking of for a "hard-coded path to libraries" stamped into them? Checked and my linker line does not pass any -rpath argument.

@Mark81
One thing to eliminate: does either machine have an LD_LIBRARY_PATH environment variable set??

Next: Install chrpath. I had to sudo apt install chrpath. Run chrpath -l executable on each of your two executables. Does this show yours do have an RPATH?

Mark81

@JonB said in Qt libraries not found is the executable is copied from another machine:

@Mark81
One thing to eliminate: does either machine have an LD_LIBRARY_PATH environment variable set??

No one has this env var set, until I export it in order to run the binaries built with the other machine.

I never heard about chrpath. Actually this can explain the behavior because if I run it against Untitled1 it returns the RUNPATH of Machine 1 and for Untitled 2 for Machine 2 (they just differ by the user name).

So what are the conclusions?

is it possible to change this RUNPATH after the compilation?
I have to export the LD_LIBRARY_PATH for any installation?
I have to provide the libraries in the same folder for any application? It will end up with a lot of duplicated file, though...

JonB

@Mark81 said in Qt libraries not found is the executable is copied from another machine:

I never heard about chrpath. Actually this can explain the behavior because if I run it against Untitled1 it returns the RUNPATH of Machine 1 and for Untitled 2 for Machine 2 (they just differ by the user name).

Ah ha! At least this explains why they don't run on each others' machines, right? And is as disastrous as I thought ;-)

First things first. Remember I said my executables do not have any RPATH in them. I would guess this is because I accepted the Qt supplied with Ubuntu, which puts it in a system area not a user area, and that must be searched by default, so does not need one? Whereas you installed into a non-default area, so that got passed via -rpath when you link on each machine. And then does not work on the other.

You have choices:

Use the default installation area on both when linking and when running. No RPATH.
Install your Qt in some directory which is not user-specific and which will be used on both machines, so RPATH is correct for both.
Tell user they will need to set LD_LIBRARY_PATH for where their Qt is installed if different from yours.
Supply/distribute all the Qt .so files with your executable. This is actually what is normally done when you release/distribute a Qt program, e.g. using linuxdeployqt. This is the tool to ensure the right stuff is packaged in the right place. It puts them relative to (I think in the same directory as) your executable. And that is automatically searched when running an executable. All this is also the case under Windows. And is what you are supposed to do.

jsulm

@Mark81 If you want to make sure your app runs on every machine then deploy it properly, so that the app package also contains all needed libraries. See https://doc.qt.io/qt-6/linux-deployment.html

@JonB Well, libraries installed in system folders like /usr/lib are always searched by the loader. If you link an application against libraries which are in some folders unknown to the system then you can't expect that the loader on another system knows where to look for the libs. That's why we often see users report issues due to the fact that wrong Qt libs are loaded.

JonB

@jsulm said in Qt libraries not found is the executable is copied from another machine:

the loader on another system knows where to look for the libs

I knew about LD_LIBRARY_PATH, but until this discussion not about RPATH. Now I do, thank you! They must have snuck this in sometime between UNIX System V.0 and now, while I wasn't looking ;-)

jsulm

@JonB said in Qt libraries not found is the executable is copied from another machine:

while I wasn't looking ;-)

How dare they :-)

JKSH

@JonB said in Qt libraries not found is the executable is copied from another machine:

I knew about LD_LIBRARY_PATH, but until this discussion not about RPATH. Now I do, thank you! They must have snuck this in sometime between UNIX System V.0 and now, while I wasn't looking ;-)

Wait till you hear about RUNPATH ;-)

https://www.qt.io/blog/2011/10/28/rpath-and-runpath

JonB

@JKSH Thanks, have read that, even more complicated!