How to compile my application so that it searches for all Qt libraries and dependences in the application folder

floatingWoods

Hello,

I compile my Qt application via Qt creator on Linux (Ubuntu) and never really worried about making sure Qt will search the appropriate locations to load the additional Qt libraries and dependencies. I simply copy all required Qt libraries into the apllication folder, so that I will also be able to run several of my application versions, that rely on various Qt versions.

Until now, it always worked fine, but I didn't explicitely do anything for that. Do I need to worry about something?
In some configurations, it seems that some Qt items can't be found (e.g. 'This application failed to start because it could not find or load the Qt platform plugin "xcb"') even if the item is located in the "platform" folder, itself located in my application's folder. How come?

Do I need to build my Qt application with specific flags? If yes, what flags?

Thanks for any help

SGaist

Hi,

This thread of the interest mailing list should help you get going

floatingWoods

Thank you again for your help SGaist!

I feel this is already quite complex for me.. ;)

So, if I understood the thread you mentioned correctly, I would need following line in my *.pro project file:

LIBS += -Wl,-rpath=$ORIGIN,--enable-new-dtags

Is that correct? Then I can normally build my application with Qt Creator as I did until now. That way Qt will search for needed Qt libraries in my executable's folder. As a precision: I am NOT compiling the Qt from source, simply copying the libraries that ship with Qt into my executable's folder.

Thanks

ChrisW67

There are three ways go about deploying on Linux
http://doc.qt.io/qt-5/linux-deployment.html#creating-the-application-package
The thread SGaist has pointed you at contains a couple of them.

Thiago's suggestion is to use the RPATH. The Rpath embeds a location to search in the executables themselves, and the magic value "$ORIGIN" says "in the same place as the executing binary". The other option should be "--disable-new-dtags" so that the older RPATH is used in preference to the newer RUNPATH. You may be able to set the path more easily using the QMAKE_RPATHDIR variable in your pro file.

Edit: I would expect the options you posted, in the form you posted them, in the QMAKE_CXXFLAGS not LIBS. You may have to take care that $ORIGIN survives passing through the shell without expansion.

hskoglund

Hi, about setting that RPATH in your .pro file, best is to use QMAKE_LFLAGS like this:

QMAKE_LFLAGS += -Wl,-rpath,"'\$$ORIGIN'"

Rondog

I ran into these issues recently myself. I noticed that if you change the RPATH you might introduce a new problem. You can try this by adding the suggested modifications to the .PRO file but I would suggest using something like 'PatchElf' to change and test (make a copy of the libraries in question and modify these) to see if this is the right answer. What I found is that by adding a relative path (i.e. $ORIGIN is the same as '.' meaning 'from here') the program would work only if I tried running it from a terminal window while in the same directory as the program. If, for example, I had the program under /usr/bin I could not run it from the $HOME folder because everything is searched relative to $HOME and not the application as intended.

One possible solution might be to do nothing. If you are packaging your application as an RPM or DEB I believe all dependencies are identified so when the application is installed, if something is missing that is required, it should be automatically installed. Most distributions will have Qt5 libraries and everything related. I haven't tried this but I am begining to think this is the best solution.

The plugins might be a problem. They are loaded by the Qt application and not dynamically linked. They might have dependencies that are not properly identified. You might need to manually include dependencies from necessary plugins (?).

My solution for this problem was to change the relative path (RPATH) of the libraries I wanted to distribute with the application (Qt5Lib*) and use a shell script to change to location to the directory of the application before launching it. I don't know if this is a good idea or not but it seems to work. I distribute some of the Qt libraries necessary for the application and let the installer add the rest.

SGaist

@Rondog the script is even the official solution explained in the documentation linked by @ChrisW67

Rondog

@SGaist I did look at the GNU/Linux deployment article a few times. I originally tried the version with the LD_LIBRARY_PATH script as described in the article. This worked for the application but not for the plugins. The plugins were found and loaded but they didn't respect the LD_LIBRARY_PATH variable so they couldn't find the needed Qt libraries that they relied on. The article seemed to drop off at this point.

Changing the RPATH of the deployed application and libraries worked but you need to have a script to 'cd' to the application directory first. Didn't need to set LD_LIBRARY_PATH when doing it this way.

I am still a bit conflicted on this and still looking for a better way. I am starting to lean on the idea of letting the OS install all dependencies and only distributing the application (if possible).

floatingWoods

Thank you all for the very detailed information. I ended up using the recommended script solution (where I modify the LD_LIBRARY_PATH variable before launching my application).

This works great. Then, since in some rare cases I had problems with the plugins (typically 'This application failed to start because it could not find or load the Qt platform plugin "xcb"'), I tried to add following line of code at the very beginning of my application:

QCoreApplication::addLibraryPath("./");

Printing out the library paths with:

QStringList paths(QCoreApplication::libraryPaths());
for (int i=0;i<paths.size();i++)
    printf("Library path %i: %s\n",i,paths[i].toStdString().c_str());

Gives me in first position the path of my application.
The application starts, but when I open a QFileDialog, it crashes with following message:

Cannot mix incompatible Qt library (version 0x50201) with this library (version 0x50400)
Aborted (core dumped)

My question here is: how come? This only happens on my development environment, not on a clean Linux install (without Qt installed). My application is packaged with all required Qt libraries and plugins in the application's folder. Also, on my development environment, I have never installed Qt version 5.2.1. Where does it go to look for that library, and why? And as far as I know QFileDialog is part of the QWidget library, that otherwise runs fine in my application.

Rondog

You must have more than one copy of the Qt libraries on your computer. Might be from some application written that used and installed Qt 5.2x. Use 'find' to locate all the libQt5*.so files so you know what you have.

QFileDialog is a widget but likely is closely connected to the platform plugin. The default is to use the native dialogs.

I had problems using the LD_LIBRARY_PATH method. On my test system I don't have anything that provided Qt5 library's and could easily 'hide' my copy of Qt so it could not be found. Sounds like the same problem except your program is finding the Qtlibs or plugins from somewhere else.

floatingWoods

Thanks for your reply Rondog

That's also what I thought, and it is probably so. But: why are all the techniques so unreliable in some situations? I always thought that Linux handled libraries better than Windows. But in Windows I simply put everything in my application folder, and it always works.
Mac also works flawlessly with "macdeployqt".

Rondog

Windows is easy to deploy. No argument there. I don't know if this makes it better though. I get the impression the GNU/Linux is better suited for distributing source code so 'binary' installers have a harder time (?).

Funny you mention 'macdeployqt'. I looked at this quite a bit. This solves some of the same problems you have on GNU/Linux. What this does in a nutshell is put everything required into the .app folder and change the hard coded paths to relative versions. I did end up modifying the version of 'macdeployqt' I use; it doesn't always work 'flawlessly'.

I am not sure if this is the best solution but this is what I came up with so far. I did borrow a few ideas from the 'macdeployqt' utility.

The generic application tree looks like this:

    usr
    ├── bin
    │   └── [appname]
    ├── lib64
    │   └── [appname]
    │       ├── imageformats
    │       │   ├── libqdds.so
    │       │   ├── libqgif.so
    │       │   ├── libqicns.so
    │       │   ├── libqico.so
    │       │   ├── libqjp2.so
    │       │   ├── libqjpeg.so
    │       │   ├── libqmng.so
    │       │   ├── libqtga.so
    │       │   ├── libqtiff.so
    │       │   ├── libqwbmp.so
    │       ├── platforms
    │       │   └── libqxcb.so
    │       ├── qtlibs
    │       │   ├── libQt5Core.so.5.3.2
    │       │   ├── libQt5Gui.so.5.3.2
    │       │   ├── libQt5Widgets.so.5.3.2
    │       │   └── libQt5Xml.so.5.3.2
    │       ├── [appname]
    │       └── [appname].bin
    └── share
        ├── applications
        │   └── [appname].desktop
        └── pixmaps
            └── [appname].png

The installer is compiled into an RPM so some choices are automatic (/usr/lib64 for example on x86_64).
All of the distributed Qt5 libraries are put into the folder 'qtlibs' under the name of the application. All distributed plugins are put in the proper folder suitable for the plugin. On each of the files I run the 'strip' command to prepare them for distribution:

strip libQt5Core.so.5.3.2
strip libqxcb.so
...
[or this for several files]
find *.so.* -print0 | xargs -0 -I lib  strip lib

For the application and all plugin libraries I change the RPATH parameter. I am using 'patchelf' for this. Note: There is a subtle difference in the path that is critical. The application must be './qtlibs' where the plugins must be '../qtlibs' in this setup.

patchelf --set-rpath ../qtlibs libqdds.so
patchelf --set-rpath ../qtlibs libqxcb.so
...
[or this for several files]
find *.so -print0 |xargs -0 -I lib patchelf --set-rpath ../qtlibs lib

[and the application must be changed]
patchelf --set-rpath  ./qtlibs [appname].bin

The entry /usr/bin/[appname] is just a link to the shell script /usr/lib64/[appname]. The shell script figures out where it is, changes to that directory, then executes the program [appname].bin. The LD_LIBRARY_PATH could be set as well but I didn't find it necessary to do this.

#!/bin/sh
# Generic startup script
# Gets the name and path of the script file
# Executes the program <script_path>/<name>.bin

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do
  SCRIPT_PATH="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
  SOURCE="$(readlink "$SOURCE")"
  [[ $SOURCE != /* ]] && SOURCE="$SCRIPT_PATH/$SOURCE"
done
SCRIPT_PATH="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
NAME=`basename $0`
cd $SCRIPT_PATH
exec $SCRIPT_PATH/$NAME.bin "$@"

The '/usr/share/applications/[appname].desktop' and '/usr/share/pixmaps/[appname].png' are for the system menu. The [appname].desktop item has the exec entry set to '[appname]' without a path.

SGaist

Did you use a qt.conf file besides your application ?

floatingWoods

Hello,

No, I didn't. Maybe it is just me that I didn't understand something, but wouldn't this make everything much simpler? Or do I need to combine the various techniques?

SGaist

qt.conf is the recommended way for the plugins handling part