Deployment problems on Mac OSX

floatingWoods

Hello,

First I need to mention that I am new to Mac and that many of my problems might seem "easy" to most people here... great!

Well, I have a console application ("myConsole") that dynamically loads a library ("libmyLibrary.dylib"). Both the console and the library are Qt code and Qt compiled (gcc). They work like a charm when launched from the QtCreator (i.e. I copy "myConsole" into the release directory of "myLibrary" project, then compile and run "myLibrary". When asked what application should be launched to test the library I indicate "myConsole").

So I want my application to be bundled. Those are the steps I did:

In my console application, in the *.pro file I mention: "CONFIG += app_bundle" (was "CONFIG -=app_bundle" before)
In the main routine of my console application, I load my library with:

@
QLibrary* lib=new QLibrary("libmyLibrary")
if (lib->load())
{
printf("Load successful for 'libmyLibrary'\n")
myFunction1=(ptrMyFunction1)(lib->resolve("myFunction1"));
...
}
else
printf("Failed loading 'libmyLibrary'\n");
@

I build my console application in release mode
In the directory where my console application was built, I type "macdeployqt myConsole.app". That takes a while. But it doesn't output any error or confirmation message
When I inspect the package content, I have 2 (or 3) additional directories directly under "Contents": "Frameworks" and "Plugins"
In "Frameworks" I have several directories and sub-directories (e.g. "QtCore.framework/Versions/4/QtCore" (why is that not a dylib file??)
I copy my library file ("libmyLibrary.dylib") into the bundle, inside the "MacOS" directory so that the console application can find it.
From the terminal, I type "open myConsole.app". Nothing happens. No error, no other information. No additional terminal opens.
From the terminal, if I type "open myConsole.app/Contents/MacOS/myConsole" then it opens another terminal window saying that the library ("libmyLibrary") failed loading. That message originates from the console application, where I printf an error message if lib->load() failed.

What am I doing wrong? I am at a loss here, I tried so many different combinations, but wasn't successful. Any help would be greatly appreciated.

Thanks

goetz

A couple of questions here. Let's start with the easy ones:

QtCore.framework is a so called "framework bundle":http://en.wikipedia.org/wiki/Application_bundle. The fact that it is actually a library comes from the .framework extension. It's necessary that the actual lib is the same name without any extension. Take it as granted :-)

Application bundles on the mac never print something to the terminal console. You can see the output that would normally go to stdout/stderr with the Console.app (usually in /Applications/Utilities/Console.app).

To load a library using QLibrary, Qt uses a library path. If the directory where you have put your libmyLibrary.dylib isn't in that path, Qt cannot find it. You can add that path using "QCoreApplication::addLibraryPath() ":/doc/qt-4.8/qcoreapplication.html#addLibraryPath or you can use an absolute path. In both cases you can use "QCoreApplication::applicationDirPath() ":/doc/qt-4.8/qcoreapplication.html#applicationDirPath at runtime, it gives you the path containing your app. You can then go backwards with "/../" to reach your library.

floatingWoods

Thank you Volker!

I did following:

I printed out the application path with

@
printf(QCoreApplication::applicationDirPath().toLocal8Bit().data());
@

The result is: /Users/me/Documents/MyProject/myConsoleRelease/myConsole.app/Contents/MacOS

I copied into above's directory the libmylibrary.dylib file, so that my console application can find that library.
In the console application, when trying to load "libmylibrary.dylib", it fails with following error message: "Cannot load library libmylibrary". Following code is used to load it:

@
QLibrary* lib=new QLibrary("libmylibrary");
lib->load();
@

If I understood correctly, Qt can find the library, but not successfully load it. Am I right? unfortunately the "lib->errorString()" function isn't very explicit.

So the only explanation I could find why "myLibrary" doesn't get loaded, is that "myLibrary" cannot dynamically link to Qt (both of "myConsole" and "myLibrary" rely on Qt)

My bundle dtructure looks like following:

@
Contents
Frameworks
QtCore.framework (with sub-directories)
QtDeclarative.framework (with sub-directories)
QtGui.framework (with sub-directories)
...
Info.plist
MacOS
myConsole
libmylibrary.dylib
PkgInfo
Plugins
accessible (with sub-directories)
bearer (with sub-directories)
codecs (with sub-directories)
...
Resources
empty.lproj
qt.conf
@

Looking at above's bundle, when I launch the bundle, "myConsole" gets started (and obviously can find the Qt libraries, since I can use Qt functions), but can't load "libmylibrary.dylib". I really suspect that "libmylibrary.dylib" cannot find the Qt framework to link against...

Sorry, I still need a little bit of help here. Thanks a lot

goetz

The linkage error for the libmylibrary.dylib could be a case.

You should check the output of

@
otool -L myConsole.app/Contents/MacOS/libmylibrary.dylib
@

it should look similar to that of

@
otool -L myConsole.app/Contents/MacOS/myConsole
@

and it should not contain any links to the standard Qt paths (/usr/local/... or /Developer/... for example).

floatingWoods

Thanks again Volker.
I ran the otool command and this is the result for myConsole and libmylibrary:

otool -L myConsole.app/Contents/MacOS/myConsole:

@
myConsole.app/Contents/MacOS/myConsole:
@executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.7.0, current version 4.7.4)
@executable_path/../Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.7.0, current version 4.7.4)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1094.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
@

otool -L myConsole.app/Contents/MacOS/libmylib.1.0.0.dylib:

@
myConsole.app/Contents/MacOS/libmylib.1.0.0.dylib:
libmylib.1.dylib (compatibility version 1.0.0, current version 1.0.0)
@executable_path/../Frameworks/QtOpenGL.framework/Versions/4/QtOpenGL (compatibility version 4.7.0, current version 4.7.4)
@executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.7.0, current version 4.7.4)
@executable_path/../Frameworks/QtNetwork.framework/Versions/4/QtNetwork (compatibility version 4.7.0, current version 4.7.4)
@executable_path/../Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.7.0, current version 4.7.4)
/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1094.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
@

I am not sure if the "/usr/lib" paths or the "/System/Library" paths might be problematic?

I actually ran "macdeployqt myConsole.app" after I copied the libmylibrary.dylib into the MacOS folder.

miroslav

macdeployqt is your friend. The second tool listing looks good. Is the app still not starting?

Is your mylibrary lib a plugin? If not, there is no need to load it with QLibrary, just link it.
If you use open, the command line output is swallowed. Just run the executable instead, without open: ./myConsole.app/Contents/MacOS/myConsole It should then also print errors in case the dynamic linker misses something.

[EDIT: fixed list formating (* instead of -), Volker]

floatingWoods

Thank you Miroslav,

I am using macdeployqt. The "libmylibrary.dylib" is not a Qt plugin. It is just a shared library that also uses Qt. Actually the libmylibrary will build the QApplication. It runs fine from QtCreator.

There are 2 things I am wondering about:

when I load the "libmylibrary.dylib", I specify

@
QLibrary* lib=new QLibrary("libmylibrary");
@

and not

@
QLibrary* lib=new QLibrary("libmylibrary.1.0.0.dylib");
@

The other point I was wondering about: when I copy the "libmylibrary.1.0.0.dylib" into my bundle, I do a simple copy and paste. Should I use the "cp -R" command instead?

Edit: Actually, compiling "myLibrary" produces following 4 files:

@
libmylibrary.1.0.0.dylib
libmylibrary.1.0.dylib
libmylibrary.1.dylib
libmylibrary.dylib
@

I always copied and pasted all 4 files into the bundle's MacOS directory before running macdeployqt

miroslav

Just answering from the top of my head:

If mylibrary is just a plain library, you need to link it at link time, not load it at runtime. Check out the -L <path> and -l <libname> statements in qmake.
Once the library is properly linked, macdeployqt should take care of copying it over automatically. No manual copying should be required.
The use of QLibrary should be unnecessary. You are trying to do the linker's job.

floatingWoods

Thanks again for your help Miroslav.

myLibrary is a shared library. It is loaded by "myConsole", but also by non-qt plugins. Each of thoses plugin will be able to call myLibrary's API functions. Following are the 3 types of elements that I have:

@
*myConsole (client application)
- Is in charge of dynamically loading and binding "myLibrary"
- Is in charge of dynamically loading and binging various plugins (non-qt plugins)
- Except for the QLibrary function to load and bind "myLibrary", this application doesn't directly require Qt

*myLibrary (collection of API functions)
- Contains a collection of API functions. Those functions need to be accessed by "myConsole" and the plugins (see below)
- Heavily relies on Qt

*various plugins
- They do not rely on Qt
- They need to access the API functions of "myLibrary"
@

For the tests however, I do not load any of above mentionned plugins.

what makes me crazy is that everything works like a charm when launched from QtCreator. I only have trouble with the bundling.

I could actually also write the "myConsole" without Qt, by using XCode. Would that simplify my problem? I used Qt for "myConsole" simply because I don't yet have any experience with XCode, but already know Qt from other platforms

miroslav

Ok, understood. The QLibrary use is needed in that case (sorry if I led you in the wrong direction).

What could happen is that the paths deviate from what QLibrary expects. You could assemble an absolute path using QCoreApplication::applicationFilePath to see if that is the problem (try QCoreApplication::applicationFilePath + ../MacOs/libmylibrary.1.0.0.dylib, check that it is really there). From the docs, it seems like QLibrary will first try using the passed in name as an absolute path.

miroslav

What you could also try is adding the frameworks directory of your bundle to the library path of QCoreApplication (QCoreApplication::addLibraryPath). I think the bundle frameworks directory is not added automatically.

The docs are a bit vague about the search order of QLibrary, though.

floatingWoods

Thanks again Miroslav!

I got it to work, but there is something strange going on:

Following does not work:

@
QCoreApplication::addLibraryPath("/Users/me/Documents/myConsole/myConsoleRelease/myConsole.app/Contents/MacOS");
QLibrary* lib=new QLibrary("libmylibrary");
@

But following works:

@
QLibrary* lib=new QLibrary("/Users/me/Documents/myConsole/myConsoleRelease/myConsole.app/Contents/MacOS/libmylibrary");
@

Might this be a bug in the addLibraryPath function?
Does anyone know from the top of his head a non-Qt command to get the path location of the current application? (including the inside folders of the bundle, so that I can build the absolute path of my library without having to build a QApplication object in order to use "applicationDirPath")

miroslav

Regarding the first question, the docs do not specify if QLibrary looks into the app library path.

Qt's applicationDirPath() is the most portable way to get to the executable path as far as I know.

floatingWoods

Thanks Miroslav,

I don't wanna use applicationDirPath() because this requires me to build a QApplication object (and this is done in the "MyLibrary"). Finally, I solved my problems with following code:

@
std::string libLocation(argv[0]);
libLocation.erase(libLocation.end()-9,libLocation.end()); // not elegant, but I just remove the "myConsole" string
chdir(libLocation.c_str());
QLibrary lib=new QLibrary("libmylibrary");
@

Thanks to both of you for your help... as it seems now it was a small thing, but it took a lot of effort for me and I wouldn't have found it without your help!!

Cheers

goetz

the library path of QCoreApplication is not used. The docs state:

bq. When loading the library, QLibrary searches in all system-specific library locations (e.g. LD_LIBRARY_PATH on Unix), unless the file name has an absolute path.

QCoreApplication::libraryPath() is not mentioned here. You might want to try to set the DYLD_LIBRARY_PATH environment variable before you try to load the library. You can try the following snippet, I did not test it, though.

@
QByteArray dyldPath = qgetenv("DYLD_LIBRARY_PATH");
if(!dyldPath.isEmpty())
dyldPath.append(":");
dyldPath.append(QCoreApplication::applicationDirPath());
qputenv("DYLD_LIBRARY_PATH", dyldPath);

// call QLibrary load after the stuff above
@

Please be aware that on Linux the environment variable is LD_LIBRARY_PATH and on Windows you will have to use PATH (although by default Windows looks into the directory that contains the .exe).

floatingWoods

Thank you very much Volker!