Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

(Windows) Minimum files in deployment that cannot be moved



  • I am working on a relatively thin Qt/QML 5.12 application. It allows the user to configure, update, and load a non-Qt video game.

    Because the users want to do things like install mods to the video game, I'd want to bury as much Qt files as possible behind a qt sub-directory. A handful of files (such as Qt5Core.dll) sitting next to the executable would be okay, but I'm curious what's the bare minimum required in a dynamically-linked configuration.

    I've tried several things that were suggested over the last ten-or-so years of forum and StackOverflow posts that I found on Google. The one that looked the most promising was putting a qt.conf file next to MyApplication.exe and adding the following lines to it:

    [Paths]
    Prefix=qt
    

    I've also tried things like ./qt, the full path to my application, and so forth. I've also tried other keys, like Plugins, Platforms, and so forth. Most of these tests were with the .dlls in the main directory left alone... only moving the other folders to a qt sub-folder.

    As far as I can tell -- nothing I do to qt.conf makes a difference. I keep getting "This application failed to start because no Qt platform plugin could be initialized."

    One example of an application that uses Qt/QML and doesn't have anything in the .exe directory is Blizzard's Battle.Net launcher. It's possible that they're doing something ugly like having a non-Qt exe in the main folder and building the Qt app as a DLL instead of an EXE... but I doubt it.

    So, in summary. What is the minimum number of files that cannot be moved to a qt sub-directory, and what would I need to do to get my Qt application to see them (in a dynamically linked configuration)?


  • Lifetime Qt Champion

    Don't copy dlls by yourself - take a look at https://doc.qt.io/qt-5/windows-deployment.html



  • I already use windeployqt --qmldir (my project folder) (my release build exe location) to build my application.

    One of the various things I've tried is a handful of command-line arguments to make it copy libs to a sub-directory. I've been unsuccessful.

    What arguments would be required to have windeployqt accomplish my goal, and what files is it unable to relocate to a qt subdirectory?



  • You may use a packer or something? That would increase the size of your .exe file,
    but make your environment look more clear.
    Check Enigma Virtual Box for example. It is a tool that packs .exe files with every kind of components (including multiple exe's, jpg, dll, ini or any kind of files).
    Or check how to create a symbolic link on Windows.
    Wish you luck with your project!


  • Lifetime Qt Champion

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    to make it copy libs to a sub-directory.

    Which libs? The Qt dlls must be in the same folder as the executable - otherwise Windows won't find them.



  • @closx Thanks for the advice, but I am looking for dynamic linking solutions.

    @Christian-Ehrlicher That's the question that I am asking. What is the minimum set of files and folders that cannot be moved, and how do we move the rest? For instance, can we get away with just Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll in the same folder as the .exe, and move every other file and folder into a child directory (which I'll call "qt" for the sake of discussion)? If so, how do we get the application to look in that sub-directory? If we need more than Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll, then what else do we need, and how do we move the rest?


  • Lifetime Qt Champion

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    away with just Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll in the same folder as the .exe, and move

    -->

    The Qt dlls must be in the same folder as the executable

    This is a windows limitation, not a Qt one.



  • @Christian-Ehrlicher You're misreading the quoted text. I have not been asking what you think I have been asking.

    I am saying that I am aware that Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll must be in the same folder as the .exe. I am aware that those cannot move without doing some very hacky things. I would be okay with leaving a handful of files where it's easy for Windows to grab them.

    My question is about the other 12 files and 15 folders. How many of them can be moved into a sub-folder, and how?

    Could it be done with some combination of windeployqt command-line options and qt.conf entries?


  • Lifetime Qt Champion

    @ScottMichaud All DLLs (not only Qt DLLs) must be in same directory as your executable, this is not a Qt thing.



  • @jsulm No. No, that is not true. Here is the Windows API that can load DLLs from arbitrary locations.

    The only DLLs that must be in the same directory as my executable (or somewhere else that Windows looks, like on the path environment variable) are the ones that are attached by the linker.

    In fact, if what you were saying was true, then the DLLs in the "platforms" and the "mediaservice" (etc.) sub-directories would never be loaded (except by a sub-process in those directories, which does not exist).

    It looks like Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll are attached by the linker, so those must be in the same directory (unless I build my app as a DLL, toss it in the sub-folder, and load it from a thin EXE, which I don't really want to support). How many of the other 12 files are not, and how do I relocate the ones that aren't (and the other 15 folders)? Does Qt provide a mechanism to relocate them? It really looks like that's what qt.conf is supposed to do, but, when I try creating one and modifying it, it doesn't seem to change anything.



  • @ScottMichaud Just asking because I wonder, plz do not offend or anything but, why do you want to move your dll files to another location so much?



  • @closx As mentioned in the first post, the project configures, updates, and loads a non-Qt video game. Many of our users install mods and otherwise customize their experience. Dumping 15 folders and 15 files around the folder that they're looking for will intimidate and confuse them, especially if they don't have much experience with modding games. (ex: They're following a friend's advice.)

    In fact, excessive files and folders was the very first complaint that I received in private testing. It was a very important issue to them.



  • @ScottMichaud
    One thing I don't get: you are developing a Qt app to manage some other third-party app. It's to the other app folder that users go to add their mods, right? Why is your app installed into the same folder as the other app? If it were somewhere else of its own, as I'd expect, what is in your app's folder wouldn't matter?



  • @JonB Both the manager (Qt) and the game are portable applications. I download and unpack the game into my manager's main folder to simplify the mod installation process (and allow them to leave it in My Documents, a secondary or external hard drive, etc. without triggering admin privileges).

    I'm also leaving the door open for things like controller remapping, adjusting game settings, etc.



  • @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    Both the manager (Qt) and the game are portable applications.

    means nothing to me, if there is a significance to "portable" I'm afraid I won't know about that.

    I download and unpack the game into my manager's main folder to simplify the mod installation process

    And thereby plonk the game into your Qt executable/DLL folder which is causing you the headache!

    If the game at least went one level down into a subdirectory of the manager again it would be neater.

    It's not what you want to hear, but it's the root cause of your problem :)

    That said: you want to separate "the other 12 files and 15 folders" off from " Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll ". Can you give me some examples of what these other files/folders/DLLs are (especially DLLs), as I don't know?



  • @JonB They are:

    audio/
    bearer/
    iconengines/
    imageformats/
    mediaservice/
    platforms/
    playlistformats/
    qmltooling/
    QtGraphicsEffects/
    QtMultimedia/
    QtQuick/
    QtQuick.2/
    scenegraph/
    styles/
    transitions/
    D3Dcompiler_47.dll
    libEGL.dll
    libGLESV2.dll
    opengl32sw.dll
    Qt5Multimedia.dll
    Qt5MultimediaQuick.dll
    Qt5Network.dll
    Qt5Quick.dll
    Qt5QuickControls2.dll
    Qt5QuickTemplates2.dll
    Qt5Svg.dll
    Qt5Widgets.dll

    ... as well as the aforementioned Qt5Core.dll, Qt5Qml.dll, and Qt5Gui.dll.

    Also, no, I'm not going to have the manager blindly modify folders in the parent of its working directory. I'd rather build the manager as a DLL, toss it in a sub-directory, and invoke it from a thin EXE in the main directory. That seems to be what Blizzard does with the Qt/QML BattleNet Launcher.


  • Moderators

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    What is the minimum number of files that cannot be moved to a qt sub-directory, and what would I need to do to get my Qt application to see them (in a dynamically linked configuration)?

    The "minimum number" depends on:

    • Which compiler did you use to build your project?
    • Which Qt modules do your project link to?
    • Are you willing to modify your system's PATH?

    Broadly, there are 2 groups of DLLS:

    1. Load-time DLLs need to be found by Windows before it launches your app. Examples:
      • libstdc++-6.dll (if you used the MinGW compiler)
      • Qt5Core.dll
    2. Run-time DLLs need to be found by your app before it tries to initialize specific features. Examples:
      • platforms/qwindows.dll
      • QtQuick.2/qtquick2plugin.dll

    Most of the time, load-time DLLs as placed in the same folder as your executable, because Windows will always search that folder. Technically, load-time DLLs can be placed in a subfolder if that folder is in your PATH. You could, for example, launch a Batch script instead of launching your executable directly. The Batch script could temporarily add the load-time DLLs' folder to the PATH before it launches your executable.

    WARNING: Do not permanently add Qt DLLs to your user's system PATH, or you could cause DLL Hell.

    Run-time DLLs are the ones that you can configure using qt.conf or by calling a function. If you call QCoreApplication::addLibraryPath("C:/MyPlugins/"); before you construct your QApplication, then you can have your run-time DLLs in, for example, C:\MyPlugins\platforms\qwindows.dll


  • Lifetime Qt Champion

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    Here is the Windows API that can load DLLs from arbitrary locations.

    This link describes how to load libraries at runtime WITHOUT linking against them at linking time - this is not the same...



  • @JKSH said in (Windows) Minimum files in deployment that cannot be moved:

    Broadly, there are 2 groups of DLLS:

    1. Load-time DLLs need to be found by Windows before it launches your app. Examples:
      • libstdc++-6.dll (if you used the MinGW compiler)
      • Qt5Core.dll
    2. Run-time DLLs need to be found by your app before it tries to initialize specific features. Examples:
      • platforms/qwindows.dll
      • QtQuick.2/qtquick2plugin.dll

    Most of the time, load-time DLLs as placed in the same folder as your executable, because Windows will always search that folder. Technically, load-time DLLs can be placed in a subfolder if that folder is in your PATH. You could, for example, launch a Batch script instead of launching your executable directly. The Batch script could temporarily add the load-time DLLs' folder to the PATH before it launches your executable.

    WARNING: Do not permanently add Qt DLLs to your user's system PATH, or you could cause DLL Hell.

    Run-time DLLs are the ones that you can configure using qt.conf or by calling a function. If you call QCoreApplication::addLibraryPath("C:/MyPlugins/"); before you construct your QApplication, then you can have your run-time DLLs in, for example, C:\MyPlugins\platforms\qwindows.dll

    Thanks! It looks like addLibraryPath() is exactly what I need, but I'll circle back and try qt.conf again in case I was doing something stupid.

    @jsulm said in (Windows) Minimum files in deployment that cannot be moved:

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    Here is the Windows API that can load DLLs from arbitrary locations.

    This link describes how to load libraries at runtime WITHOUT linking against them at linking time - this is not the same...

    Correct, as I said in the very next sentence of the post you quoted:

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    @jsulm No. No, that is not true. Here is the Windows API that can load DLLs from arbitrary locations.

    The only DLLs that must be in the same directory as my executable (or somewhere else that Windows looks, like on the path environment variable) are the ones that are attached by the linker.

    This entire thread was asking which DLLs are runtime-linked, and done so in a configurable way, and what mechanisms exist to move them. I am aware that some of these files could not move. I was asking which ones could (and how).



  • Okay so it looks like in my project, I'm able to move every folder that doesn't start with Qt and every dll that doesn't start with Qt5 (unless something subtly broke somewhere that I'm not seeing) by adding...

    QCoreApplication::addLibraryPath("./QtMyFolder/");
    

    ... to the top of my main function and tossing the rest into QtMyFolder. (I don't actually call it QtMyFolder of course.)

    That brings me down to five folders (six if you count the subfolder that I created to hold the others) and eleven files... and they also visually "go together".

    Hopefully this will be good enough. I don't really want to do one of the .bat-loads-exe or .exe-loads-dll hacks. :\ They are options, though.



  • @ScottMichaud

    Also, no, I'm not going to have the manager blindly modify folders in the parent of its working directory. I'd rather build the manager as a DLL, toss it in a sub-directory, and invoke it from a thin EXE in the main directory.

    I do respect that you seem to know what you're doing, but this is a very strange comment to make.

    You have a folder, third-party-game, containing the exe for the game, its DLLs, and any sub-folders it might have for its stuff. A bunch of files & folders, which as I understand it you want to be able to modify from your manager app.

    You are planning to put your manager's .exe, plus its DLLs/sub-folders, into that same folder. Which leads to at least some of its files lying around next to the app you plan to manage, which is what you object to.

    There is absolutely no reason why you shouldn't root your exe, plus its DLLs etc., in sub-folder third-party-game\manager, rather than in third-party-folder. In no way does that interfere with its ability to mange or access files in third-party-folder, the parent directory of manager. It is no easier to access/manage files in the same directory as an exe lives than it is if they are in the parent folder (btw, not necessarily its "working" directory as you wrote). There is nothing "blind" about this, nor is there any need to create a wrapping exe/bat etc. Indeed, it is easier to manage for both you & your users if all your app's stuff is it in its own folder downward, precisely so that the game & your stuff do not share a folder where their respective files are mixed together, e.g. when upgrading either app.

    It doesn't stop you using addLibraryPath() either. Just that QtMyFolder will be a sub-directory of third-party-game\manager. Whatever files/folders you say you still have not managed to moved out of your exe folder will no longer sit next to the third-party's exe.

    If you're happy with what you have now obviously that's fine, but I still don't get why you wouldn't prefer to store your stuff in third-party-game\manager even from where you have got to now, especially given your objective....



  • @JonB The manager is not a child of the game folder. The manager is the parent. The game is the child.

    The idea is that the user downloads the manager and extracts it to a folder. We'll call that /Documents/GameManager/ for the sake of argument.

    They would then run /Document/GameManager/GameManager.exe <-- Qt application.

    In GameManager.exe, there's a button to download and install the game (or update it if there's a new version). GameManager.exe puts it in its folder.

    /Documents/GameManager/TheGame/(all of the games files)

    The problem is that /Documents/GameManager/TheGame/ was one of 16 folders (technically 18 because I had two other folders of my own in there) so it was intimidating to some users. Ideally, I'd have the folder structure be:

    /Documents/GameManager/ManagerFiles/(all the dependencies that Qt and I need)
    /Documents/GameManager/TheGame/(all the game's files) <-- if the game is installed
    /Documents/GameManager/GameManager.exe

    ... but I figured that Qt had some hardcoded (because they're attached by the linker, or because Qt didn't provide a config mechanism, or because a third-party like ANGLE didn't provide a config mechanism) values so I was willing to compromise from that ideal model.


  • Moderators

    @ScottMichaud said in (Windows) Minimum files in deployment that cannot be moved:

    /Documents/GameManager/ManagerFiles/(all the dependencies that Qt and I need)
    /Documents/GameManager/TheGame/(all the game's files) <-- if the game is installed
    /Documents/GameManager/GameManager.exe

    Another possible solution is

    • /Documents/GameManager/GameManager.exe
    • /Documents/GameManager/*.dll
    • /Documents/GameManagerGames/TheGame/

    EA Origin uses this structure.



  • @JKSH Yeah that's an option.

    Now that you mention it, I could also have the manager put the game in %AppData% and add a button in the manager to open the file explorer to the game folder for people who want to look into it. Honestly, that would be more professional than what I'm doing now.



  • @ScottMichaud
    Dear Scott,

    I've tried all along to convince that you that, whatever the layout of files/folders you want is/might be, just don't place your own exe plus its DLLs/sub-directories into the same directory as wherever the third-party exe/DLLs/mods are. Doesn't matter so much where either one is a sub-directory of the other, or they are siblings, or they are quite separate. For all sorts of reasons anyway, but especially since you are bothered by their files sitting in the same place and confusing your users. It seems my efforts have been in vain, but @JKSH has somehow hit the sweet spot! It's the right move, good luck! :)



  • @JonB said in (Windows) Minimum files in deployment that cannot be moved:

    @ScottMichaud
    Dear Scott,

    I've tried all along to convince that you that, whatever the layout of files/folders you want is/might be, just don't place your own exe plus its DLLs/sub-directories into the same directory as wherever the third-party exe/DLLs/mods are. Doesn't matter so much where either one is a sub-directory of the other, or they are siblings, or they are quite separate. For all sorts of reasons anyway, but especially since you are bothered by their files sitting in the same place and confusing your users. It seems my efforts have been in vain, but @JKSH has somehow hit the sweet spot! It's the right move, good luck! :)

    I was never going to put the game's exe and dependencies and the manager's exe in the exact same folder. That would be awful. When I said "unpack the game into my manager's main folder" I always meant extracting and placing the game's folder in my manager's main folder. Portable apps are a self-contained folder.

    Also, I thought a little bit more and %AppData% won't work because I'd need to support external hard drives and USB keys. I really don't like accessing (especially deleting) folders that are not direct descendants of mine, although it's kind-of the best option short of a bootstrapping (bat-load-Qtexe or exe-load-Qtdll).

    (If you've found this thread in the future: loading the same exe that used addLibraryPath() a couple days ago started complaining about platforms/ after turning my computer on the next day, and OpenGL was complaining on a clean Windows Server 2008 R2 VM. I might still have a problem.)


Log in to reply