QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe

MarKS

I am having a weird error while loading plugins using QPluginloader. I built 4 plugins in total all specified with Q_PLUGIN_METADATA, Q_OBJECT and Q_INTERFACES. I can see all the plugins are built properly and there are no missing .dlls (checked with Dependency walker). All these plugins are placed in separate plugins folder.

While loading, 2 plugins can be loaded without any problems. The other 2 plugins throw the error as mentioned in the title. If i copy the plugin dll and all its dependencies in the same folder with .exe, QPluginloader loads the plugin fine.

What can be reasons for this weird behavior?

I am using Qt 5.7 with VS 2015 on Win 10 x64

MarKS

While reading through Dynamic-Link Library Search Order i found out it is possible to modify the search order using SetDllDirectoryA() as described here. It finally worked.

Although, this would still be half the answer as it will fail in Linux or Mac. I believe there should be a way to achieve this in Linux but right now i am happy with windows. If you guys know how to achieve this in Linux or Mac please feel free to add your answer here.

Would help others.

SGaist

Hi,

How are you loading these plugins ?
What dependencies do they have ?

MarKS

After much digging i found a way around it. But first the issues:

My plugins have dependencies on OpenCV and Boost libs
Dependencies are located in the same folder as the plugin lib

The structure looks like this:

.exe directory
   |--- plugins folder
      |----DeviceA folder
          |----deviceA_plugin.dll
          |----opencv_dependencies.dll
          |----boost dependencies

I use this to load the plugins:

QPluginloader* loader = new QPluginloader("/path/to/DeviceA/deviceA_plugin", this); and evidently it fails.

I found out that if i move all the dependencies(not the plugin itself) to the .exe directory, suddenly plugins can be loaded without any issues.

This defeats my purpose of using plugins. I do not want to all my plugins dependencies in the .exe folder as some of my plugins share same dlls and they create conflicts.

I am not sure if this is Qt way or Windows way of searching for dll dependencies in the runtime.

Is there a smarter way to address this issue keeping the my folder structure intact as above? As it keeps all my plugins and dependencies isolated from each other?

kshegunov

@MarKS said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

This defeats my purpose of using plugins. I do not want to all my plugins dependencies in the .exe folder as some of my plugins share same dlls and they create conflicts.

Well you should rather want them to share the same dlls, not the other way around, but anyway ...

I am not sure if this is Qt way or Windows way of searching for dll dependencies in the runtime.

That's how the windows loader does it by default, it uses the current working directory first to search for the libraries.

Is there a smarter way to address this issue keeping the my folder structure intact as above? As it keeps all my plugins and dependencies isolated from each other?

Two possible options:

Change the current directory before loading the plugin to the path where the plugin's located. I haven't tested this, but should work in principle.
Caveats:
- if the loader decides this module is already loaded (i.e. decides you're trying to load the same library) you may get nothing done, which may lead to strange results (a static initialization having an unexpected value for example)
- if you ever deploy this on linux, you may get symbols overwritten which can lead to strange segfaults or behaviors
- you probably should restore the working directory after the plugin loading (usually one expects it's the application's directory), you may want to use it later, and if you don't specify absolute paths you may get surprises.
~~Switch to manually loading the external libraries at runtime from your plugin. Usually this is a massive PITA, but with QLibrary and a ResolveAllSymbols hint it should more or less be okay-ish.~~
Okay, I forgot we are talking about classes ... the aforementioned isn't really an option. This may be alternative: https://docs.microsoft.com/en-us/cpp/build/reference/linker-support-for-delay-loaded-dlls?view=msvc-160

MarKS

@kshegunov

Thank you for this nice explanation.

Well you should rather want them to share the same dlls, not the other way around, but anyway ...

My bad. I meant that i have dlls with same names but belong to different SDK versions.

I will try changing the working directory and see if it actually works.

jsulm

@MarKS said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

i have dlls with same names but belong to different SDK versions

Why different SDK versions? This is asking for troubles (like mixing different Qt versions).

MarKS

@jsulm I know but can't help it. They are proprietary sensor SDKs. That is why i chose to go plugins way. But it seems like changing the working directory is not helping either.

I tried this:

QFileInfo pluginPathInfo(path);

qInfo() << "Current plugin dir:" << pluginPathInfo.dir().path();
bool status = QDir::setCurrent(pluginPathInfo.dir().path());

qInfo() << "Changed Current working dir:" << QDir::currentPath();

I can see this changes the current working dir to my plugin directory but still the plugin fails to load.

kshegunov

Okay, I guess I was wrong then. Anyway, could you dump the load-time dependencies from the plugin? On a related note, how do you link these libraries (what paths do you use) when building the actual plugin?

MarKS

@kshegunov I use CMake to configure my project. A sample plugin cmakelists.txt looks like this:

# library name
set(LIBRARY_NAME deviceA_plugin)

add_library(${LIBRARY_NAME} 
            SHARED 
            ${Headers}
            ${Sources}
           )
           
# Link to libs
target_link_libraries(${LIBRARY_NAME} 
                      PRIVATE
                      Boost::filesystem
                      Boost::date_time
                      Boost::regex
                      opencv_core
                      opencv_video
                      opencv_videoio
                      opencv_imgcodecs
                      opencv_imgproc
                      Qt5::Core                      
                      )

target_include_directories(${LIBRARY_NAME}  
                           PRIVATE 
                           ${CMAKE_CURRENT_SOURCE_DIR}
                           ${CMAKE_CURRENT_SOURCE_DIR}/..
                           ${CMAKE_CURRENT_SOURCE_DIR}/../..
                          )
                          
set(Used_OpenCV_Version "${OpenCV_VERSION_MAJOR}${OpenCV_VERSION_MINOR}${OpenCV_VERSION_PATCH}")

install(TARGETS ${LIBRARY_NAME}
        RUNTIME
        DESTINATION "plugins/DeviceA"
        COMPONENT ${LIBRARY_NAME} 
        )

install(FILES
        "${OpenCV_DIR}/bin/Release/opencv_videoio${Used_OpenCV_Version}.dll"
        "${OpenCV_DIR}/bin/Release/opencv_video${Used_OpenCV_Version}.dll"
        "${OpenCV_DIR}/bin/Release/opencv_imgcodecs${Used_OpenCV_Version}.dll"
        "${OpenCV_DIR}/bin/Release/opencv_imgproc${Used_OpenCV_Version}.dll"
        "${OpenCV_DIR}/bin/Release/opencv_core${Used_OpenCV_Version}.dll"
        DESTINATION "plugins/DeviceA"
        COMPONENT ${LIBRARY_NAME} 
        CONFIGURATIONS Release
       )

Unfortunately, i can't dump my dependencies. I have plugins which have dependencies to OpenCV, OpenCL, proprietary SDKs, boost and so on..

kshegunov

Right, well you could try this instead (not tested), if that doesn't work ... well I'm starting to doubt it's possible to do ...:

// Instead of QDir::setCurrent(pluginPathInfo.dir().path())
auto handle = AddDllDirectory((PCWSTR) str.utf16());
Q_ASSERT(handle);

// QPluginLoader ... load code

RemoveDllDirectory(handle);

I'm not sure on windows there's something equivalent to the RPATH, but you could try setting it in the cmake file and see if it works.

SGaist

Sadly it looks like Windows does not have that feature. At least based on the Microsoft documentation.

kshegunov

@SGaist said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

Sadly it looks like Windows does not have that feature.

Which feature do you mean?

MarKS

@kshegunov It didn't work. We are running out of options aren't we? I still believe there must be a way to deal with this. I don't think big applications put all their dependencies in one place (application directory) so that windows can find them at run-time.

I preferred to build plugins the Qt-way because it is a bit easier to integrate into my Qt application. Now, i realize even, if i use
boost.dll to load plugins i will again run into this issue of windows not able to find the dependencies since it is an OS issue now. Or am i wrong?

kshegunov

@MarKS said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

I preferred to build plugins the Qt-way because it is a bit easier to integrate into my Qt application. Now, i realize even, if i use
boost.dll to build plugins i will again run into this issue of windows not able to find the dependencies since it is an OS issue now. Or am i wrong?

Well I'm pretty much out of ideas. A few years back (or at least to my knowledge) MS introduced SxS which is/was supposed to combat this particular problem, however I personally have no experience in deploying with it.

Additional link: https://docs.microsoft.com/en-us/windows/win32/sbscs/isolated-applications-and-side-by-side-assemblies-portal

SGaist

@kshegunov said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

@SGaist said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

Sadly it looks like Windows does not have that feature.

Which feature do you mean?

RPATH equivalent

@MarKS one other possibility could be to use a script to start your application that would first set the PATH environment variable to include the folders of your plugins.

MarKS

this seems to be a promising idea. However, i would also need to unset the PATH variables when the application is terminated. I do not want to spam the users PATH variable if i keep on adding new plugins. I will try it out and will update here.

SGaist

The goal of the script is to change the PATH locally for the script and start your application as well. Just like it's done by Firefox on Linux to ensure the bundled libraries are found.

kshegunov

@SGaist said in QPluginloader error: "Cannot load library: The specified module could not be found" unless the plugin is in the same folder as in .exe:

The goal of the script is to change the PATH locally for the script and start your application as well. Just like it's done by Firefox on Linux to ensure the bundled libraries are found.

I don't believe that's going to work (properly), because to have this work the expected way the PATH should be modified between loading of the plugins, not at the time the executable is loaded ...

@MarKS, two more (rather more obvious suggestions):

Rename the dlls and libs each plugin uses, so the loader knows what to map where and link the plugin to each of the ones it needs. You can apply this with the usual deployment - put everything in the app dir. Granted it's not very clean, but it's (almost) guaranteed to work.

or

(And I hate myself for suggesting this, but we live in interesting times ...)
Link statically the libraries in the plugin. If you decide to go this way, do not forget to compile the libraries yourself and enable /GL on compilation and /LTCG at link time to have the MSVC do the LTO so you don't get overly fat binaries.

SGaist

For number 2: only the non Qt stuff. Otherwise you'll have interesting issues with the QObject based classes meta object having multiple copies.

MarKS

While reading through Dynamic-Link Library Search Order i found out it is possible to modify the search order using SetDllDirectoryA() as described here. It finally worked.

Although, this would still be half the answer as it will fail in Linux or Mac. I believe there should be a way to achieve this in Linux but right now i am happy with windows. If you guys know how to achieve this in Linux or Mac please feel free to add your answer here.

Would help others.