Call to QLibrary resolved address works in Debug build, but not Release build
On a 64-bit machine running Windows 7, a pointer to a function in a DLL is returned via QLibrary::resolve. A class member function is called via QtConcurrent::run, and inside that function it calls the function in the DLL. As long as I do a QtCreator Debug build, everything works perfectly fine. But when I do a Release build, after calling and getting a normal return from the DLL function when the calling function returns, instead of signalling finished() (which is what QtConcurrent::run normally does), it crashes. How can I even debug this, since it works just fine in the Debug build?
This is hard to say, unless we know more. How does your code look like? How does the dll look like? What does the method do?
Is the library you're loading available to your release build (e.g. is it in the same directory?).
Are you checking if a valid pointer was returned by "QLibrary::reslove(...)":http://qt-project.org/doc/qt-4.8/qlibrary.html#resolve? The returned pointer will be NULL if the symbol name couldn't be resolved - and if the library could not be loaded it will certainly return NULL and thus crash when you're not checking.
I have copied the DLL library to both the release and debug build areas and checked the QLibrary::resolve, which gives back a legitimate pointer in both cases (debug and release), and in each case the call to the DLL function runs successfully returning valid data. After the return from the DLL function I can continue to do anything I want to up to the point where I return from my function. It is as if the DLL function is corrupting the call stack, or corrupting some memory that Qt has reserved for it's own use. I have obtained the source code of the DLL functions. The function I am calling is very simple, but does write into it's global memory using an index that is not initialized in that function. It is initialized in the function:
BOOL WINAPI DllMain(HINSTANCE hInst, WORD wReason, LPVOID lpReserved), and is expected to be called with wReaon = DLL_PROCESS_ATTACH. This is supposed to happen automatically after a call to LoadLibrary.
Since the guy who wrote the DLL library is here, maybe we can rebuild the DLL library and debug it, to see what may be going on.
I was hoping someone knew of existing problems with using DLL's in debug vs release mode.
I assume this is no problem of QLibrary but of the DLL itself. As I'm not programming on Windows systems regularly I'm not sure about that, but I assume that DllMain may be not called. If the function you are using is relying on an index that is initialized in DllMain it may be possible that using that index to write somewhere in the memory might lead to very interesting behaviour.
Maybe you need to call DllMain manually or possibly - as you've got the library developer at hand - the DLL could provide an other entry point that initializes all necessary values? As said before: I'm not sure if DllMain should be called when using QLibrary!