Unable to debug code from one specific library
-
Have been using Qt for quite some time, but have been stumped by this occurrence. I have developed a number of libraries, both Qt and non-Qt that I re-use in most of the applications I develop. I have always been able to set breakpoints in all of the libraries, whether the source code is part of the present project or not. I just open the appropriate source code, set the breakpoint, and wait for the present project to reach that place in the libraries code.
For one specific project, however, that fails. When the app is not executing, I am able to open the source code and set some appropriate breakpoints. As would normally be the case, I can double click on the various breakpoints and the editor jumps to the associated location in the source code.
However, once execution starts, I can no longer access these breakpoints. Further, rather than stop at the desired breakpoint, execution stops at the place in my current project code that calls that library routine. The result is that I cannot debug the code for this library. While the code is executing, When I click on a breakpoint in this library, I get an error that Qt "could not open [the source code] for reading. Either the file does not exist or you do not have the permissions to open it.
I suspect this result is from some compiler or linker option that somehow disables access to the debug info. There are quite a few such options, and I am not familiar with a bunch of them. I am not aware of what details in the source code itself that could disable my ability to debug, but I suppose such could exist.
I hope to get a few suggestions for the sorts of things that can lead to this behavior, because the code in question really deserves a bit of debugging.
Thanks
-
It's quite difficult to understand what's going on without more details.
When I got a similar issue in the past, was due the fact that at runtime the application was finding a release library that got installed in the system (by a previous make install I did on release mode), but when the application was stopped, the Qt Creator was let me navigate the code of the library in debug mode.
So, my first suggestion is to try to find in your whole computer the library in question and see if there are some of them got installed in some system directory or in a path that at runtime the application can load libraries from.
-
It would be helpful if you mentioned which platform, compiler and debugger you're using, as the debugging setup differs from toolchain to toolchain.
In any case the first step is to verify if you're loading the debug version of that library. Might sound silly but it's a common mistake and it would be a shame not to check.
Next would be to confirm if you have debug symbols loaded for that lib.
Both these things you can check in the modules window of Qt Creator when you're running the app under debugger. (Window->Views->Modules if you don't have it shown).
If both of these check out the next step is to verify if the debug info has the right paths to the sources. This step varies depending on the toolchain and debug info format so can't help you without that information. If the paths don't match Qt Creator lets you setup path mapping in the debugger options (Options->Debugger->General tab-> Source Paths Mapping). -
It would be helpful if you mentioned which platform, compiler and debugger you're using, as the debugging setup differs from toolchain to toolchain.
In any case the first step is to verify if you're loading the debug version of that library. Might sound silly but it's a common mistake and it would be a shame not to check.
Next would be to confirm if you have debug symbols loaded for that lib.
Both these things you can check in the modules window of Qt Creator when you're running the app under debugger. (Window->Views->Modules if you don't have it shown).
If both of these check out the next step is to verify if the debug info has the right paths to the sources. This step varies depending on the toolchain and debug info format so can't help you without that information. If the paths don't match Qt Creator lets you setup path mapping in the debugger options (Options->Debugger->General tab-> Source Paths Mapping).@Chris-Kawa Thanks.
-
Running Windows 10, and using g++ from MinGW-w64-x86 (recent version) as the compiler and the corresponding gdb for the debugger.
-
I am loading the debug version of the library - at least the one with the "d" suffix. I am "pretty" confident I created it with the appropriate debug switches set properly. However, I did build this library by a somewhat different means than I normally use, so that leaves some room for doubt.
-
not confident I'm looking at the debug symbols question correctly. The "modules" window shows that for my main executable, the "Symbols Read" and "Symbols Type" are "Unknown" - but the debugger reads and displays them correctly. When I tried this the first time, both for the executable and for the library in question, these were both "yes". Not at all sure what has changed.
-
Pretty sure the debugger has the correct paths, but not sure how to verify. Execution breaks at all the right times, but at a location in my executable's code that corresponds to where a breakpoint would be encounted in the library - just not in the library code. There are no explicit source path mappings configured - but I have never set any and never needed to set any with any other library.
-
-
Have been using Qt for quite some time, but have been stumped by this occurrence. I have developed a number of libraries, both Qt and non-Qt that I re-use in most of the applications I develop. I have always been able to set breakpoints in all of the libraries, whether the source code is part of the present project or not. I just open the appropriate source code, set the breakpoint, and wait for the present project to reach that place in the libraries code.
For one specific project, however, that fails. When the app is not executing, I am able to open the source code and set some appropriate breakpoints. As would normally be the case, I can double click on the various breakpoints and the editor jumps to the associated location in the source code.
However, once execution starts, I can no longer access these breakpoints. Further, rather than stop at the desired breakpoint, execution stops at the place in my current project code that calls that library routine. The result is that I cannot debug the code for this library. While the code is executing, When I click on a breakpoint in this library, I get an error that Qt "could not open [the source code] for reading. Either the file does not exist or you do not have the permissions to open it.
I suspect this result is from some compiler or linker option that somehow disables access to the debug info. There are quite a few such options, and I am not familiar with a bunch of them. I am not aware of what details in the source code itself that could disable my ability to debug, but I suppose such could exist.
I hope to get a few suggestions for the sorts of things that can lead to this behavior, because the code in question really deserves a bit of debugging.
Thanks
Hi @drmhkelley,
debugging such problems is indeed not easy, and without having the sources nearly impossible.
You can use Windows > Views > Debugger Log to see the interaction with the debug engine, probably that gives some more information.
One more thing that cames to my mind immediately, AFAIK there are problems if functions have the same name in different projects, the debugger might not be able to resolve them.
If you want more help from us, the usual information is needed:
- platform
- compiler
- debugger
Regards
-
Hi @drmhkelley,
debugging such problems is indeed not easy, and without having the sources nearly impossible.
You can use Windows > Views > Debugger Log to see the interaction with the debug engine, probably that gives some more information.
One more thing that cames to my mind immediately, AFAIK there are problems if functions have the same name in different projects, the debugger might not be able to resolve them.
If you want more help from us, the usual information is needed:
- platform
- compiler
- debugger
Regards
@aha_1980 Well, again I find myself stumped by some pretty deep intricacies in how Windows uses libraries - in particular DLLs.
- My immediate debugging problem seems to be that the debugger can't find the source code for the library giving me trouble. I found this note in the Global Debugger Log: "No source file named xxxxxx.cpp", where xxxxxx.cpp is the file I'm trying to debug. Presumably, that is why the debugger doesn't know what to do with my breakpoints.
I have been unsuccessful telling Creator how to provide the source code location to the debugger.
- I have not been successful making this library the same way I make all of my others.
In the past, I have always used "ar" to create a library, say mylib, as libmylib.a and to link with the executable with -lmylib where the loader has libmylib.a in its search path. There has never been specific need to tell the debugger anything special - it always just found the source code, presumably from association of the breakpoint with where it was set in the source code via the Creator editor.
This library is created using g++ with options I don't fully understand. It creates mylib.dll, which is what the loader uses to create the executable for my app, and which must be in my PATH when I execute the app. The reason I do it this way is because it works. if I use my usual approach, the loader fails with a bunch of unresolved references having mangled names like "__imp__ZN5Qxxxx6FormatD1Ev". I fought a similar problem a few years back, but never fully understood how I made it go away. But it was also related to how libraries were linked and loaded.
Bottom line, I am considerably confused by all things related to .a and .dll libraries, and would very much like to know how to know for certain that the dll in question has the relevant debugging info and how to tell the debugger how to find it.
Thanks for listening to my tirade.
-
@aha_1980 Well, again I find myself stumped by some pretty deep intricacies in how Windows uses libraries - in particular DLLs.
- My immediate debugging problem seems to be that the debugger can't find the source code for the library giving me trouble. I found this note in the Global Debugger Log: "No source file named xxxxxx.cpp", where xxxxxx.cpp is the file I'm trying to debug. Presumably, that is why the debugger doesn't know what to do with my breakpoints.
I have been unsuccessful telling Creator how to provide the source code location to the debugger.
- I have not been successful making this library the same way I make all of my others.
In the past, I have always used "ar" to create a library, say mylib, as libmylib.a and to link with the executable with -lmylib where the loader has libmylib.a in its search path. There has never been specific need to tell the debugger anything special - it always just found the source code, presumably from association of the breakpoint with where it was set in the source code via the Creator editor.
This library is created using g++ with options I don't fully understand. It creates mylib.dll, which is what the loader uses to create the executable for my app, and which must be in my PATH when I execute the app. The reason I do it this way is because it works. if I use my usual approach, the loader fails with a bunch of unresolved references having mangled names like "__imp__ZN5Qxxxx6FormatD1Ev". I fought a similar problem a few years back, but never fully understood how I made it go away. But it was also related to how libraries were linked and loaded.
Bottom line, I am considerably confused by all things related to .a and .dll libraries, and would very much like to know how to know for certain that the dll in question has the relevant debugging info and how to tell the debugger how to find it.
Thanks for listening to my tirade.
So you're on Windows, right?
You still didn't tell us, which compiler you use. From
gccI assume it would be MinGW, but could also by cygwin. You probably know, that on Windows you cannot mix libraries from different compilers; and I assume the library in question is build with debug symbols.This library is created using g++ with options I don't fully understand.
Well, that may be true, but how can I help you there? You didn't gave us the options to tell you if there is anything fishy.
and which must be in my PATH when I execute the app.
Which brings us to another point: Are you sure the correct library is loaded, and not an older version that is somewhere on your hard disk?
Regards
-
@aha_1980 OP already said he's using MinGW-w64-x86 on Windows 10.
which is what the loader uses to create the executable for my app, and which must be in my PATH when I execute the app. The reason I do it this way is because it works.
That's horrible. Both the fact you're using PATH for that and the reason for it.
You should have a build step that copies dependencies to your output directory.There are 2 ways to link a library - static and dynamic. Static library is just a .a file containing the actual code that is linked into the executable at link time. Dynamic libraries consist of a .a file and a .dll file. The dll contains the actual code while the .a file is linked into the executable and contains just the "glue" code that loads the dll at runtime and resolves function pointers from it.
In case of static libraries the debug information is placed inside the executable (at least MinGW does that, MSVC uses separate .pdb files). In case of dynamic libraries the debug info is split into the executable and the dll, each containing the info for the code it holds.
Debug info consists, among other things, of source file name and path the library or executable was build from. When you start a debug session and hit a breakpoint the debugger looks for the source file specified in the debug info in the following locations: next to the library dll file, in the path contained in the debug info which is the original location at compile time, in some debugger specific locations (I don't know what they are for GDB specifically, I suspect some env variable) and in the mapped directories.So, if the sources are at the location specified in the debug info then it will "just work". If you moved the sources, renamed a directory they are in or have the .a and .dll from some other machine which had a different path then the debugger won't find it.
So the steps are: Make sure you're linking to the debug version. Make sure the debugger loads the debug info. Check where it looks for the sources. Check if the sources exist at exactly that location. If not either move them to that exact location or use the path mapping feature of Qt Creator.
-
@aha_1980 OP already said he's using MinGW-w64-x86 on Windows 10.
which is what the loader uses to create the executable for my app, and which must be in my PATH when I execute the app. The reason I do it this way is because it works.
That's horrible. Both the fact you're using PATH for that and the reason for it.
You should have a build step that copies dependencies to your output directory.There are 2 ways to link a library - static and dynamic. Static library is just a .a file containing the actual code that is linked into the executable at link time. Dynamic libraries consist of a .a file and a .dll file. The dll contains the actual code while the .a file is linked into the executable and contains just the "glue" code that loads the dll at runtime and resolves function pointers from it.
In case of static libraries the debug information is placed inside the executable (at least MinGW does that, MSVC uses separate .pdb files). In case of dynamic libraries the debug info is split into the executable and the dll, each containing the info for the code it holds.
Debug info consists, among other things, of source file name and path the library or executable was build from. When you start a debug session and hit a breakpoint the debugger looks for the source file specified in the debug info in the following locations: next to the library dll file, in the path contained in the debug info which is the original location at compile time, in some debugger specific locations (I don't know what they are for GDB specifically, I suspect some env variable) and in the mapped directories.So, if the sources are at the location specified in the debug info then it will "just work". If you moved the sources, renamed a directory they are in or have the .a and .dll from some other machine which had a different path then the debugger won't find it.
So the steps are: Make sure you're linking to the debug version. Make sure the debugger loads the debug info. Check where it looks for the sources. Check if the sources exist at exactly that location. If not either move them to that exact location or use the path mapping feature of Qt Creator.
@aha_1980 OP already said he's using MinGW-w64-x86 on Windows 10.
Ah, right. I somehow missed that post on my phone and didn't scroll back later.
-
@aha_1980 OP already said he's using MinGW-w64-x86 on Windows 10.
which is what the loader uses to create the executable for my app, and which must be in my PATH when I execute the app. The reason I do it this way is because it works.
That's horrible. Both the fact you're using PATH for that and the reason for it.
You should have a build step that copies dependencies to your output directory.There are 2 ways to link a library - static and dynamic. Static library is just a .a file containing the actual code that is linked into the executable at link time. Dynamic libraries consist of a .a file and a .dll file. The dll contains the actual code while the .a file is linked into the executable and contains just the "glue" code that loads the dll at runtime and resolves function pointers from it.
In case of static libraries the debug information is placed inside the executable (at least MinGW does that, MSVC uses separate .pdb files). In case of dynamic libraries the debug info is split into the executable and the dll, each containing the info for the code it holds.
Debug info consists, among other things, of source file name and path the library or executable was build from. When you start a debug session and hit a breakpoint the debugger looks for the source file specified in the debug info in the following locations: next to the library dll file, in the path contained in the debug info which is the original location at compile time, in some debugger specific locations (I don't know what they are for GDB specifically, I suspect some env variable) and in the mapped directories.So, if the sources are at the location specified in the debug info then it will "just work". If you moved the sources, renamed a directory they are in or have the .a and .dll from some other machine which had a different path then the debugger won't find it.
So the steps are: Make sure you're linking to the debug version. Make sure the debugger loads the debug info. Check where it looks for the sources. Check if the sources exist at exactly that location. If not either move them to that exact location or use the path mapping feature of Qt Creator.
@Chris-Kawa Thanks for the input.
"That's horrible. Both the fact you're using PATH for that and the reason for it." - I'd be happy to learn a better way.
"a build step that copies dependencies to your output directory" - I don't know what this means or how to accomplish it. In general, I have control over where the output directory is and what goes into it but I don't understand putting "dependencies" in it.
I broadly understand static vs dynamic linking, but certainly do not many fine details. For the bulk of libraries I create, I just make the .a with no affiliated .dll. One of my big confusions is why the relevant info is not available through whatever mechanism it always has been. For the immediate situation, the .a and .dll are in the same location, and I think the compile process specifies that location. Not sure what you mean by "mapped directories". Mapped where and by whom?
The sources are where they were at compile time. I have, and have always, copied the created libraries to a general library location. that is somehow magically found when some app is ultimately executed. All of the libraries under question were built by me from source with the same set of tools.
I don't know how to make the debugger load debug info and I apparently don't know where it looks for sources. I have monkeyed around with the path mapping feature, but apparently not yet in a useful way
for reference, below is an extract from the transcript for the build process for this library showing the exact compiler and linker options. Below that is a comparable transcript from building another of my libraries. The second library is rather heavily used and regularly part of a debugging session.
\usr\local\MinGW-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin\g++ -c -MT debug/zzzzsomefile.o -MMD -MP -MF debug.d/zzzzsomefile.Td -g -fno-keep-inline-dllexport -Og -std=c++1z -fno-exceptions -Wall -W -Wextra -Wvla -Wdate-time -Wshift-overflow=2 -Wduplicated-cond -Wno-stringop-overflow -DUNICODE -D_UNICODE -DWIN32 -DMINGW_HAS_SECURE_API=1 -DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -DQT_NO_NARROWING_CONVERSIONS_IN_CONNECT -DQT_BUILD_zzzz_LIB -DQT_BUILDING_QT -D_CRT_SECURE_NO_WARNINGS -D_USE_MATH_DEFINES -DQT_NO_CAST_TO_ASCII -DQT_ASCII_CAST_WARNINGS -DQT_MOC_COMPAT -DQT_USE_QSTRINGBUILDER -DQT_DEPRECATED_WARNINGS -DQT_DISABLE_DEPRECATED_BEFORE=0x040800 -DQT_NO_EXCEPTIONS -DQT_QML_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -I. -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtGui\5.12.0 -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtGui\5.12.0\QtGui -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtGui -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtCore\5.12.0 -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtCore\5.12.0\QtCore -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtCore -isystem \usr\local\Boost\boost_1_69_0\include\boost_1_69 -o debug/zzzzsomefile.o zzzzsomefile.cpp
\usr\local\MinGW-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin\g++ -Wl,-s -shared -Wl,-subsystem,windows -Wl,--out-implib,debug/libzzzzzzzd.a -o debug/zzzzzzzd.dll long list of .o files -LC:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\lib C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\lib\libQt5Gui.a C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\lib\libQt5Core.a debug/zzzzzzzd_resource_rec.o -lglu32 -lopengl32 -lgdi32 -luser32
copy debug\libzzzzzzzd.a \users\owner\documents\mike\src\lib\x86_64-7.2.0-posix-seh-rt_v5-rev1\gcc_dll
copy debug\zzzzzzzd.dll \users\owner\documents\mike\src\lib\x86_64-7.2.0-posix-seh-rt_v5-rev1\gcc_dll
\usr\local\MinGW-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin\g++ -c -MT debug/myLogWindow.o -MMD -MP -MF debug.d/myLogWindow.Td -g -fno-keep-inline-dllexport -O0 -std=gnu++11 -Wall -W -Wextra -fexceptions -mthreads -DUNICODE -D_UNICODE -DWIN32 -DQT_QML_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_NEEDS_QMAIN -I. -I\users\owner\documents\mike\src\include -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1/mkspecs\win32-g++ -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtWidgets -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtGui -isystem C:\usr\local\Qt\Qt-5.12_x86_64-7.2.0-posix-seh-rt_v5-rev1\include\QtCore -isystem \usr\local\Boost\boost_1_69_0\include\boost_1_69 -o debug/myLogWindow.o myLogWindow.cpp
ar -ruv debug/libmyqtlibd.a debug/myLogWindow.o *****long list of .o files *****
copy debug\libmyqtlibd.a \users\owner\documents\mike\src\lib\x86_64-7.2.0-posix-seh-rt_v5-rev1\gcc_dll\libmyqtlibd.a
