Trying to use AddressSanitizer with Qt on Windows

[jazzco2]

@jsulm Yes, I deleted the build folder for a raw debug rebuild.

[jsulm]

@jazzco2 Is nine > 8?

[jazzco2]

@jsulm said in Trying to use AddressSanitizer with Qt on Windows:

@jazzco2 Is nine > 8?

Yes,, I have one screen -> it's nine. Writing it directly produces clang warnings but still runs. However there's no complaint from AdressSanitizer. The code has been added directly to main.cpp:

int nine = 9;
int x[9];
x[nine] = 5;
qDebug() << "X: " << x[nine];

[JoeCFD]

@jazzco2
MSVC has native ASan and you can turn it on in VS to see how the flag is defined.
Test your code in VS at first to see if Asan catches the bug.

Try

QMAKE_CXXFLAGS += -fsanitize=address
QMAKE_CFLAGS += -fsanitize=address

in pro file to see if asan is on. If there is an error message which complains about an Asan related lib can not be loaded, simply add the path in your build.

I can test the settings later at home.

[jazzco2]

@JoeCFD Thanks for the hint.

After rebuilding the app crashes now instantly - even without the illegal code. The stack trace looks like there's something about a failing file request. But the C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\<version>\bin\Hostx86\x86 to the clang_rt.asan_dynamic-x86_64.dll is in the path. I also copied the dll to the current exe-folder, started the app, and while it stopped in the debugger I tried to delete the dll - which was locked as expected. So this was in fact the dll being in use.

Here's the stack trace:

.  0  Id: 302c.447c Suspend: 1 Teb: 00000047`9202c000 Unfrozen
Child-SP          RetAddr               Call Site
(Inline Function) --------`--------     clang_rt_asan_dynamic_x86_64!__asan::QuickCheckForUnpoisonedRegion(void)+0x3b [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_interceptors_memintrinsics.h @ 59]
00000047`922fdfc0 00007ffb`768c5621     clang_rt_asan_dynamic_x86_64!__asan_wrap_strlen(char * s = 0x00007ffa`77f97678 "llvm-symbolizer.exe")+0x136 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc @ 425]
00000047`922fe850 00007ffb`741ef915     ntdll!RtlInitAnsiStringEx+0x21
00000047`922fe880 00007ffa`77f2c13a     KERNELBASE!GetFileAttributesA+0x25
00000047`922fe8d0 00007ffa`77f25aee     clang_rt_asan_dynamic_x86_64!__sanitizer::FileExists(char * filename = <Value unavailable error>)+0xa [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_win.cpp @ 105]
00000047`922fe900 00007ffa`77f37c56     clang_rt_asan_dynamic_x86_64!__sanitizer::FindPathToBinary(char * name = 0x00007ffa`77f97678 "llvm-symbolizer.exe")+0xe [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_file.cpp @ 207]
00000047`922fe970 00007ffa`77f37ff0     clang_rt_asan_dynamic_x86_64!__sanitizer::ChooseSymbolizerTools(struct __sanitizer::IntrusiveList<__sanitizer::SymbolizerTool> * list = 0x00000047`922fe9c0, class __sanitizer::LowLevelAllocator * allocator = 0x00007ffa`789a8068)+0x146 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_symbolizer_win.cpp @ 268]
00000047`922fe9a0 00007ffa`77f35a10     clang_rt_asan_dynamic_x86_64!__sanitizer::Symbolizer::PlatformInit(void)+0x30 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_symbolizer_win.cpp @ 291]
00000047`922fea10 00007ffa`77f772b3     clang_rt_asan_dynamic_x86_64!__sanitizer::Symbolizer::GetOrInit(void)+0x30 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_symbolizer_libcdep.cpp @ 24]
00000047`922fea50 00007ffa`77f77113     clang_rt_asan_dynamic_x86_64!__asan::AsanInitInternal(void)+0x183 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_rtl.cpp @ 575]
00000047`922feab0 00007ffb`73e5e716     clang_rt_asan_dynamic_x86_64!__asan::AsanInitFromRtl(void)+0x33 [D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_rtl.cpp @ 588]
00000047`922feae0 00007ffa`77f8d176     ucrtbase!initterm+0x36
00000047`922feb10 00007ffa`77f8d2e7     clang_rt_asan_dynamic_x86_64!dllmain_crt_process_attach(struct HINSTANCE__ * instance = 0x00007ffa`77f20000, void * reserved = 0x00000047`922ff500)+0x9a [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp @ 66]
00000047`922feb40 00007ffb`76852983     clang_rt_asan_dynamic_x86_64!dllmain_dispatch(struct HINSTANCE__ * instance = 0x00007ffa`77f20000, unsigned long reason = 1, void * reserved = 0x00000047`922ff500)+0x6f [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp @ 276]
00000047`922feba0 00007ffb`7685193c     ntdll!RtlDeactivateActivationContext+0xdd3
00000047`922fee80 00007ffb`7681bc7a     ntdll!RtlDeleteElementGenericTableAvlEx+0x4fc
00000047`922fef90 00007ffb`7681bca6     ntdll!LdrGetProcedureAddressForCaller+0xb1a
00000047`922fefd0 00007ffb`768456ac     ntdll!LdrGetProcedureAddressForCaller+0xb46
00000047`922ff010 00007ffb`76843864     ntdll!EtwEventWriteNoRegistration+0x266c
00000047`922ff3e0 00007ffb`7684364e     ntdll!EtwEventWriteNoRegistration+0x824
00000047`922ff460 00007ffb`76815fce     ntdll!EtwEventWriteNoRegistration+0x60e
00000047`922ff4b0 00000000`00000000     ntdll!LdrInitializeThunk+0xe

[jazzco2]

I created a minimal example and saw a clang warning about asan not working with a debug build:

However, building a release containing the invalid code in main.cpp the app runs without a crash, but the sanitizer doesn't recognize the wrong code.

#include "mainwindow.h"
#include <QApplication>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    MainWindow w;

    int nine = 9;
    int x[9];
    x[nine] = 5;
    qDebug() << "X: " << x[nine];

    w.show();
    return a.exec();
}

[JoeCFD]

@jazzco2 Take a look at how VS set Asan flags.

[jazzco2]

Thanks for the reminder. Unfortunately I'm really unfamiliar with VS. I managed to switch on the sanitizer but the VS build output declines to show a log output from the build like Qt does. I found a hint from 2017 that the project build output verbosity needs to be set to "Normal" at least, but that didn't help. I still just get for target ClCompile and Link the processed cpp or vcxproj filename only - no command line or parameters are shown.

Looks like that hint is outdated :-(

[jazzco2]

Update: One needs to select "Diagnostic" now, then in the middle of some thousand lines there appears the cl call :rolleyes:

[JoeCFD]

@jazzco2 Go through Commands in the settings in VS and you will be able to see the flags.

[jazzco2]

There are lots of differences in the command line options between VS and Qt. Basic project in both cases is the default GUI application.

Qt:

-c -nologo -Zc:wchar_t -FS -Zc:rvalueCast -Zc:inline -Zc:strictStrings -Zc:throwingNew -permissive- -Zc:__cplusplus -Zc:externConstexpr -fsanitize=address -Zi -MDd -std:c++17 -utf-8 -W3 -w34100 -w34189 -w44456 -w44457 -w44458 -wd4577 -wd4467 -EHsc /Fddebug\asan_test.vc.pdb -DUNICODE -D_UNICODE -DWIN32 -D_ENABLE_EXTENDED_ALIGNED_STORAGE -DWIN64 -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I..\..\dev\asan_test -I. -I..\..\Qt\6.9.0\msvc2022_64\include -I..\..\Qt\6.9.0\msvc2022_64\include\QtWidgets -I..\..\Qt\6.9.0\msvc2022_64\include\QtGui -I..\..\Qt\6.9.0\msvc2022_64\include\QtCore -Idebug -I. -I/include -I..\..\Qt\6.9.0\msvc2022_64\mkspecs\win32-msvc -Fodebug\ @C:\Users\jazzco\AppData\Local\Temp\main.obj.18184.15.jom

VS:

/JMC /permissive- /ifcOutput "WindowsASanTest\x64\Debug\" /GS /W3 /Zc:wchar_t /Zi /Gm- /Od /sdl /Fd"WindowsASanTest\x64\Debug\vc143.pdb" /Zc:inline /fp:precise /D "_DEBUG" /D "_WINDOWS" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /fsanitize=address /WX- /Zc:forScope /RTC1 /Gd /MDd /FC /Fa"WindowsASanTest\x64\Debug\" /EHsc /nologo /Fo"WindowsASanTest\x64\Debug\" /Fp"WindowsASanTest\x64\Debug\WindowsASanTest.pch" /diagnostics:column 

The options differ in VS are:

/JMC         = Just My Code
/WX-         = Treat Warnings as Errors OFF
/sdl         = Enables recommended Security Development Lifecycle 
/Od          = disable optimization
/D <>        = define constant/macro
/Gm-         = disable miminal build (deprecated)
/RTC1        = Enable fast runtime checks
/GS          = Checks buffer security
/fp:precise  = "precise" floating-point model; results are predictable.
/Zc:forScope = Enforce Standard C++ for scoping rules (on by default).
/Fo<>        = Creates an object file.
/external:W3 = Set warning level for external headers.
/Gd          = Uses the __cdecl calling convention. (x86 only)
/TP          = Specifies all source files are C++.
/FC          = Displays the full path of source code files passed to cl.exe in diagnostic text.
/errorReport = deprecated

I added several options to Qt, like -JMC -GS -sdl -RTC1 to get it up but the process still crashes in debug mode at the same spot even with a clean default project without the faulty source code.

In release mode it works but doesn't recognize the malicious code.

EDIT: fixed wrong VS command line