std::string destructor crashing in Release when using ::toStdString methods



  • I'm developing an app using QT 5 on Windows. What I am seeing is MSVC10 in RELEASE mode causing std::string to throw an exception. The application then crashes, hitting a breakpoint in the destructor's call to _tidy(). The call stack is useless in this case, as there is a stack/heap corruption. I turned on /DEBUG symbols for the project and walked the code from the debugger. Here is the distilled version of the code I am running:

    Foo::doSomething(const QString& temp)
    {
    std::string temp_str = temp.toStdString();
    <do something with temp_str>
    }

    What I am seeing is that the application crashes in the destructor for the std::string object "temp_str" when the method goes out of scope. This only happens in RELEASE mode. In DEBUG mode, everything works fine.

    I am using the DLLs that where installed with Qt via the binary installer. I did not compile Qt for my platform as the installer package already contains what I need.

    Any ideas what could be going wrong?


  • Moderators

    If it's 100% reproducible and you're sure this isn't a random symptom of some other bug in your code then it might be a case of mixed debug/release dlls. Make sure your app doesn't use debug dllls in release mode by running it through profile mode in Dependency Walker or Process Explorer.



  • I would add to Chris's comment, if this under Window you have to make sure all dependencies use the same C runtime library.
    It is a bit more then just release/debug , it also static and dynamic C runtime, multi-/single-threaded version ( last is mostly disappeared from modern compilers, but just in case) .



  • Thanks for the replies. I am trying to navigate DependencyWalker results now. I don't see anything obvious so far.

    By the way, I misspoke in my original post. It is NOT Qt that is crashing explicitly. It is the destructor for the std::string that is crashing, but only when the std::string in question is generated from QString::toStdString method



  • @DRoscoe said:

    std::string temp_str = temp.toStdString();

    Can you try what happens with:

    std::string temp_str(temp.toStdString());
    

    ?



  • That was how the code originally was written, when I first observed the problem. I though perhaps I could solve it by creating a non-temporary copy, but it didn't work.

    DependencyWalker did not show me anything obvious. At least I didn't see any reference to debug DLL's. I'm in the process of recompiling the Qt Libraries on my compiler. I am using Visual Studio 2010



  • Hi,

    could you post the code inside your function?



  • The example I originally posted is already sufficient to reproduce the problem in my environment, but here is the exact code that is causing the problem:

    void CommunicationHandler::decodeMessage(const QByteArray& msg_in)
    {
      //DAR@20150416 Because the NAV Database contains a large binary payload, we
      //             need a way to omit this portion of the message for logging
      if (cur_msg_type == TapMessageClass::NAV_DATABASE)
      {
        // The NAV Database is an AOP formatted binary object which begins with the
        // letters "aop" and a null terminator.
        QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));
        TapMessagingLogger::instance()->logMessage(TapMessagingLogger::RECEIVED,
                                                   to_log.toStdString());
      }
    

    When the call to logMessage goes out of scope, it crashes. If I modify the code as follows:

    void CommunicationHandler::decodeMessage(const QByteArray& msg_in)
    {
      //DAR@20150416 Because the NAV Database contains a large binary payload, we
      //             need a way to omit this portion of the message for logging
      if (cur_msg_type == TapMessageClass::NAV_DATABASE)
      {
        // The NAV Database is an AOP formatted binary object which begins with the
        // letters "aop" and a null terminator.
        QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));
        std::string temp_str = to_log.toStdString();
        TapMessagingLogger::instance()->logMessage(TapMessagingLogger::RECEIVED, temp_str);
      }
    

    The crash occurs when the if() clause goes out of scope. It is important to note that this code causes the crash only because its the first time this type of operation is performed. If I prevent it from executing, the problem occurs at a different point using a similar operation.

    The bottom line is, in Release mode, whenever a std::string is produced using ::toStdString, the std::string destructor will cause a crash. I added a code snippet like the one in my original post in the constructor of my main application class and it crashed immediately



  • Hi,

    if I understood correctly you're trying to log something that is in binary format; IIRC toStdString() converts Unicode (QString) into a std::string using utf-8 encoding.

    Could you try to use directly QByteArray::toStdString() instead of passing through a QString? In that case you can avoid encoding step



  • I am not trying to log the binary portion of the payload. The QByte array is an array of chars from a socket read. The front portion of the array is string data representing the header of the message. The following line:

    QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));
    

    Isolates the string portion, starting at position zero, continuing until the first instance of "aop\0", which denotes the start of the binary payload. The resulting QString contains just the header, which is string data. It is this portion that is being logged The binary payload is handled farther downstream.



  • Ok,

    have you tried to Debug you application to understand what happens??

    And, as I said before, you can use directly QByteArray::toStdString() do avoid double conversion ascii --> UNICODE --> ascii



  • In my original post, I pointed out that this has been impossible to debug thus far, and believe me I tried for quite a few days before my first post. The call stack is unusable as the crash occurs deep within Qt and and Windows libraries for which I have nothing but disassembly code to look at. The call to_tidy() in the std::string is what's crashing indicating a stack/heap corruption, and the crash report supports this.

    Here is another example of a crash that WAS occurring, which is much more straightforward:

    CommunicationHandler::CommunicationHandler(QObject* owner_in)
    : QObject(owner_in)
    {
      QString cur_dir("../output/");
    
      dna::FilenameTimestamp start_timestamp;
      std::string filename = 
        start_timestamp.getTimestampedFilename(
                                        cur_dir.toStdString(),
                                        "TAPDisplayMessagingLog",
                                        ".xml");
      TapMessagingLogger::instance()->openLogFile(filename);
    }
    

    In the above case, the crash was occurring in the call to ::getTimestampedFilename() This was due to the destruction of the temporary std::string created from the call to cur_dir.toStdString() in the argument list

    I modified the code to eliminate the temporary variable thinking something might be wrong there:

    CommunicationHandler::CommunicationHandler(QObject* owner_in)
    : QObject(owner_in)
    {
      QString cur_dir("../output/");
      std::string dir_str = cur_dir.toStdString();
    
      dna::FilenameTimestamp start_timestamp;
      std::string filename =
        start_timestamp.getTimestampedFilename(dir_str ,
                                               "TAPDisplayMessagingLog",
                                               ".xml");
      TapMessagingLogger::instance()->openLogFile(filename);
    }
    

    In this case, the crash was occurring when the constructor went out of scope. Again, the crash was occurring in the _tidy() method of std::string while trying to destroy dir_str. I modified the code once again, eliminating the initial QString altogether:

    CommunicationHandler::CommunicationHandler(QObject* owner_in)
    : QObject(owner_in)
    {
      std::string dir_str = "../output/";
    
      dna::FilenameTimestamp start_timestamp;
      std::string filename =
        start_timestamp.getTimestampedFilename(dir_str ,
                                               "TAPDisplayMessagingLog",
                                               ".xml");
      TapMessagingLogger::instance()->openLogFile(filename);
    }
    

    And the crash was resolved. This is not a code problem. There is some insidious problem with my compiled code and the precompiled binaries installed with the Qt Installer. I am in the process of trying to compile the Qt Libraries from source using my compiler to see if this resolves the problem



  • Compiling the Qt library did not resolve the problem.



  • Hi,

    I created a sample program:

    #include <QFileDialog>
    
    #include <string>
    
    Widget::Widget(QWidget *parent) :
        QWidget(parent),
        ui(new Ui::Widget)
    {
        ui->setupUi(this);
    }
    
    Widget::~Widget()
    {
        delete ui;
    }
    
    void Widget::on_pushButton_clicked()
    {
        std::string my_str = ui->lineEdit->text().toStdString();
    
        qDebug("Hello %s", my_str.c_str());
    }
    
    void Widget::on_lineEdit_textChanged(const QString &arg1)
    {
        ui->pushButton->setDisabled(arg1.isEmpty());
    }
    
    void Widget::on_toolButton_clicked()
    {
        QString fileName = QFileDialog::getOpenFileName(
                    this, tr("Choose a file"), ".");
    
        if(!fileName.isEmpty())
            ui->lineEdit->setText(fileName);
    }
    

    In a form I use a QFileDialog to choose a file from the file-system and when a button is clicked I convert the fileName to a std::string and print it to the stdout.
    I run this program without crash.

    I'm on OS X with Qt 5.4.1.
    In your function use the the conversion with toStdString() and comment the following lines; verify is the crash is still there

    CommunicationHandler::CommunicationHandler(QObject* owner_in)
    : QObject(owner_in)
    {
        QString cur_dir("../output/");
        std::string dir_str = cur_dir.toStdString();
    
        // dna::FilenameTimestamp start_timestamp;
        // std::string filename =
            // start_timestamp.getTimestampedFilename(dir_str ,
            //                                   "TAPDisplayMessagingLog",
            //                                   ".xml");
    //  TapMessagingLogger::instance()->openLogFile(filename);
    }
    


  • I posted almost the exact same pseudocode in my original post, and yes I have already tried this in code, as I alluded to in a follow-up post. On my system it crashes, which is a Windows system. I don't have this problem on the Mac. This is a problem (for me) on the Windows platform using MSVC2010.



  • Qt and your code are probably not compiled with the same STL version. Something like this:
    http://stackoverflow.com/questions/14090341/difference-in-byte-size-of-stl-containers-in-visual-studio-2010-and-2012



  • @sandy.martel23 said:

    http://stackoverflow.com/questions/14090341/difference-in-byte-size-of-stl-containers-in-visual-studio-2010-and-2012

    I recompiled the Qt Library from source using the same compiler and that did not solve the problem. I am trying different C++11 options now to see if that's implicated



  • I think I might be on the right track to figuring out what's going on. My project builds three executables. I recreated the same code that causes a crash in the problem executable in another of the remaining executables and the crash is not reproducible there. This leads me to suspect there is an alignment issue or something else that is affecting ONLY this one executable.



  • My previous assertion that only one executable had the problem was false. I quickly mocked up the following in the main method of all three executables and got similar results:

    using std::string;
    
    int main(int argc, char *argv[])
    {
      {
        QString temp("blah");
        string temp_str = temp.toStdString();
        std::cout << temp_str << std::endl;
        std::cout << "blah2" << std::endl;
        string temp_str2 = temp_str;
        temp_str2.append("a");
        std::cout << temp_str2 << std::endl;
      }
    

    The output:

    £♫  â─♦â° un §$
    blah2
    £♫  â─♦â° un §$a
    

    As you can see, the string is already junk when it's streamed to std::cout but looks fine in the watch list in the debugger. The string literal outputs fine. The copying of temp_str to temp_str2 does not crash, nor does the append call, as evidenced by the last line. When the code goes out of scope, the crash occurs as normal. This suggests that even the original call to temp.toStdString() resulted in what looked like a valid string, but the std::cout results and the copy output clearly shows that something is wrong even before we go out of scope.



  • Do you have MSVC10 service packs installed?
    I remember MSVC10 in 64 bit had a nasty bug which took microsoft about a year to fix.



  • I'm running 10.0.40219.1 SP1Rel

    I have many other large legacy projects which don't have any problems with STL in general. It's just the use of Qt's STL conversions that are presenting a problem.



  • SP1 should have solved the problem I mentioned, so it is not it.



  • @DRoscoe said:

    I recompiled the Qt Library from source using the same compiler and that did not solve the problem.

    But it seems that your still mixing debug and release STL object.
    sizeof( debug std::string ) != sizeof( release std::string ).



  • DependencyWalker has confirmed I am not using Debug STL libraries. However, I will use the sizeof() test as you suggest. At this point, I'll try anything!



  • I ran the test and the results support DependencyWalker:

    21:47:49.858 INFO: SIZE OF STD::STRING (DEBUG): 32
    21:51:34.968 INFO: SIZE OF STD::STRING (RELEASE): 32


  • Moderators

    Hi @DRoscoe,

    1. Did you run your RELEASE program inside the IDE, or by double-clicking the executable?
    2. Could you post the contents of your PATH?
       QString temp("blah");
       string temp_str = temp.toStdString();
       std::cout << temp_str << std::endl;
       std::cout << "blah2" << std::endl;
       string temp_str2 = temp_str;
       temp_str2.append("a");
       std::cout << temp_str2 << std::endl;
    

    The output:

    £♫ â─♦â° un §$
    blah2
    £♫ â─♦â° un §$a

    There's something very wrong indeed. What happens when you:

    • Print through qDebug() instead of std::cout?
    • Print the QString itself?
    • Do an indirect conversion that bypasses QString::toStdString()?
    QString temp;
    QByteArray arr = temp.toUtf8();
    std::string temp_str(arr.data());
    
    qDebug() << temp;
    qDebug() << arr;
    qDebug() << temp_str.c_str();
    

    At this point, I'll try anything!

    One other thing I can think of is to install the MinGW version of Qt and use that to build you project. The binaries are very different, and might free you from the strange string issue.



  • I modified the code as suggested:

      {
        QString temp("blah");
        qDebug() << "QString: " << temp;
        string temp_str = temp.toStdString();
        qDebug() << "TEMP_STR: " << temp_str.c_str();
        qDebug() << "blah2";
        string temp_str2 = temp_str;
        temp_str2.append("a");
        qDebug() << temp_str2.c_str();
      }
    

    In this case, the copy from temp_str to temp_str2 caused the crash. The output from what did execute:

    QString:  "blah"
    TEMP_STR:
    blah2
    

    In both tests, I ran from the IDE, since the behavior is easily reproducible. However, I also ran it from the command line with the same results. Here is the contents of my PATH variable:

    C:\Users\roscoe.NASALAB>echo %PATH%
    C:\Python27\;C:\Python27\Scripts;C:\Perl64\site\bin;C:\Perl64\bin;C:\Program Fil
    es (x86)\IBM\RationalSDLC\common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Pr
    ogram Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System3
    2\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(
    R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Manage
    ment Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Com
    ponents\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\I
    PT;c:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;c:\Program Files\
    Microsoft SQL Server\100\Tools\Binn\;c:\Program Files\Microsoft SQL Server\100\D
    TS\Binn\;C:\Program Files (x86)\IBM\RationalSDLC\ClearCase\bin;C:\Program Files\
    SourceGear\Common\DiffMerge\;C:\Program Files\Microsoft Windows Performance Tool
    kit\;C:\Program Files (x86)\Common Files\Plantronics\;C:\Program Files (x86)\Pla
    ntronics\MyHeadsetUpdater\;C:\Program Files (x86)\Plantronics\VoyagerEdge\;C:\Pr
    ogram Files\IBM\RationalPurifyPlus;C:\Ruby21\bin;%APPDATA%\Python\Scripts
    

    Using an indirect conversion as you suggest would not be convenient, thus I've avoided it, but may be my only option. I implemented the following:

      {
        QString temp("blah");
        QByteArray arr = temp.toUtf8();
        std::string temp_str(arr.data());
    
        qDebug() << temp;
        qDebug() << arr;
        qDebug() << temp_str.c_str();
      }
    

    The program did not crash and the output:

    "blah"
    "blah"
    blah
    

    I could try the minGW version, but I have concerns. Our Visual Studio projects are generated using CMake. The CMake scripts generate our project files for Windows, Linux/Unix and MacOS. I am fairly certain it can handle an output compatible with MinGW, but may introduce difficulties if changes need to be made, as the ripples could cause us to rework the scripts to maintain the ability to generate project files for the other platforms. Again, I may have no other option, so I won't discount it.

    Another thing I have not considered, we use the Adaptive Communications Environment (ACE) which is a public-domain, platform-independent middleware that abstracts our threading, networking and many other tasks. This requires that our executables have ACE_main as its entry point. Since its only an entry point to the main execution thread, I doubt it's implicated, but I do plan on trying a sample app without it to remove this variable to be certain. Nothing is making sense, so it's worth a shot. Without a fix for this problem our project is dead in the water with a deadline looming.


  • Moderators

    @DRoscoe said:

    In this case, the copy from temp_str to temp_str2 caused the crash. The output from what did execute:

    QString:  "blah"
    TEMP_STR:
    blah2
    

    There's no doubt then that the std::string is being constructed with invalid data (possibly with an invalid char* pointer).

    Here is the contents of my PATH variable:

    I was looking for common crash culprits in Qt-based projects, but didn't find any in your PATH. I don't recognize a number of the tools you're using though.

    Using an indirect conversion as you suggest would not be convenient, thus I've avoided it, but may be my only option.
    ...
    The program did not crash and the output:

    "blah"
    "blah"
    blah
    

    Ok, we now have one workaround, at least.

    The implementation of QString::toStdString() is pretty straightforward:

    Maybe try temp.toUtf8().toStdString() and see if that avoids the crash. If so, perhaps you can do a find-and-replace.

    I could try the minGW version, but I have concerns.

    Agreed; that is a drastic change. I would use the multi-step conversion, personally.

    Another thing I have not considered, we use the Adaptive Communications Environment (ACE).... I do plan on trying a sample app without it to remove this variable to be certain.

    Yes, do test it.



  • I tested the shorthand conversion you suggested with the same result. I have also confirmed that the ACE_main entry is not causing a problem. What I have done is implement a static function to do the conversion you gave me, that I've already confirmed works. I hate not having a proper solution for this, but at least I can move forward.

    I'm not going to mark this one as solved, as it technically is not. I am planning on trying this on MSVC 2013 to see if it makes a difference.

    Thanks!


  • Moderators

    @DRoscoe:

    You're welcome :)

    I tested the shorthand conversion you suggested with the same result. I have also confirmed that the ACE_main entry is not causing a problem.

    Very very strange. It appears that an QByteArray::toStdString() doesn't work on rvalues in your system. I've never seen this before.

    (For academic curiosity, I'm tempted to try qDebug() << QByteArray("blah").toStdString();)

    I am planning on trying this on MSVC 2013 to see if it makes a difference.

    If you have time and a fresh PC, try the same compiler version and Qt version on the fresh PC. I'm curious to know if the corruption is due to your environment, or a long-standing and undetected bug.

    But, I know you have a deadline to meet; these experiments aren't crucial. (I did try string temp_str = temp.toStdString(); qDebug() << "TEMP_STR: " << temp_str.c_str(); on my machine (MSVC 2013 32-bit, Qt 5.4.1) but it was fine in both Debug and Release mode.)

    Anyway, all the best!



  • " (For academic curiosity, I'm tempted to try qDebug() << QByteArray("blah").toStdString();)"

    I tried this with the same result. I had to add .c_str() as QDebug does not support std::string

    "If you have time and a fresh PC, try the same compiler version and Qt version on the fresh PC. I'm curious to know if the corruption is due to your environment, or a long-standing and undetected bug."

    I probably won't have time to try this fully, but I do know other developers in our group are experiencing the same problem

    UPDATE: I created an empty MSVC2010 console application project that does nothing but bring in QT5Core and a few other Qt support libraries and tried it from there. Same behavior. It's something very basic and fundamental. I'd say it was a Qt bug, but that wouldn't explain why more people aren't having the same problem. With the simpler project, its even crashing in DEBUG



  • UPDATE2:

    I have found two more workarounds for this problem:

    QString temp("blah");
    std::string temp_str(std::string(temp.toLatin1().data()));
    

    and

    QString temp("blah");
    std::string temp_str = temp.toLatin1();
    

    Both of these result in valid strings which do not crash. I have now tried my original problem on several different systems at two different facilities with the same result. Either there is something lacking in the documentation for the MSVC10 build of Qt or its an outright bug. Why more people haven't reported this still baffles me.

    I am going to label this as SOLVED, though its a bit of a stretch. Thanks to all for your help!



  • @DRoscoe said:

    Why more people haven't reported this still baffles me

    Personally, I haven't reported it because it has always worked perfectly, in Qt4, all Qt5 versions and with all MSVC versions I've used. I suspect it's the same for most people.

    It is also tested in the Qt5 test suite, so I doubt Qt would be released with this failing.

    Sorry, there is something with your configuration that you haven't told us (unintentionally, for sure) and we haven't guess.

    Out of curiosity, what version of Qt and Windows ? (you left that out, we only know it's Qt5). And what configure command do you use to compile it.


  • Moderators

    Thanks for posting your updates, @DRoscoe.

    Either there is something lacking in the documentation for the MSVC10 build of Qt or its an outright bug. Why more people haven't reported this still baffles me.

    It definitely doesn't belong in the documentation, as the crash should not happen. Like @sandy.martel23, I believe it hasn't been reported much because very few people are affected by it -- I tried to reproduce it on my system but couldn't (not that I have MSVC 2010 anymore).

    Anyway, I'm glad you can get on with your project.



  • @JKSH It can't just be a local problem either. Several other developers at my location are having the same problem. I created a very simple Win32 console app that reproduces the problem, eliminating all of the third-party libraries we use. I sent the project to another group down in Virginia and all who tried it had the same problem (our group is standardized on MSVC 2010, but we are moving to 2015 when available). What I failed to realized was that the basic console app also crashed in DEBUG, whereas our main project crashes only in RELEASE. I now believe that its a problem with MSVC 2010 only, as my MSVC 2013 project doesn't have the problem. I think the problem occurred when Qt went to UTF8 which would explain why toLatin1() and toLocal8Bit() work.



  • LAST UPDATE

    The problem is solved! Prior to installing Qt 5.4, I had been running Qt 4.8.6. This was installed with a binary installer package (not web installer). When I installed Qt 5.4, I used the web installer. It ended up installing several earlier versions of Qt as well. There were also some VC redistributables in the Qt folder. I gutted the whole thing and re-installed only Qt 5.4 clean. I installed no other versions. I rebuilt my project and ran the test again, and voila! The problem disappeared.

    The one thing in common in my case, our other internal developers and the group in Virginia is that we all started fro Qt 4.8.6 using the same installer package and upgraded via web-installer.



  • A configuration issue ? I can't say that I'm shocked...

    I have several Qt versions installed on Windows, version 4, version 5, for 64 & 32 bits and for different VS (2010 and 2013). I would suspect those VC redistributables, if they were ot in sync with the compiler used.

    On the other hand, I only use Qts that I compile myself, so I keep a close control on access paths and get a "clean" dev install for each, no upgrades, maybe the (web) installer is mixing up versions.

    Glad you found it.



  • @sandy.martel23 When you build your Qt sources for VS2010 can you tell me how you do it? When I did it, I ended up having the same problem. Do you specify any special compiler settings?



  • OK, this time, I have nailed down the specific line in our projects that are causing the problem:

    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /D \"_ITERATOR_DEBUG_LEVEL=1\"")

    Turning on STL iterator debugging is not compatible with Qt libraries. I suspect that turning this feature on when compiling Qt would solve the problem, but for now it's easier to turn it off in my own projects. It is only marginally useful.



  • 9 days ago, @sandy.martel23 said:

    But it seems that your still mixing debug and release STL objects

    told you :-)

    As for building Qt, I do nothing special.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.