std::string destructor crashing in Release when using ::toStdString methods

cybercatalyst

std::string temp_str = temp.toStdString();

Can you try what happens with:

std::string temp_str(temp.toStdString());

?

DRoscoe

That was how the code originally was written, when I first observed the problem. I though perhaps I could solve it by creating a non-temporary copy, but it didn't work.

DependencyWalker did not show me anything obvious. At least I didn't see any reference to debug DLL's. I'm in the process of recompiling the Qt Libraries on my compiler. I am using Visual Studio 2010

mcosta

Hi,

could you post the code inside your function?

DRoscoe

The example I originally posted is already sufficient to reproduce the problem in my environment, but here is the exact code that is causing the problem:

void CommunicationHandler::decodeMessage(const QByteArray& msg_in)
{
  //DAR@20150416 Because the NAV Database contains a large binary payload, we
  //             need a way to omit this portion of the message for logging
  if (cur_msg_type == TapMessageClass::NAV_DATABASE)
  {
    // The NAV Database is an AOP formatted binary object which begins with the
    // letters "aop" and a null terminator.
    QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));
    TapMessagingLogger::instance()->logMessage(TapMessagingLogger::RECEIVED,
                                               to_log.toStdString());
  }

When the call to logMessage goes out of scope, it crashes. If I modify the code as follows:

void CommunicationHandler::decodeMessage(const QByteArray& msg_in)
{
  //DAR@20150416 Because the NAV Database contains a large binary payload, we
  //             need a way to omit this portion of the message for logging
  if (cur_msg_type == TapMessageClass::NAV_DATABASE)
  {
    // The NAV Database is an AOP formatted binary object which begins with the
    // letters "aop" and a null terminator.
    QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));
    std::string temp_str = to_log.toStdString();
    TapMessagingLogger::instance()->logMessage(TapMessagingLogger::RECEIVED, temp_str);
  }

The crash occurs when the if() clause goes out of scope. It is important to note that this code causes the crash only because its the first time this type of operation is performed. If I prevent it from executing, the problem occurs at a different point using a similar operation.

The bottom line is, in Release mode, whenever a std::string is produced using ::toStdString, the std::string destructor will cause a crash. I added a code snippet like the one in my original post in the constructor of my main application class and it crashed immediately

mcosta

Hi,

if I understood correctly you're trying to log something that is in binary format; IIRC toStdString() converts Unicode (QString) into a std::string using utf-8 encoding.

Could you try to use directly QByteArray::toStdString() instead of passing through a QString? In that case you can avoid encoding step

DRoscoe

I am not trying to log the binary portion of the payload. The QByte array is an array of chars from a socket read. The front portion of the array is string data representing the header of the message. The following line:

QString to_log(msg_in.mid(0, (msg_in.indexOf("aop\0"))));

Isolates the string portion, starting at position zero, continuing until the first instance of "aop\0", which denotes the start of the binary payload. The resulting QString contains just the header, which is string data. It is this portion that is being logged The binary payload is handled farther downstream.

mcosta

Ok,

have you tried to Debug you application to understand what happens??

And, as I said before, you can use directly QByteArray::toStdString() do avoid double conversion ascii --> UNICODE --> ascii

DRoscoe

In my original post, I pointed out that this has been impossible to debug thus far, and believe me I tried for quite a few days before my first post. The call stack is unusable as the crash occurs deep within Qt and and Windows libraries for which I have nothing but disassembly code to look at. The call to_tidy() in the std::string is what's crashing indicating a stack/heap corruption, and the crash report supports this.

Here is another example of a crash that WAS occurring, which is much more straightforward:

CommunicationHandler::CommunicationHandler(QObject* owner_in)
: QObject(owner_in)
{
  QString cur_dir("../output/");

  dna::FilenameTimestamp start_timestamp;
  std::string filename = 
    start_timestamp.getTimestampedFilename(
                                    cur_dir.toStdString(),
                                    "TAPDisplayMessagingLog",
                                    ".xml");
  TapMessagingLogger::instance()->openLogFile(filename);
}

In the above case, the crash was occurring in the call to ::getTimestampedFilename() This was due to the destruction of the temporary std::string created from the call to cur_dir.toStdString() in the argument list

I modified the code to eliminate the temporary variable thinking something might be wrong there:

CommunicationHandler::CommunicationHandler(QObject* owner_in)
: QObject(owner_in)
{
  QString cur_dir("../output/");
  std::string dir_str = cur_dir.toStdString();

  dna::FilenameTimestamp start_timestamp;
  std::string filename =
    start_timestamp.getTimestampedFilename(dir_str ,
                                           "TAPDisplayMessagingLog",
                                           ".xml");
  TapMessagingLogger::instance()->openLogFile(filename);
}

In this case, the crash was occurring when the constructor went out of scope. Again, the crash was occurring in the _tidy() method of std::string while trying to destroy dir_str. I modified the code once again, eliminating the initial QString altogether:

CommunicationHandler::CommunicationHandler(QObject* owner_in)
: QObject(owner_in)
{
  std::string dir_str = "../output/";

  dna::FilenameTimestamp start_timestamp;
  std::string filename =
    start_timestamp.getTimestampedFilename(dir_str ,
                                           "TAPDisplayMessagingLog",
                                           ".xml");
  TapMessagingLogger::instance()->openLogFile(filename);
}

And the crash was resolved. This is not a code problem. There is some insidious problem with my compiled code and the precompiled binaries installed with the Qt Installer. I am in the process of trying to compile the Qt Libraries from source using my compiler to see if this resolves the problem

DRoscoe

Compiling the Qt library did not resolve the problem.

mcosta

Hi,

I created a sample program:

#include <QFileDialog>

#include <string>

Widget::Widget(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::Widget)
{
    ui->setupUi(this);
}

Widget::~Widget()
{
    delete ui;
}

void Widget::on_pushButton_clicked()
{
    std::string my_str = ui->lineEdit->text().toStdString();

    qDebug("Hello %s", my_str.c_str());
}

void Widget::on_lineEdit_textChanged(const QString &arg1)
{
    ui->pushButton->setDisabled(arg1.isEmpty());
}

void Widget::on_toolButton_clicked()
{
    QString fileName = QFileDialog::getOpenFileName(
                this, tr("Choose a file"), ".");

    if(!fileName.isEmpty())
        ui->lineEdit->setText(fileName);
}

In a form I use a QFileDialog to choose a file from the file-system and when a button is clicked I convert the fileName to a std::string and print it to the stdout.
I run this program without crash.

I'm on OS X with Qt 5.4.1.
In your function use the the conversion with toStdString() and comment the following lines; verify is the crash is still there

CommunicationHandler::CommunicationHandler(QObject* owner_in)
: QObject(owner_in)
{
    QString cur_dir("../output/");
    std::string dir_str = cur_dir.toStdString();

    // dna::FilenameTimestamp start_timestamp;
    // std::string filename =
        // start_timestamp.getTimestampedFilename(dir_str ,
        //                                   "TAPDisplayMessagingLog",
        //                                   ".xml");
//  TapMessagingLogger::instance()->openLogFile(filename);
}

DRoscoe

I posted almost the exact same pseudocode in my original post, and yes I have already tried this in code, as I alluded to in a follow-up post. On my system it crashes, which is a Windows system. I don't have this problem on the Mac. This is a problem (for me) on the Windows platform using MSVC2010.

sandy.martel23

Qt and your code are probably not compiled with the same STL version. Something like this:
http://stackoverflow.com/questions/14090341/difference-in-byte-size-of-stl-containers-in-visual-studio-2010-and-2012

DRoscoe

@sandy.martel23 said:

http://stackoverflow.com/questions/14090341/difference-in-byte-size-of-stl-containers-in-visual-studio-2010-and-2012

I recompiled the Qt Library from source using the same compiler and that did not solve the problem. I am trying different C++11 options now to see if that's implicated

DRoscoe

I think I might be on the right track to figuring out what's going on. My project builds three executables. I recreated the same code that causes a crash in the problem executable in another of the remaining executables and the crash is not reproducible there. This leads me to suspect there is an alignment issue or something else that is affecting ONLY this one executable.

DRoscoe

My previous assertion that only one executable had the problem was false. I quickly mocked up the following in the main method of all three executables and got similar results:

using std::string;

int main(int argc, char *argv[])
{
  {
    QString temp("blah");
    string temp_str = temp.toStdString();
    std::cout << temp_str << std::endl;
    std::cout << "blah2" << std::endl;
    string temp_str2 = temp_str;
    temp_str2.append("a");
    std::cout << temp_str2 << std::endl;
  }

The output:

£♫  â─♦â° un §$
blah2
£♫  â─♦â° un §$a

As you can see, the string is already junk when it's streamed to std::cout but looks fine in the watch list in the debugger. The string literal outputs fine. The copying of temp_str to temp_str2 does not crash, nor does the append call, as evidenced by the last line. When the code goes out of scope, the crash occurs as normal. This suggests that even the original call to temp.toStdString() resulted in what looked like a valid string, but the std::cout results and the copy output clearly shows that something is wrong even before we go out of scope.

alex_malyu

Do you have MSVC10 service packs installed?
I remember MSVC10 in 64 bit had a nasty bug which took microsoft about a year to fix.

DRoscoe

I'm running 10.0.40219.1 SP1Rel

I have many other large legacy projects which don't have any problems with STL in general. It's just the use of Qt's STL conversions that are presenting a problem.

alex_malyu

SP1 should have solved the problem I mentioned, so it is not it.

sandy.martel23

@DRoscoe said:

I recompiled the Qt Library from source using the same compiler and that did not solve the problem.

But it seems that your still mixing debug and release STL object.
sizeof( debug std::string ) != sizeof( release std::string ).

DRoscoe

DependencyWalker has confirmed I am not using Debug STL libraries. However, I will use the sizeof() test as you suggest. At this point, I'll try anything!