QSqlQuery hangs/crashes with QODBC driver - ODBC driver verified working with Python

jars121 · 5 Dec 2023, 23:56

Quick update to the above. I've done some further debugging and have found that the QSqlQuery itself is working as expected, it's the QSqlRecord which is causing the observed hang/memory leak.

If I change the while() loop to the below, the application executes correctly:

int i = 0;
while (testQuery.next()) {
    qDebug() << ++i;
}

The numbers 1 through 118 are printed as expected and the application works correctly. However, if I try and access the QSqlRecord data either by directly instantiating a QSqlRecord object or via the QSqlQuery object itself, the hang/memory leak and crash is observed:

while (testQuery.next()) {
    QSqlRecord rec = testQuery.record();
    qDebug() << rec.value(0);
    //qDebug() << testQuery.record().value(0);
}

jars121 · 6 Dec 2023, 05:17

Another quick update.

I've just installed SAP's odbc-cpp-wrapper and linked it into my application.

I've replicated the above simple example as follows:

#include <iostream>
#include <odbc/Connection.h>
#include <odbc/Environment.h>
#include <odbc/Exception.h>
#include <odbc/PreparedStatement.h>
#include <odbc/ResultSet.h>

void testFunction() {
    try {

        odbc::EnvironmentRef env = odbc::Environment::create();
        odbc::ConnectionRef conn = env->createConnection();
        conn->connect("Driver=Gupta SQLBase 12.31 (64-bit);Server=127.0.0.1;ServerName=SERVER1;Database=DB1;Port=2155;UID=USER1;PWD=TEST1");
        conn->setAutoCommit(false);

        odbc::PreparedStatementRef psSelect = conn->prepareStatement("SELECT * FROM TABLE1");
        odbc::ResultSetRef rs = psSelect->executeQuery();
        while (rs->next())
        {
            qDebug() << rs->getNString(1)->c_str();
        }
    }
    catch (const odbc::Exception &e) {
        qDebug() << e.what();
    }
}

The above correctly prints the 118 ID strings from the first column of TABLE1 as expected. As such, it looks like there is a bug in QSqlRecord.

Christian Ehrlicher · 6 Dec 2023, 06:33

Please provide a minimal, compilable example to reproduce the issue including the creation of the data in the code.
What database do you use?
Also you should try with QSqlQuery::setForwardOnly() and don't go through record() and print the single values with qDebug() to see where exactly it is hanging. A backtrace would also be nice.

jars121 · 6 Dec 2023, 06:50

@Christian-Ehrlicher Thank you for your prompt response Christian. I've just verified the issue on a brand new project; here is the .pro file:

QT = core sql

CONFIG += c++17 cmdline

# You can make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

SOURCES += \
        main.cpp

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

And the main.cpp file:

#include <QCoreApplication>

#include <QSqlDatabase>
#include <QSqlQuery>
#include <QSqlRecord>
#include <QSqlError>
#include <QDebug>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QSqlDatabase testDatabase = QSqlDatabase::addDatabase("QODBC");
    testDatabase.setDatabaseName("Driver=Gupta SQLBase 12.3 (64-bit);Server=127.0.0.1;ServerName=SERVER1;Database=DB1;Port=2155;UID=USER1;PWD=TEST1");

    if (!testDatabase.open()) {

        qDebug() << testDatabase.lastError();
    }

    QSqlQuery testQuery(testDatabase);
    testQuery.setForwardOnly(true);

    testQuery.prepare("SELECT * FROM TABLE1");
    testQuery.exec();

    while (testQuery.next()) {

        //Both of the below qDebug() statements cause the issue
        qDebug() << testQuery.value(0).toString();
        //qDebug() << testQuery.record().value(0);
    }

    return a.exec();
}

The database is SQLBase version 12.3. As shown above, I've tried using setForwardOnly() (I tried that earlier but didn't include it in my earlier posts) but the issue remains.

I put a break point on the qDebug() statement in the while() loop and created the following backtrace:

Thread 1 (Thread 32952.0x134c):
#0  0x00007ffc01911a10 in ntdll!RtlCompareMemoryUlong () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#1  0x00007ffc0189dd00 in ntdll!RtlAllocateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#2  0x00007ffc0189b44d in ntdll!RtlAllocateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#3  0x00007ffc019688d8 in ntdll!RtlRegisterSecureMemoryCacheCallback () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#4  0x00007ffc0189d255 in ntdll!RtlAllocateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#5  0x00007ffc0189b44d in ntdll!RtlAllocateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
No symbol table info available.
#6  0x00007ffc011a9d40 in msvcrt!malloc () from C:\WINDOWS\System32\msvcrt.dll
No symbol table info available.
#7  0x00007ffb302818c0 in QArrayData::allocate(QArrayData**, long long, long long, long long, QArrayData::AllocationOption) () from C:\Qt\6.6.0\6.6.0\mingw_64\bin\Qt6Core.dll
No symbol table info available.
#8  0x00007ffb30249303 in QString::QString(long long, Qt::Initialization) () from C:\Qt\6.6.0\6.6.0\mingw_64\bin\Qt6Core.dll
No symbol table info available.
#9  0x00007ffb3025e110 in QStringDecoder::decoderForHtml(QByteArrayView) () from C:\Qt\6.6.0\6.6.0\mingw_64\bin\Qt6Core.dll
No symbol table info available.
#10 0x00007ffb302447e1 in QString::fromUtf8(QByteArrayView) () from C:\Qt\6.6.0\6.6.0\mingw_64\bin\Qt6Core.dll
No symbol table info available.
#11 0x00007ffbdb51af49 in qt_plugin_instance () from C:\Qt\6.6.0\6.6.0\mingw_64\plugins\sqldrivers\qsqlodbc.dll
No symbol table info available.
#12 0x00007ffbdb51b285 in qt_plugin_instance () from C:\Qt\6.6.0\6.6.0\mingw_64\plugins\sqldrivers\qsqlodbc.dll
No symbol table info available.
#13 0x00007ffbdb51d8f2 in qt_plugin_instance () from C:\Qt\6.6.0\6.6.0\mingw_64\plugins\sqldrivers\qsqlodbc.dll
No symbol table info available.
#14 0x00007ffbdf26c0e8 in QSqlQuery::value(int) const () from C:\Qt\6.6.0\6.6.0\mingw_64\bin\Qt6Sql.dll
No symbol table info available.
#15 0x00007ff7bce917a8 in main (argc=1, argv=0x1f74d113c10) at ..\SQLBaseTest\main.cpp:30
        a = <incomplete type>
        testDatabase = {static defaultConnection = 0x7ffbdf2992ca "qt_sql_default_connection", d = 0x1f74d116100}
        __PRETTY_FUNCTION__ = "int main(int, char**)"
        testQuery = {d = 0x1f74d11c140}

Christian Ehrlicher · 6 Dec 2023, 07:07

As written in the bug report please also create the table and data in the code. I would guess there is garbage in the database.

jars121 · 6 Dec 2023, 07:14

The database is sourced from a third party; it's not created/controlled by myself or my code base.

I have verified the data in the database with both Python and the odbc-cpp-wrapper library without issue. I've also viewed the data using the SQLTalk Plus client software provided by SQLBase.

This particular database is used extensively elsewhere in our environment, this is just the first time interfacing with it using Qt.

If it will help with diagnosis I can create a new database and dummy table and populate it with dummy data for testing. Unfortunately access to the SQLBase driver requires registration (as far as I can recall) and installation and setup of the server isn't particularly intuitive.

Christian Ehrlicher · 6 Dec 2023, 08:33

You should extend your testcase to create and populate your table.

jars121 · 6 Dec 2023, 20:57

I'll endeavour to put together CREATE and INSERT code as well.

In the meantime I've just tested with Qt 5.15.10 (also MinGW with qmake on Windows 10) and it works perfectly. I haven't checked the release notes but it looks like perhaps something in the Qt5 -> Qt6 transition may be to blame.

ChrisW67 · 7 Dec 2023, 02:55

@jars121 Change "SELECT * ..." to SELECT column1, column2 ...", selecting only those you need. Querying by name is good practice for a number of reasons. Then, things I would try:

Rebuild any index on the table to eliminate corrupted indexes.
Try the query with a WHERE condition that bisects the data (using a key in some way) to isolate a trigger row if there is one. It looks like any row where TEST_DATE is in this year is suspect. That might be one row that you can already pinpoint, but it may also be a red herring.
Try the query one column at a time to isolate a trigger column if there is one. IMHO Columns most likely to break things would be dates/times or blobs.

jars121 · 7 Dec 2023, 04:25

@ChrisW67 I'll give this a go shortly, thanks for your input.

I do find it strange that the 'worst case' example code above (i.e. SELECT *) works perfectly fine on 5.15.10 though (as well as with Python and the odbc-cpp-wrapper library); if there is a trigger causing issues it seems to only trip up 6.6.0.

ChrisW67 · wrote on 7 Dec 2023, 04:25

@jars121 said in QSqlQuery hangs/crashes with QODBC driver - ODBC driver verified working with Python:

if there is a trigger causing issues it seems to only trip up 6.6.0.

You only know it trips up the Qt 6 version you have tested. Others may or may not work. Until the specific data that trips it up is isolated or the actual data is eliminated as the cause, little progress will happen on your QTBUG-119753. You have been asked for more information here and in the ticket for that reason.

Christian Ehrlicher · wrote on 7 Dec 2023, 05:18

@jars121 said in QSqlQuery hangs/crashes with QODBC driver - ODBC driver verified working with Python:

if there is a trigger causing issues it seems to only trip up 6.6.0.

When I'm correct it will also not work with the latest Qt5.15 (lts) and Qt6.5/6.2. If so then you have to blame the odbc driver since it does not return the correct string lengths. There was a fix for it to follow the odbc standard (and avoid wrong usage) but some ancient odbc drivers are not following the standard. So - please provide a fully working example.