qRegisterMetaObject, SEGFAULT and QByteArray

webzoid

We've got a bit of a strange problem which as suddenly developed.

Take the following code:

int main(int argc, char *argv[])
{
    qRegisterMetaType<My::Namespace::Type>("My::Namespace::Type");
    // Other code here
}

When we come to run the application, we get a SEGFAULT instantly and I'm taken to the following location in the qarraydata.h file within the Qt source:

void *data()
{
    Q_ASSERT(size == 0
	    || offset < 0 || size_t(offset) >= sizeof(QArrayData));
    return reinterpret_cast<char *>(this) + offset;
}

Having evaluated each expression within the Q_ASSERT macro, the failure comes from size_t(offset) >= sizeof(QArrayData) which is evaluating to true.

For reference:
size == 23
offset == 16
sizeof(QArrayData) == 16
size_t(offset) == 16

My question is: why would this suddenly start occurring when it hasn't been an issue before?

Extra information:

Qt 5.11.3
Linux 4.19.38+
Wayland 1.16

SGaist

Hi,

Does it also happen if you have an empty app with only your custom type registered in main ?

kshegunov

@webzoid said in qRegisterMetaObject, SEGFAULT and QByteArray:

When we come to run the application, we get a SEGFAULT instantly and I'm taken to the following location in the qarraydata.h file within the Qt source:

Define instantly, please. Is it before or after it has entered main()? Give us a stack trace too, if you don't mind.

webzoid

@SGaist No, it doesn't happen with an empty app.

@kshegunov It was happening before main() was being entered.

We did several "clean" and "rebuild"s but this made no difference at all. However, deleting the build directory and rebuilding fixed the issue. We have no idea what went on but it was certainly strange.

Similar to the error initially reported, we have an application running on an embedded Linux device (Qt 5.11.3, Texas Instruments MPU) which is also debug asserting in the same way (from the void *data() call in QArrayData).

To describe this a bit more: we are reading from a QTcpSocket using the readyRead signal and the following code:

if (this->socket->bytesAvailable() > 0) {
        QMutexLocker(&this->mutex);
        this->readBuffer.append(this->socket->readAll());
}

On occasions, after periods of running (10, 15, 20 minutes), the application crashes with the below stack trace. Equally though, the application can run for well over an hour without any signs of issue.

Non-blocking swap buffers not supported. Subsurface rendering can be affected.
QSocketNotifier: Socket notifiers cannot be enabled or disabled from another thread
ASSERT: "size == 0 || offset < 0 || size_t(offset) >= sizeof(QArrayData)" in file /home/dev/ti-processor-sdk-linux-am57xx-evm-06.00.00.07/linux-devkit/sysroots/armv7at2hf-neon-linux-gnueabi/usr/include/QtCore/qarraydata.h, line 60

Thread 3 "ClientThread" received signal SIGABRT, Aborted.
[Switching to Thread 0xb20863f0 (LWP 1605)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) 
(gdb) 
(gdb) bt full
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {0, 0, 44, 3052270660, 0, 4, 3, 2986889698, 2986889696, 2986889698, 
            2986889928, 3057049169, 0, 6, 2, 4, 3058957492, 2986889944, 3058951824, 3057301469, 
            3058951904, 20, 3058951824, 0, 3, 3056997091, 2976907280, 0, 65, 3058952280, 
            2986890068, 0}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0xb5e86208 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0xffffffff, sa_sigaction = 0xffffffff}, 
          sa_mask = {__val = {2986893100, 3051835224, 3070156800, 2986889944, 2986889948, 
              2986890036, 0, 3053116992, 3053116672, 3060510892, 2986895536, 0, 1, 3204445142, 
              0, 3204445260, 2986893100, 3070088076, 3019817216, 1, 5, 0, 3, 3051835224, 3, 
              2986890036, 2986890048, 60, 2986890096, 3070112588, 2986890036, 0}}, 
          sa_flags = 12644, sa_restorer = 0x0}
        sigs = {__val = {32, 0 <repeats 31 times>}}
#2  0xb6350fda in QMessageLogger::fatal(char const*, ...) const () from /usr/lib/libQt5Core.so.5
No symbol table info available.
#3  0xb6350d64 in qt_assert(char const*, char const*, int) () from /usr/lib/libQt5Core.so.5
No symbol table info available.
#4  0x0001742e in QArrayData::data (this=0x4a94b8)
    at /home/dev/ti-processor-sdk-linux-am57xx-evm-06.00.00.07/linux-devkit/sysroots/armv7at2hf-neon-linux-gnueabi/usr/include/QtCore/qarraydata.h:59
No locals.
#5  0x00017a82 in QTypedArrayData<char>::data (this=0x4a94b8)
    at /home/dev/ti-processor-sdk-linux-am57xx-evm-06.00.00.07/linux-devkit/sysroots/armv7at2hf-neon-linux-gnueabi/usr/include/QtCore/qarraydata.h:206
No locals.
#6  0x0001765a in QByteArray::data (this=0x50c5c)
--Type <RET> for more, q to quit, c to continue without paging--
    at /home/dev/ti-processor-sdk-linux-am57xx-evm-06.00.00.07/linux-devkit/sysroots/armv7at2hf-neon-linux-gnueabi/usr/include/QtCore/qbytearray.h:489
No locals.
#7  0x00016d0e in ClientThread::checkForMessage (this=0x50c50) at src/Client/ClientThread.cpp:109
        header = 0xb2085dc8
        headerLength = 16
        __PRETTY_FUNCTION__ = "void ClientThread::checkForMessage()"
#8  0x00016c04 in ClientThread::run (this=0x50c50) at src/Client/ClientThread.cpp:52
No locals.
#9  0xb636d486 in ?? () from /usr/lib/libQt5Core.so.5
No symbol table info available.
#10 0xb61ce390 in start_thread (arg=0xb20863f0) at pthread_create.c:486
        ret = <optimized out>
        start = <optimized out>
        pd = 0xb20863f0
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {1902834589, 1971211977, -1308072976, 
                -1090522036, -1308072976, 338, -1090522154, 0, -1090522036, -1308074196, 
                0 <repeats 54 times>}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, 
            data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        robust = <optimized out>
#11 0xb5f3f918 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/libc.so.6
No locals.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

And yes, the QTcpSocket is created in a different thread.

kshegunov

@webzoid said in qRegisterMetaObject, SEGFAULT and QByteArray:

It was happening before main() was being entered.

That suggests that you depend on statics initialization order, which is undefined. That'd be the first thing to check, even if it works now as it can appear at any time.

Similar to the error initially reported, we have an application running on an embedded Linux device (Qt 5.11.3, Texas Instruments MPU) which is also debug asserting in the same way (from the void *data() call in QArrayData).

This, along with the trace and the warning you get, smells like a race condition. Make sure that all the methods are called in the proper threads and data access is serialized, and that the ClientThread object outlives any calls to it's member functions!

The warning:

QSocketNotifier: Socket notifiers cannot be enabled or disabled from another thread

Makes me very suspicious of the socket init/moving to another thread. You should regard it as an error in the user code.

webzoid

@kshegunov Sorry - my bad, I should have added this additional information (I'm still waking up!).

The reason it was happening before main is because it was actually happening in qRegisterMetaObject calls from internal Qt GUI classes. As previously mentioned, when we cleared out the build directory and did a rebuild, the issue went away.

The QSocketNotifier warning is from a QUdpSocket also contained within the application which is being created on one thread then polled on another thread.

I believe this is aside from the main issue though as it is the QTcpSocket mechanism which is causing us the problem - the QUdpSocket seems to be working just fine (even though it should be regarded as an error as you've stated).

Some additional information about our application:

We create a QTcpSocket to read network data from a network device in a separate thread (using QMutexLocker for safety). Data comes in and we update a display. It's a pretty self-servicing application - as long as data keeps coming in, the data is validated and the display is updated. When the application crashes, there is no user interjection, no changes to the quantity of data coming in from the device, no interruptions from other aspects of the application, no delete or clean-up operations triggered (as far as we know). When the application is running at minutes 5, 10, 15, 90, etc, it is doing the same thing: reading and updating the display.

The ClientThread is currently alive and kicking at the time of the crash.

kshegunov

@webzoid said in qRegisterMetaObject, SEGFAULT and QByteArray:

The QSocketNotifier warning is from a QUdpSocket also contained within the application which is being created on one thread then polled on another thread.

Why would anyone poll an asynchronous object is beyond my ability to comprehend, but okay, it is what it is.

I believe this is aside from the main issue though as it is the QTcpSocket polling mechanism which is causing us the problem - the QUdpSocket seems to be working just fine (even though it should be regarded as an error as you've stated).

It may, but I'd not bet my life on it. Bear in mind that there's internal machinery with the socket notifiers, and you can't guarantee something isn't shared between the UDP and TCP socket objects.

We create a QTcpSocket to read network data from a network device in a separate thread (using QMutexLocker for safety).

QMutexLocker serializes access according to the mutex it locks. The mutex may protect one data field, many objects or none. Whatever it is, you must make sure that at all times you don't have concurrent write-write or write-read to the same memory address.

Data comes in and we update a display. It's a pretty self-servicing application - as long as data keeps coming in, the data is validated and the display is updated. When the application crashes, there is no user interjection, no changes to the quantity of data coming in from the device, no interruptions from other aspects of the application, no delete or clean-up operations triggered (as far as we know). When the application is running at minutes 5, 10, 15, 90, etc, it is doing the same thing: reading and updating the display.

If it is a race condition none of this matters, because the very definition of a race condition includes it not being deterministic. You may miss the crash by a nanosecond and the program can continue happily lugging along for ages. And yet you may overlap for the minutest fraction and it can crash after a month of work. People have died from such errors, quite literally, and these are one of the hardest ones to track too ...

The ClientThread is currently alive and kicking at the time of the crash.

Your trace begs to differ.

#7  0x00016d0e in ClientThread::checkForMessage (this=0x50c50) at src/Client/ClientThread.cpp:109

Indicates the crash is rooted at line 109 in ClientThread.cpp in the very class and in the very method that checks for a message. What does the full backtrace say? Where are the other threads at that point?

Also, would you humor me and add:

Q_ASSERT(socket->thread() == QThread::currentThread());

just before:

if (this->socket->bytesAvailable() > 0)

webzoid

Why would anyone poll an asynchronous object is beyond my ability to comprehend, but okay, it is what it is.

Apologies, I didn't mean "poll" in that sense - I should have edited the post and re-phrased that sentence. The readyRead signal is also being used on the QUdpSocket.

Also, would you humor me and add:

Q_ASSERT(socket->thread() == QThread::currentThread());

just before:

if (this->socket->bytesAvailable() > 0)

Yep, will add this to the code and report back later today.

Thanks for your comments.

Christian Ehrlicher

@webzoid said in qRegisterMetaObject, SEGFAULT and QByteArray:

The QSocketNotifier warning is from a QUdpSocket also contained within the application which is being created on one thread then polled on another thread.

You must create the QTcp/UdpSocket in the thread you're reading/writing from. Everything else may work until it doesn't. Sharing a QTcpSocket (even guarded by a mutex) is not supported!