Detect out-of-memory

MHermann

Hi all,

What is the best way to detect high memory consumption and detect out-of-memory error?
My application crashes "randomly" and now we assume that there is a problem with memory consumption.
Any tipps for me how I can detect the problem and ideally also the origin of the problem?

Regards

Christian Ehrlicher

Linux or Windows?
Linux: valgrind, asan
Windows: asan with MSVC 2019/22

MHermann

@Christian-Ehrlicher:
The application is running on Raspbian Buster.
ASan is already in use, but does not show helpful information after crash.

Christian Ehrlicher

Then run your program and start it after a while. There is also valgrind.
How do you think it's a memory problem?
Compile your app with debug symbols and run it in a debugger.

J.Hilk

@MHermann the exception std::bad_alloc should be thrown

https://en.cppreference.com/w/cpp/memory/new/bad_alloc

alternative with nothrow applied the pointer should be nullptr

Foo* foo = new (std::nothrow) Foo;

wrosecrans

@MHermann said in Detect out-of-memory:

now we assume that there is a problem with memory consumption.

Assumptions are always dangerous when debugging. What exactly is the crash that you get? Are you getting an error about something like bad_alloc? You you replicate the crash in a debugger, is it crashing in an allocation?

There is probably some other bug besides "out of memory" at play.

MHermann

The problem is that we can not replicate the error.
It happens in Release-Mode and than the application is crashing and then closing.
There is also no output from ASan in the logfile after the crash.

J.Hilk

@MHermann

It happens in Release-Mode and than the application is crashing and then closing

smells like uninitialised varibales/pointers

MHermann

@J-Hilk: Okay. Could be.
Unfortunately I am still facing the problem with the crashes.
Any tips how I can find the root cause for the crashes (maybe unitialised variables/pointers)?
I was thinking that there is maybe a problem with the garbage collection in javascript. But here I also don't know how I could locate the problem.

Christian Ehrlicher

@MHermann said in Detect out-of-memory:

Any tips how I can find the root cause for the crashes (maybe unitialised variables/pointers)?

Valgrind, ASAN or static code checker, remove all compiler warnings, ...

MHermann

Valgrind : Application gets too slow when using Valgrind. So that was not practicable.
ASAN: Is in use at the moment. ASAN gives following output, just before crash. But we were not able to find the problem...

AddressSanitizer:DEADLYSIGNAL
=================================================================
==1178==ERROR: AddressSanitizer: SEGV on unknown address 0x00000007 (pc 0xb578b3f8 bp 0xa8a3bd50 sp 0xbea93a80 T0)
==1178==The signal is caused by a READ memory access.
==1178==Hint: address points to the zero page.
#0 0xb578b3f7 in QV4::QQmlContextWrapper::getPropertyAndBase(QV4::QQmlContextWrapper const*, QV4::PropertyKey, QV4::Value const*, bool*, QV4::Value*, QV4::Lookup*) (/usr/local/qt5pi/lib/libQt5Qml.so.5+0x1693f7)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/usr/local/qt5pi/lib/libQt5Qml.so.5+0x1693f7) in QV4::QQmlContextWrapper::getPropertyAndBase(QV4::QQmlContextWrapper const*, QV4::PropertyKey, QV4::Value const*, bool*, QV4::Value*, QV4::Lookup*)
==1178==ABORTING

Static code checker: I have to read about this.
Remove all compiler warnings: Was already done.

J.Hilk

@MHermann said in Detect out-of-memory:

Remove all compiler warnings: Was already done.

what are your compiler settings ? Do you use -Weffc++and -Wuninitialized ?

seeing this:

AddressSanitizer:DEADLYSIGNAL

==1178==ERROR: AddressSanitizer: SEGV on unknown address 0x00000007 (pc 0xb578b3f8 bp 0xa8a3bd50 sp 0xbea93a80 T0)
==1178==The signal is caused by a READ memory access.
==1178==Hint: address points to the zero page.
#0 0xb578b3f7 in QV4::QQmlContextWrapper::getPropertyAndBase(QV4::QQmlContextWrapper const*, QV4::PropertyKey, QV4::Value const*, bool*, QV4::Value*, QV4::Lookup*) (/usr/local/qt5pi/lib/libQt5Qml.so.5+0x1693f7)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/usr/local/qt5pi/lib/libQt5Qml.so.5+0x1693f7) in QV4::QQmlContextWrapper::getPropertyAndBase(QV4::QQmlContextWrapper const*, QV4::PropertyKey, QV4::Value const*, bool*, QV4::Value*, QV4::Lookup*)
==1178==ABORTING

do you have a c++ class/instance that you expose to QML and that you think will "always" exists, but it has neither a valid cpp parent nor was the object ownership explicitly set to QQmlEngine::CppOwnership ? Singletons are a prime example here.

The QML engine may have taken ownership and the GC deleted it.

MHermann

@J-Hilk said in Detect out-of-memory:

Do you use -Weffc++and -Wuninitialized ?

No, I don't use at the moment.
But I will add these options and then see if I get more warnings.

MHermann

@J-Hilk said in Detect out-of-memory:

do you have a c++ class/instance that you expose to QML and that you think will "always" exists, but it has neither a valid cpp parent nor was the object ownership explicitly set to QQmlEngine::CppOwnership ?

Yes. I have C++ classes and I use some of their funtionalites in qml (Q_INVOKABLE). I don't use QQmlEngie::CppOwnership. To be honest I don't know in which situations I have to use it.

But I read the last days already about the "ownership-problem" and was thinking that this could be a possbile reason for the crashes.

Or what about the javascript garbage collector? I read that this could also lead to some problems.

MHermann

@MHermann said in Detect out-of-memory:

But I will add these options and then see if I get more warnings.

Compiling with QMAKE_CXXFLAGS += -Wuninitialized did not output new warnings.
But compiling with QMAKE_CXXFLAGS += -Weffc++ gave me a bunch of new warnings. But most of them are located in qt libs.

J.Hilk

@MHermann said in Detect out-of-memory:

But most of them are located in qt libs

yes, that's sadly not unusual :(

To be honest I don't know in which situations I have to use it

Potentially when you use Singletons and you want to manage those lifetimes yourself:
https://doc.qt.io/qt-6/qqmlengine.html#qmlRegisterSingletonType-1

or when you assign context Properties:
https://doc.qt.io/qt-6/qqmlcontext.html#setContextProperty //generally unlikely because one usually doesn't reload the root component/qmlengine

or when passing QObjects from cpp to qml, for example via a Q_INVOKABLE function

MHermann

@J-Hilk:

I have three singleton classes. Now I added in each constructor QQmlEngine::setObjectOwnership(this, QQmlEngine::CppOwnership);.
Is that correct?

I use setContextProperty. But you think that it is the least likely that this is the problem, right?

I also use Q_INVOKABLE. I checked the types of the return values. The types are QString, bool, QJsonObject and int.
There is no QObject. Can there still be the problem?

J.Hilk

@MHermann said in Detect out-of-memory:

I have three singleton classes. Now I added in each constructor QQmlEngine::setObjectOwnership(this, QQmlEngine::CppOwnership);.
Is that correct

yes and no, One would assume, that is correct, but I found, that this is actually too early. The QML-Engine has not jet the pointer to the instance received, therefore I do it like this:

static QObject *instance(QQmlEngine *engine, QJSEngine *)
    {
        if(DeviceInfo){
            QMetaObject::invokeMethod(DeviceInfo, [=]()->void{
                engine->setObjectOwnership(DeviceInfo, QQmlEngine::CppOwnership);
            }, Qt::QueuedConnection);
            return DeviceInfo;
        }

        DeviceInfo = new DeviceInformation(nullptr);
        QMetaObject::invokeMethod(DeviceInfo, [=]()->void{
            engine->setObjectOwnership(DeviceInfo, QQmlEngine::CppOwnership);
        }, Qt::QueuedConnection);
        return DeviceInfo;
    }

keep in mind, this is only necessary, if you access that singleton also inside your c++ code. if only your QML part access it, the memory management will be fine on its own.

. The types are QString, bool, QJsonObject and int.
There is no QObject. Can there still be the problem

there shouldn't