Memory leak when using QNetworkReply::finished signal

Captain Crutches

I have the following (seemingly) simple bit of code to download an image from a URL, where downloader is a QNetworkAccessManager that's a member of the class where this happens:

auto reply = downloader.get(QNetworkRequest(imgUrl));
connect(reply, &QNetworkReply::finished, [reply]
{
    // Eventually, do some stuff with the reply, e.g. read data from it...
    qDebug() << "Image fetched!";

    reply->deleteLater();
});

It works, the lambda is called, etc... but valgrind (run with --leak-check=full) reports a memory leak (and I do indeed see my program's footprint growing) and I'm having trouble figuring out why.

If I remove the connect call and just deleteLater() on the reply right away, there is no leak and valgrind outputs the following (benign, I assume) detections:

==28567== 320 bytes in 1 blocks are possibly lost in loss record 2,141 of 2,400
==28567==    at 0x4C2B777: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28567==    by 0x4011C61: allocate_dtv (in /lib64/ld-2.24.so)
==28567==    by 0x40125FD: _dl_allocate_tls (in /lib64/ld-2.24.so)
==28567==    by 0x7146C86: pthread_create@@GLIBC_2.2.5 (in /lib64/libpthread-2.24.so)
==28567==    by 0x60D9985: QThread::start(QThread::Priority) (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0xD3352ED: ??? (in /usr/lib64/libQt5XcbQpa.so.5.9.2)
==28567==    by 0xD33B09C: QXcbConnection::QXcbConnection(QXcbNativeInterface*, bool, unsigned int, char const*) (in /usr/lib64/libQt5XcbQpa.so.5.9.2)
==28567==    by 0xD33EEB8: QXcbIntegration::QXcbIntegration(QStringList const&, int&, char**) (in /usr/lib64/libQt5XcbQpa.so.5.9.2)
==28567==    by 0x40263F2: ??? (in /usr/lib64/qt5/plugins/platforms/libqxcb.so)
==28567==    by 0x5BF9D1A: QPlatformIntegrationFactory::create(QString const&, QStringList const&, int&, char**, QString const&) (in /usr/lib64/libQt5Gui.so.5.9.2)
==28567==    by 0x5C0A1FF: QGuiApplicationPrivate::createPlatformIntegration() (in /usr/lib64/libQt5Gui.so.5.9.2)
==28567==    by 0x5C0AE7C: QGuiApplicationPrivate::createEventDispatcher() (in /usr/lib64/libQt5Gui.so.5.9.2)
==28567==    by 0x62A299A: QCoreApplicationPrivate::init() (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0x5C0BF5A: QGuiApplicationPrivate::init() (in /usr/lib64/libQt5Gui.so.5.9.2)
==28567==    by 0x515B1A8: QApplicationPrivate::init() (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x4BA834: main (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567== 
==28567== 352 bytes in 1 blocks are possibly lost in loss record 2,159 of 2,400
==28567==    at 0x4C2B777: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28567==    by 0x4011C61: allocate_dtv (in /lib64/ld-2.24.so)
==28567==    by 0x40125FD: _dl_allocate_tls (in /lib64/ld-2.24.so)
==28567==    by 0x7146C86: pthread_create@@GLIBC_2.2.5 (in /lib64/libpthread-2.24.so)
==28567==    by 0x60D9985: QThread::start(QThread::Priority) (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0x4F0E784: QNetworkConfigurationManagerPrivate::initialize() (in /usr/lib64/libQt5Network.so.5.9.2)
==28567==    by 0x4F089BC: qNetworkConfigurationManagerPrivate() (in /usr/lib64/libQt5Network.so.5.9.2)
==28567==    by 0x4F08A4A: QNetworkConfigurationManager::QNetworkConfigurationManager(QObject*) (in /usr/lib64/libQt5Network.so.5.9.2)
==28567==    by 0x4E9F583: QNetworkAccessManager::QNetworkAccessManager(QObject*) (in /usr/lib64/libQt5Network.so.5.9.2)
==28567==    by 0x4EE210: ChatDocument::ChatDocument(QWidget*) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x50958A: ChatWidget::ChatWidget(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, QWidget*) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x512A72: MainWindow::loggedIn(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x513D93: QtPrivate::FunctorCall<QtPrivate::IndexesList<0>, QtPrivate::List<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, void, void (MainWindow::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::call(void (MainWindow::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), MainWindow*, void**) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x513C60: void QtPrivate::FunctionPointer<void (MainWindow::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::call<QtPrivate::List<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, void>(void (MainWindow::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), MainWindow*, void**) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x513B85: QtPrivate::QSlotObject<void (MainWindow::*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), QtPrivate::List<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x62C5F3C: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0x527DB3: LoginWidget::loginSuccess(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x511FC2: LoginWidget::loginSubmit() (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x5127F0: QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (LoginWidget::*)()>::call(void (LoginWidget::*)(), LoginWidget*, void**) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x512773: void QtPrivate::FunctionPointer<void (LoginWidget::*)()>::call<QtPrivate::List<>, void>(void (LoginWidget::*)(), LoginWidget*, void**) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x5126DD: QtPrivate::QSlotObject<void (LoginWidget::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (in /home/captaincrutches/Projects/anekichat-qt/anekichat-qt)
==28567==    by 0x62C5F3C: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0x52B1454: ??? (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x62C5DC2: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib64/libQt5Core.so.5.9.2)
==28567==    by 0x52B8C2C: QWidgetLineControl::processKeyEvent(QKeyEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x52AB009: QLineEdit::keyPressEvent(QKeyEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x5198F3B: QWidget::event(QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x52B0D29: QLineEdit::event(QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x5154BB3: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28567==    by 0x515E7CA: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)

With the signal connected, I now have some blocks definitely/indirectly lost:

==28668== 222 bytes in 13 blocks are definitely lost in loss record 2,063 of 2,431
==28668==    at 0x4C2DF41: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28668==    by 0x79792B7: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7931D73: ASN1_STRING_set (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x79276F2: asn1_ex_c2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x792796D: ??? (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928689: ASN1_item_ex_d2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928B41: ??? (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928DB7: ??? (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x792840B: ASN1_item_ex_d2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928A8B: ??? (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928DB7: ??? (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x792826F: ASN1_item_ex_d2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7928F70: ASN1_item_d2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7948564: X509V3_EXT_d2i (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x4F7AF84: QSslCertificate::subjectAlternativeNames() const (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F71F42: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F8923C: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F897A9: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F73030: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x62C5DC2: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x4F3BFA2: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F3E711: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F4F000: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x5154BB3: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28668==    by 0x515CF9E: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28668==    by 0x629B704: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x62F232C: ??? (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x94DB8B6: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==    by 0x94DBAC7: ??? (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==    by 0x94DBB4B: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==
==28668== 1,212 (56 direct, 1,156 indirect) bytes in 1 blocks are definitely lost in loss record 2,314 of 2,431
==28668==    at 0x4C2DF41: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28668==    by 0x79792B7: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x78CC529: EC_KEY_new (in /usr/lib64/libcrypto.so.1.0.0)
==28668==    by 0x7597F0B: ssl3_get_key_exchange (in /usr/lib64/libssl.so.1.0.0)
==28668==    by 0x7598F78: ssl3_connect (in /usr/lib64/libssl.so.1.0.0)
==28668==    by 0x75A48AB: ssl23_connect (in /usr/lib64/libssl.so.1.0.0)
==28668==    by 0x4F89014: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F897A9: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F73030: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x62C5DC2: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x4F3BFA2: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F3E711: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x4F4F000: ??? (in /usr/lib64/libQt5Network.so.5.9.2)
==28668==    by 0x5154BB3: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28668==    by 0x515CF9E: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.9.2)
==28668==    by 0x629B704: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x62F232C: ??? (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x94DB8B6: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==    by 0x94DBAC7: ??? (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==    by 0x94DBB4B: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.4800.2)
==28668==    by 0x62F199A: QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x629A32A: QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x60D5AE9: QThread::exec() (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x60DA4FE: ??? (in /usr/lib64/libQt5Core.so.5.9.2)
==28668==    by 0x7146173: start_thread (in /lib64/libpthread-2.24.so)
==28668==    by 0x6E90DBE: clone (in /lib64/libc-2.24.so)
==28668== 
==28668== LEAK SUMMARY:
==28668==    definitely lost: 278 bytes in 14 blocks
==28668==    indirectly lost: 1,156 bytes in 23 blocks

Everything I've read suggests that deleteLater in a situation like this should Just Work™ - any idea why it doesn't here? Am I missing some simple thing I need to do to fully delete the reply? Is my lambda somehow badly done?

I'd appreciate any input - suggestions, links, "wow you're dumb just do X" - and of course let me know if you need anything else. Thanks!

kenchan

@Captain-Crutches
Calling deleteLater() defers the deletes until the event loop gets around to doing it. I guess that shows up as a false positive with valgrind (sorry, never used it so I don't know for certain).
Did you try using a call to sendPostedEvents(Q_NULLPTR,QEvent::DeferredDelete) on your app object to try to force the delete event to happen when you want it to?

Captain Crutches

@kenchan

I just tried QCoreApplication::sendPostedEvents(Q_NULLPTR, QEvent::DeferredDelete); after deleteLater() in my lambda, with both Q_NULLPTR and my reply object, to no effect.

I don't think it's a false positive - like I said, I do see my program's memory footprint growing when this happens - and the delete event does occur, as I can pick up its destroyed signal at the right time. Somehow it just looks like the memory isn't getting freed properly.

I will admit I'm somewhat new to the nuances of C++ pointers and Qt pointers in particular, but this seems like odd behavior to me.

kenchan

@Captain-Crutches
Something I don't quite understand about the output you show.
In the output where you say you don't use connect to the functor, I see the functor being called. In the output where you say you use the connect I don't see the functor being called. I am reading this out put wrongly or is there something fishy going on?
Might not be relevant though :-(.

Captain Crutches

@kenchan

Ah, yeah, the first block of output is without the connect, and I believe those traces are from creating the QNetworkAccessManager (which lives in the ChatDocument class and is initialized in its constructor). The second block of output is with the connect, and those two traces are in addition to the ones in the first block (i.e. the traces in the first block show up in both cases, I omitted them in the second for brevity). I know "possibly lost" and "still reachable" are usually false positives in valgrind, so I'm not worried about those.

That's also part of what's confusing me here - the "definitely lost" things (the actual memory leaks) seem to be happening within Qt or even OpenSSL, and I can't figure out where or why.

kenchan

@Captain-Crutches
I see. My knowledge of how valgrind works with Qt are limited so I hope someone who knows more about that can help you.

Paul Colby

Hi @Captain-Crutches,

The Valgrind output definitely suggests (to me) a leak in OpenSSL, either due to a bug in OpenSSL, or Qt's use of it.

Specifically, it looks to me like you're experiencing CVE-2015-3195, which affect OpenSSL 1.0 (which you appear to have) versions prior to 1.0.0t (and other versions too).

From the OpenSSL changelog:

X509_ATTRIBUTE memory leak

When presented with a malformed X509_ATTRIBUTE structure OpenSSL will leak
memory. This structure is used by the PKCS#7 and CMS routines so any
application which reads PKCS#7 or CMS data from untrusted sources is
affected. SSL/TLS is not affected.

This issue was reported to OpenSSL by Adam Langley (Google/BoringSSL) using
libFuzzer.
(CVE-2015-3195)
[Stephen Henson]

Your OpenSSL implementation appears to be provided by the host OS (not the Qt project), so might be worth updating your OS packages.

Cheers.

Captain Crutches

@Paul-Colby

Good find - but I've got OpenSSL 1.0.2n, the latest in my distro (Funtoo), and that bug looks like it was fixed in 1.0.2e, so I should have that fix. I'll investigate if somehow I don't though...

Edit: Assuming this commit is the fix, yes, I do have it in the code that gets built here.

Paul Colby

@Captain-Crutches said in Memory leak when using QNetworkReply::finished signal:

Good find - but I've got OpenSSL 1.0.2n

You may be right, but the Valgrind output suggests otherwise, with its references to /usr/lib64/libcrypto.so.1.0.0

I could be misinterpreting that output. Its also possible that you have more than one version installed?

Captain Crutches

@Paul-Colby

No, it definitely looks like I have 1.0.2n. Why the version number on the .so file doesn't match I don't know, but...

# eix dev-libs/openssl
[I] dev-libs/openssl
     Available versions:  1.0.2n^d [M](~)1.1.0g-r2(0/1.1)^d {+asm bindist gmp kerberos rfc3779 sctp sslv2 +sslv3 static-libs test (+)tls-heartbeat vanilla zlib ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32" CPU_FLAGS_X86="sse2" ELIBC="musl"}
     Installed versions:  1.0.2n^d(01:31:08 PM 12/31/2017)(asm sslv3 tls-heartbeat zlib -bindist -gmp -kerberos -rfc3779 -sctp -sslv2 -static-libs -test -vanilla ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="64 -32 -x32" CPU_FLAGS_X86="sse2")
     Homepage:            http://www.openssl.org/
     Description:         full-strength general purpose cryptography library (including SSL and TLS)

# equery belongs /usr/lib64/libcrypto.so.1.0.0 
 * Searching for /usr/lib64/libcrypto.so.1.0.0 ... 
dev-libs/openssl-1.0.2n (/usr/lib64/libcrypto.so.1.0.0)

# strings /usr/lib64/libcrypto.so.1.0.0 | grep "^OpenSSL 1"
OpenSSL 1.0.2n  7 Dec 2017

Searched my filesystem for "libcrypto" and that's the only one that comes up outside of wine and steam. So I'm now pretty certain I've got a "good" version with respect to that fix... which means either that bug isn't what's really going on, or it's not actually fixed in 1.0.2n. (And I don't want to install 1.1 yet, it's masked because a lot of other stuff fails to build against it)