Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Memory leak in QAbstractSocketPrivate::readFromSocket() ?
Forum Updated to NodeBB v4.3 + New Features

Memory leak in QAbstractSocketPrivate::readFromSocket() ?

Scheduled Pinned Locked Moved Solved General and Desktop
12 Posts 3 Posters 1.1k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A Alexey Volkov

    Hi!
    I'm trying to find out what is causing a memory leak in my QTcpServer app. RSS is growing ~100MB for each 10GB of traffic. Have tested with Heaptrack in my local test setup, and it shows quite concerning stats: almost half of leak is contributed by QAbstractSocketPrivate::readFromSocket():

    197.7kB (43.5%) leaked in QAbstractSocketPrivate::readFromSocket() in libQt5Network.so.5 and below.

    The other major contributor is QIODevice::readAll() :
    16.4kB (3.61%) leaked in QIODevice::readAll() in libQt5Core.so.5 and below.
    (in 3 places, ~10%)

    In all cases QRingBuffer::reserve() makes un-released allocation. By the time the trace was captured, all the sockets were closed. They should release all buffers but they do not.

    Is there any good practice to avoid excess allocations in readAll() or to make sockets close all internal buffers on close?

    Thanks is advance!
    e5771b15-3daa-4ae8-817f-83e86e607ebe-image.png

    Christian EhrlicherC Online
    Christian EhrlicherC Online
    Christian Ehrlicher
    Lifetime Qt Champion
    wrote on last edited by Christian Ehrlicher
    #2

    Please provide a minimal, compilable example of your problem and also what exact Qt version you're using on which OS. I doubt there is a leak inside QAbstractSocket in current Qt6 or latest Qt5 versions but it might be.

    Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
    Visit the Qt Academy at https://academy.qt.io/catalog

    A 1 Reply Last reply
    1
    • Christian EhrlicherC Christian Ehrlicher

      Please provide a minimal, compilable example of your problem and also what exact Qt version you're using on which OS. I doubt there is a leak inside QAbstractSocket in current Qt6 or latest Qt5 versions but it might be.

      A Offline
      A Offline
      Alexey Volkov
      wrote on last edited by
      #3

      @Christian-Ehrlicher
      It is Qt5.15.2 on Ubuntu 22.04
      It is quite tricky to provide compilable example. It is QTcpServer "by the book" but the incoming connections are handled in separate threads.

      Axel SpoerlA 1 Reply Last reply
      0
      • A Alexey Volkov

        @Christian-Ehrlicher
        It is Qt5.15.2 on Ubuntu 22.04
        It is quite tricky to provide compilable example. It is QTcpServer "by the book" but the incoming connections are handled in separate threads.

        Axel SpoerlA Offline
        Axel SpoerlA Offline
        Axel Spoerl
        Moderators
        wrote on last edited by
        #4

        @Alexey-Volkov
        Qt 5.15.2 is not supported any more.

        It is quite tricky to provide compilable example.

        That makes me confident that the leak is not in Qt.
        Feel free to prove that I am wrong.

        Software Engineer
        The Qt Company, Oslo

        A 1 Reply Last reply
        0
        • Axel SpoerlA Axel Spoerl

          @Alexey-Volkov
          Qt 5.15.2 is not supported any more.

          It is quite tricky to provide compilable example.

          That makes me confident that the leak is not in Qt.
          Feel free to prove that I am wrong.

          A Offline
          A Offline
          Alexey Volkov
          wrote on last edited by Alexey Volkov
          #5

          @Axel-Spoerl
          You got the trace on a screenshot. It is only Qt code. That does not exclude mis-use, of course. That's why I'm here

          Christian EhrlicherC 1 Reply Last reply
          0
          • A Alexey Volkov

            @Axel-Spoerl
            You got the trace on a screenshot. It is only Qt code. That does not exclude mis-use, of course. That's why I'm here

            Christian EhrlicherC Online
            Christian EhrlicherC Online
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on last edited by
            #6

            @Alexey-Volkov said in Memory leak in QAbstractSocketPrivate::readFromSocket() ?:

            Here is the code:
            https://forge.bineon.team/onmydisk/onmydisk

            This is neither minimal nor simple so if you really want to have someone looking into your code you should provide a reproducer.

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            Axel SpoerlA 1 Reply Last reply
            2
            • Christian EhrlicherC Christian Ehrlicher

              @Alexey-Volkov said in Memory leak in QAbstractSocketPrivate::readFromSocket() ?:

              Here is the code:
              https://forge.bineon.team/onmydisk/onmydisk

              This is neither minimal nor simple so if you really want to have someone looking into your code you should provide a reproducer.

              Axel SpoerlA Offline
              Axel SpoerlA Offline
              Axel Spoerl
              Moderators
              wrote on last edited by
              #7

              @Alexey-Volkov
              The trace showing Qt code doesn’t mean anything.
              The expectation that we should analyse a whole application project, while the leak can’t be isolated in a minimal reproducer, makes me even more confident ;-)

              Software Engineer
              The Qt Company, Oslo

              1 Reply Last reply
              1
              • A Offline
                A Offline
                Alexey Volkov
                wrote on last edited by Alexey Volkov
                #8

                Sure, I'm not expecting anyone to analyze my code. And not trying to find a bug in some else's code, I'm trying to solve the problem.
                The leak trace from Heaptracker is quite interesting - it starts in main thread, goes through the main event loop, then reaches QAbstractSocketPrivate::readFromSocket() and ends up in QRingBuffer::reserve(). What on earth could lead to this, if the socket itself was not leaked? Could it be possible that it is "false positive" case, and the private object was not yet actually released?

                Axel SpoerlA 1 Reply Last reply
                0
                • A Alexey Volkov

                  Sure, I'm not expecting anyone to analyze my code. And not trying to find a bug in some else's code, I'm trying to solve the problem.
                  The leak trace from Heaptracker is quite interesting - it starts in main thread, goes through the main event loop, then reaches QAbstractSocketPrivate::readFromSocket() and ends up in QRingBuffer::reserve(). What on earth could lead to this, if the socket itself was not leaked? Could it be possible that it is "false positive" case, and the private object was not yet actually released?

                  Axel SpoerlA Offline
                  Axel SpoerlA Offline
                  Axel Spoerl
                  Moderators
                  wrote on last edited by
                  #9

                  @Alexey-Volkov
                  When an abstract socket is deleted or goes out of scope, the private object is always released first (not only for QAbstractSocket, but for most Qt classes). A QObject::deleteLater can also appear like a memory leak, when the event loop in charge stops spinning to early. But that's all speculation and guesswork in the absence of a reproducer.

                  We're actually moving in circles: If you want to solve the problem, you have to spend time on it, dig to the bottom and isolate it into a minimal reproducer. If that's not possible, the bug is in your code. If you don't want to do it, the problem is obviously not serious enough.

                  Software Engineer
                  The Qt Company, Oslo

                  A 1 Reply Last reply
                  0
                  • Axel SpoerlA Axel Spoerl

                    @Alexey-Volkov
                    When an abstract socket is deleted or goes out of scope, the private object is always released first (not only for QAbstractSocket, but for most Qt classes). A QObject::deleteLater can also appear like a memory leak, when the event loop in charge stops spinning to early. But that's all speculation and guesswork in the absence of a reproducer.

                    We're actually moving in circles: If you want to solve the problem, you have to spend time on it, dig to the bottom and isolate it into a minimal reproducer. If that's not possible, the bug is in your code. If you don't want to do it, the problem is obviously not serious enough.

                    A Offline
                    A Offline
                    Alexey Volkov
                    wrote on last edited by
                    #10

                    @Axel-Spoerl
                    Looks like it was actually the case. The thread was finished too early and deleteLater() did not do the job. I reworked it with explicit delete. Now in combination with malloc_trim(0) during the idle time I see some memory usage drops in docker stats.

                    The better approach could be deleting the socket on some delay after disconnected() signal, and finish the thread on socket's destroyed() signal. Will try.

                    1 Reply Last reply
                    0
                    • A Offline
                      A Offline
                      Alexey Volkov
                      wrote on last edited by
                      #11

                      @Axel-Spoerl
                      Yes, the cause confirmed. This approach works! I see drops of RSS and docker memory statistics.
                      When the socket disconnects, first the socket should be deleted, then the thread can be stopped:

                      connect(socket, &QTcpSocket::disconnected,  socket, [this,  thread, socket]() {
                              QTimer::singleShot(CLEANUP_INTERVAL, socket, [socket, thread]() {
                                  delete socket;
                                  thread->quit();
                              });
                          });
                      
                          connect(thread, &QThread::finished, thread, &QThread::deleteLater);
                      

                      Thanks for advice!

                      Axel SpoerlA 1 Reply Last reply
                      1
                      • A Alexey Volkov

                        @Axel-Spoerl
                        Yes, the cause confirmed. This approach works! I see drops of RSS and docker memory statistics.
                        When the socket disconnects, first the socket should be deleted, then the thread can be stopped:

                        connect(socket, &QTcpSocket::disconnected,  socket, [this,  thread, socket]() {
                                QTimer::singleShot(CLEANUP_INTERVAL, socket, [socket, thread]() {
                                    delete socket;
                                    thread->quit();
                                });
                            });
                        
                            connect(thread, &QThread::finished, thread, &QThread::deleteLater);
                        

                        Thanks for advice!

                        Axel SpoerlA Offline
                        Axel SpoerlA Offline
                        Axel Spoerl
                        Moderators
                        wrote on last edited by
                        #12

                        @Alexey-Volkov thanks for letting us know what it was. Glad that it works. Please don’t forget to mark the issue solved.

                        Software Engineer
                        The Qt Company, Oslo

                        1 Reply Last reply
                        0
                        • A Alexey Volkov has marked this topic as solved on

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved