Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Executing QProcess in QThread: memory leak
Forum Updated to NodeBB v4.3 + New Features

Executing QProcess in QThread: memory leak

Scheduled Pinned Locked Moved Unsolved General and Desktop
79 Posts 7 Posters 19.3k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M mrjj
    15 Nov 2020, 17:13

    Hi
    How do you know there is a leak ?

    S Offline
    S Offline
    sitesv
    wrote on 15 Nov 2020, 17:21 last edited by
    #3

    @mrjj said in Executing QProcess in QThread: memory leak:

    Hi
    How do you know there is a leak ?

    By system monitor...
    A leak is about 4k/sec. I have been checking for a few minutes...

    M C 2 Replies Last reply 15 Nov 2020, 17:24
    0
    • S sitesv
      15 Nov 2020, 17:21

      @mrjj said in Executing QProcess in QThread: memory leak:

      Hi
      How do you know there is a leak ?

      By system monitor...
      A leak is about 4k/sec. I have been checking for a few minutes...

      M Offline
      M Offline
      mrjj
      Lifetime Qt Champion
      wrote on 15 Nov 2020, 17:24 last edited by
      #4

      @sitesv
      Ok. it might internally allocate something
      that is first freed on app termination.

      If you reuse the same QProcess
      (let it be a member of the TMyThread, allocate once)

      Does it still then still leak ?

      S 1 Reply Last reply 15 Nov 2020, 17:36
      0
      • M mrjj
        15 Nov 2020, 17:24

        @sitesv
        Ok. it might internally allocate something
        that is first freed on app termination.

        If you reuse the same QProcess
        (let it be a member of the TMyThread, allocate once)

        Does it still then still leak ?

        S Offline
        S Offline
        sitesv
        wrote on 15 Nov 2020, 17:36 last edited by
        #5

        @mrjj said in Executing QProcess in QThread: memory leak:

        Does it still then still leak ?

        Yes, there is still a leak.
        QProcess myProcess; moved into the class.

        S 1 Reply Last reply 15 Nov 2020, 17:38
        0
        • S sitesv
          15 Nov 2020, 17:36

          @mrjj said in Executing QProcess in QThread: memory leak:

          Does it still then still leak ?

          Yes, there is still a leak.
          QProcess myProcess; moved into the class.

          S Offline
          S Offline
          sitesv
          wrote on 15 Nov 2020, 17:38 last edited by
          #6

          @mrjj
          If I would use this code in while(1) of the 'main' function of the console app - there is NO memory leak.

          M 1 Reply Last reply 15 Nov 2020, 17:41
          0
          • S sitesv
            15 Nov 2020, 17:38

            @mrjj
            If I would use this code in while(1) of the 'main' function of the console app - there is NO memory leak.

            M Offline
            M Offline
            mrjj
            Lifetime Qt Champion
            wrote on 15 Nov 2020, 17:41 last edited by
            #7

            @sitesv
            but you only create the thread once , so it should be the same ?

            S 1 Reply Last reply 15 Nov 2020, 17:42
            0
            • M mrjj
              15 Nov 2020, 17:41

              @sitesv
              but you only create the thread once , so it should be the same ?

              S Offline
              S Offline
              sitesv
              wrote on 15 Nov 2020, 17:42 last edited by
              #8

              @mrjj said in Executing QProcess in QThread: memory leak:

              but you only create the thread once ,

              Yes.

              1 Reply Last reply
              0
              • M Offline
                M Offline
                mrjj
                Lifetime Qt Champion
                wrote on 15 Nov 2020, 17:47 last edited by
                #9

                Hi
                Hmm really strange then.
                Is this on windows ?
                I did something similar on linux and didnt notice it would leak 4K

                S 1 Reply Last reply 15 Nov 2020, 17:54
                0
                • S sitesv
                  15 Nov 2020, 17:21

                  @mrjj said in Executing QProcess in QThread: memory leak:

                  Hi
                  How do you know there is a leak ?

                  By system monitor...
                  A leak is about 4k/sec. I have been checking for a few minutes...

                  C Offline
                  C Offline
                  Christian Ehrlicher
                  Lifetime Qt Champion
                  wrote on 15 Nov 2020, 17:50 last edited by
                  #10

                  @sitesv said in Executing QProcess in QThread: memory leak:

                  By system monitor...

                  This is no valid memory leak checker. Use a correct to to find leaks like e.g. valgrind.

                  Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                  Visit the Qt Academy at https://academy.qt.io/catalog

                  1 Reply Last reply
                  2
                  • M mrjj
                    15 Nov 2020, 17:47

                    Hi
                    Hmm really strange then.
                    Is this on windows ?
                    I did something similar on linux and didnt notice it would leak 4K

                    S Offline
                    S Offline
                    sitesv
                    wrote on 15 Nov 2020, 17:54 last edited by
                    #11

                    @mrjj said in Executing QProcess in QThread: memory leak:

                    Is this on windows ?

                    Linux

                    @Christian-Ehrlicher
                    Is it correct, If a memory value is increasing for a few minutes?

                    C 1 Reply Last reply 15 Nov 2020, 18:03
                    0
                    • S sitesv
                      15 Nov 2020, 17:54

                      @mrjj said in Executing QProcess in QThread: memory leak:

                      Is this on windows ?

                      Linux

                      @Christian-Ehrlicher
                      Is it correct, If a memory value is increasing for a few minutes?

                      C Offline
                      C Offline
                      Christian Ehrlicher
                      Lifetime Qt Champion
                      wrote on 15 Nov 2020, 18:03 last edited by
                      #12

                      @sitesv A system monitor is not a valid tool for such an investigation. It may be a hint that there is something wrong but nothing more. The system monitor only shows what the OS gave the process, but the OS may not free resources immediately but only when it needs it later on so use a correct tool for this.

                      Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                      Visit the Qt Academy at https://academy.qt.io/catalog

                      S 1 Reply Last reply 15 Nov 2020, 18:08
                      1
                      • C Christian Ehrlicher
                        15 Nov 2020, 18:03

                        @sitesv A system monitor is not a valid tool for such an investigation. It may be a hint that there is something wrong but nothing more. The system monitor only shows what the OS gave the process, but the OS may not free resources immediately but only when it needs it later on so use a correct tool for this.

                        S Offline
                        S Offline
                        sitesv
                        wrote on 15 Nov 2020, 18:08 last edited by
                        #13

                        @Christian-Ehrlicher
                        Checked app by Valgrind. There is no memory leak...
                        Thank you. I hope there really are no errors.

                        M 1 Reply Last reply 15 Nov 2020, 18:14
                        2
                        • S sitesv
                          15 Nov 2020, 18:08

                          @Christian-Ehrlicher
                          Checked app by Valgrind. There is no memory leak...
                          Thank you. I hope there really are no errors.

                          M Offline
                          M Offline
                          mrjj
                          Lifetime Qt Champion
                          wrote on 15 Nov 2020, 18:14 last edited by
                          #14

                          @sitesv
                          I would leave it running for like 48 hours and see it would stabilize.
                          If its related to actually running the process on Os level, it might only allocate up to some max.

                          S 1 Reply Last reply 15 Nov 2020, 18:44
                          1
                          • C Offline
                            C Offline
                            Christian Ehrlicher
                            Lifetime Qt Champion
                            wrote on 15 Nov 2020, 18:16 last edited by
                            #15

                            Since Qt cleans up a lot of stuff during exit maybe also stop the application started with valgrind with CTRL+C and take a look at the reachable memory in the valgrind output to see if there is something interesting (start valgrind with '--show-reachable=yes')

                            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                            Visit the Qt Academy at https://academy.qt.io/catalog

                            1 Reply Last reply
                            3
                            • M mrjj
                              15 Nov 2020, 18:14

                              @sitesv
                              I would leave it running for like 48 hours and see it would stabilize.
                              If its related to actually running the process on Os level, it might only allocate up to some max.

                              S Offline
                              S Offline
                              sitesv
                              wrote on 15 Nov 2020, 18:44 last edited by
                              #16

                              @mrjj, @Christian-Ehrlicher
                              Thank you!

                              1 Reply Last reply
                              0
                              • S Offline
                                S Offline
                                sitesv
                                wrote on 16 Nov 2020, 14:40 last edited by sitesv
                                #17

                                Hi!
                                Made a simple test:

                                void mythread::run(){
                                    while(work){
                                        bool boolRESULT;
                                        QString output_str;
                                        QByteArray output;
                                        QStringList output_strlst;
                                        QTextCodec *codec;
                                        QString exe_path = "/bin/ping";;
                                        QStringList arguments;
                                        arguments << "127.0.0.1" << "-c" << "1";
                                
                                        if(myProcess == nullptr) myProcess = new QProcess;
                                        myProcess->start(exe_path, arguments);
                                        myProcess->waitForFinished(500);
                                
                                        output = myProcess->readAll();
                                        output_str = codec->toUnicode(output);
                                        output_strlst = output_str.split("\r\n");
                                
                                        myProcess->close();
                                         
                                        boolRESULT = false;
                                        for (int i = 0; i < output_strlst.count(); i++){
                                            boolRESULT = output_strlst[i].contains("ttl",Qt::CaseInsensitive);
                                            if (boolRESULT) break;
                                        }
                                        emit setStatus(boolRESULT);
                                   
                                        msleep(1000);
                                    }
                                }
                                

                                Valgrind output:
                                ...
                                ==23633== 2,400 bytes in 15 blocks are still reachable in loss record 2,049 of 2,114
                                ==23633== at 0x4C2BBEF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
                                ==23633== by 0x6267251: resizeSignalVector (qobject_p.h:297)
                                ==23633== by 0x6267251: QObjectPrivate::addConnection(int, QObjectPrivate::Connection*) (qobject.cpp:330)
                                ==23633== by 0x6268A18: QMetaObjectPrivate::connect(QObject const*, int, QMetaObject const*, QObject const*, int, QMetaObject const*, int, int*) (qobject.cpp:3453)
                                ==23633== by 0x626FD58: QObject::connect(QObject const*, char const*, QObject const*, char const*, Qt::ConnectionType) (qobject.cpp:2911)
                                ==23633== by 0x61BFB1F: QProcessPrivate::startProcess() (qprocess_unix.cpp:384)
                                ==23633== by 0x61BA34D: QProcessPrivate::start(QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2246)
                                ==23633== by 0x61BA582: QProcess::start(QString const&, QStringList const&, QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2094)
                                ==23633== by 0x10BBBF: mythread::run() (mythread.cpp:19)

                                ==23633== by 0x605A3B4: QThreadPrivate::start(void*) (qthread_unix.cpp:342)
                                ==23633== by 0x6A164A3: start_thread (pthread_create.c:456)
                                ==23633== by 0x75B1D0E: clone (clone.S:97)
                                ...

                                mythread.cpp 19 line is:

                                myProcess->start(exe_path, arguments);
                                

                                My log by system monitor:

                                • 16:50 4300 KB
                                • 17:06 4790 KB
                                • 17:20 5200 KB
                                • 17:30 5552 KB

                                Any ideas?

                                K J 2 Replies Last reply 16 Nov 2020, 14:51
                                0
                                • S sitesv
                                  16 Nov 2020, 14:40

                                  Hi!
                                  Made a simple test:

                                  void mythread::run(){
                                      while(work){
                                          bool boolRESULT;
                                          QString output_str;
                                          QByteArray output;
                                          QStringList output_strlst;
                                          QTextCodec *codec;
                                          QString exe_path = "/bin/ping";;
                                          QStringList arguments;
                                          arguments << "127.0.0.1" << "-c" << "1";
                                  
                                          if(myProcess == nullptr) myProcess = new QProcess;
                                          myProcess->start(exe_path, arguments);
                                          myProcess->waitForFinished(500);
                                  
                                          output = myProcess->readAll();
                                          output_str = codec->toUnicode(output);
                                          output_strlst = output_str.split("\r\n");
                                  
                                          myProcess->close();
                                           
                                          boolRESULT = false;
                                          for (int i = 0; i < output_strlst.count(); i++){
                                              boolRESULT = output_strlst[i].contains("ttl",Qt::CaseInsensitive);
                                              if (boolRESULT) break;
                                          }
                                          emit setStatus(boolRESULT);
                                     
                                          msleep(1000);
                                      }
                                  }
                                  

                                  Valgrind output:
                                  ...
                                  ==23633== 2,400 bytes in 15 blocks are still reachable in loss record 2,049 of 2,114
                                  ==23633== at 0x4C2BBEF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
                                  ==23633== by 0x6267251: resizeSignalVector (qobject_p.h:297)
                                  ==23633== by 0x6267251: QObjectPrivate::addConnection(int, QObjectPrivate::Connection*) (qobject.cpp:330)
                                  ==23633== by 0x6268A18: QMetaObjectPrivate::connect(QObject const*, int, QMetaObject const*, QObject const*, int, QMetaObject const*, int, int*) (qobject.cpp:3453)
                                  ==23633== by 0x626FD58: QObject::connect(QObject const*, char const*, QObject const*, char const*, Qt::ConnectionType) (qobject.cpp:2911)
                                  ==23633== by 0x61BFB1F: QProcessPrivate::startProcess() (qprocess_unix.cpp:384)
                                  ==23633== by 0x61BA34D: QProcessPrivate::start(QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2246)
                                  ==23633== by 0x61BA582: QProcess::start(QString const&, QStringList const&, QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2094)
                                  ==23633== by 0x10BBBF: mythread::run() (mythread.cpp:19)

                                  ==23633== by 0x605A3B4: QThreadPrivate::start(void*) (qthread_unix.cpp:342)
                                  ==23633== by 0x6A164A3: start_thread (pthread_create.c:456)
                                  ==23633== by 0x75B1D0E: clone (clone.S:97)
                                  ...

                                  mythread.cpp 19 line is:

                                  myProcess->start(exe_path, arguments);
                                  

                                  My log by system monitor:

                                  • 16:50 4300 KB
                                  • 17:06 4790 KB
                                  • 17:20 5200 KB
                                  • 17:30 5552 KB

                                  Any ideas?

                                  K Offline
                                  K Offline
                                  KroMignon
                                  wrote on 16 Nov 2020, 14:51 last edited by KroMignon
                                  #18

                                  @sitesv said in Executing QProcess in QThread: memory leak:

                                  Made a simple test:

                                  You are aware that doing so (subclassing QThread and redefining slot run()), there will be not QEventLoop to handle signals/slots for all QObjects running in this thread?
                                  For example QTimer::singleShot() will not work.

                                  It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)

                                  1 Reply Last reply
                                  1
                                  • S sitesv
                                    16 Nov 2020, 14:40

                                    Hi!
                                    Made a simple test:

                                    void mythread::run(){
                                        while(work){
                                            bool boolRESULT;
                                            QString output_str;
                                            QByteArray output;
                                            QStringList output_strlst;
                                            QTextCodec *codec;
                                            QString exe_path = "/bin/ping";;
                                            QStringList arguments;
                                            arguments << "127.0.0.1" << "-c" << "1";
                                    
                                            if(myProcess == nullptr) myProcess = new QProcess;
                                            myProcess->start(exe_path, arguments);
                                            myProcess->waitForFinished(500);
                                    
                                            output = myProcess->readAll();
                                            output_str = codec->toUnicode(output);
                                            output_strlst = output_str.split("\r\n");
                                    
                                            myProcess->close();
                                             
                                            boolRESULT = false;
                                            for (int i = 0; i < output_strlst.count(); i++){
                                                boolRESULT = output_strlst[i].contains("ttl",Qt::CaseInsensitive);
                                                if (boolRESULT) break;
                                            }
                                            emit setStatus(boolRESULT);
                                       
                                            msleep(1000);
                                        }
                                    }
                                    

                                    Valgrind output:
                                    ...
                                    ==23633== 2,400 bytes in 15 blocks are still reachable in loss record 2,049 of 2,114
                                    ==23633== at 0x4C2BBEF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
                                    ==23633== by 0x6267251: resizeSignalVector (qobject_p.h:297)
                                    ==23633== by 0x6267251: QObjectPrivate::addConnection(int, QObjectPrivate::Connection*) (qobject.cpp:330)
                                    ==23633== by 0x6268A18: QMetaObjectPrivate::connect(QObject const*, int, QMetaObject const*, QObject const*, int, QMetaObject const*, int, int*) (qobject.cpp:3453)
                                    ==23633== by 0x626FD58: QObject::connect(QObject const*, char const*, QObject const*, char const*, Qt::ConnectionType) (qobject.cpp:2911)
                                    ==23633== by 0x61BFB1F: QProcessPrivate::startProcess() (qprocess_unix.cpp:384)
                                    ==23633== by 0x61BA34D: QProcessPrivate::start(QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2246)
                                    ==23633== by 0x61BA582: QProcess::start(QString const&, QStringList const&, QFlagsQIODevice::OpenModeFlag) (qprocess.cpp:2094)
                                    ==23633== by 0x10BBBF: mythread::run() (mythread.cpp:19)

                                    ==23633== by 0x605A3B4: QThreadPrivate::start(void*) (qthread_unix.cpp:342)
                                    ==23633== by 0x6A164A3: start_thread (pthread_create.c:456)
                                    ==23633== by 0x75B1D0E: clone (clone.S:97)
                                    ...

                                    mythread.cpp 19 line is:

                                    myProcess->start(exe_path, arguments);
                                    

                                    My log by system monitor:

                                    • 16:50 4300 KB
                                    • 17:06 4790 KB
                                    • 17:20 5200 KB
                                    • 17:30 5552 KB

                                    Any ideas?

                                    J Offline
                                    J Offline
                                    JonB
                                    wrote on 16 Nov 2020, 15:06 last edited by JonB
                                    #19

                                    @sitesv
                                    In addition, from @KroMignon no QEventLoop also means no deleting/disposing/freeing of Qt objects.

                                    On a side note: nothing to do with your findings, but for the record

                                     arguments << "127.0.0.1" << "-c 1";
                                    

                                    is wrong: -c & 1 are separate arguments, it ought to be << "-c" << "1". You are lucky it works as one argument.

                                    S 1 Reply Last reply 16 Nov 2020, 16:14
                                    1
                                    • S Offline
                                      S Offline
                                      sitesv
                                      wrote on 16 Nov 2020, 15:30 last edited by
                                      #20

                                      @KroMignon @JonB
                                      Are there any methods to do this correctly?
                                      If I do this in the main thread: the app will freeze while QProcess execution - this is unacceptable for me.

                                      K 1 Reply Last reply 16 Nov 2020, 15:39
                                      0
                                      • S sitesv
                                        16 Nov 2020, 15:30

                                        @KroMignon @JonB
                                        Are there any methods to do this correctly?
                                        If I do this in the main thread: the app will freeze while QProcess execution - this is unacceptable for me.

                                        K Offline
                                        K Offline
                                        KroMignon
                                        wrote on 16 Nov 2020, 15:39 last edited by
                                        #21

                                        @sitesv said in Executing QProcess in QThread: memory leak:

                                        Are there any methods to do this correctly?

                                        I don't say this is incorrect.
                                        I depends what you want to achieve.
                                        If you don't need an event queue, it works. If you are using signals/slots, this will not work.

                                        I do not know what is the purpose of this code extract, so maybe it is the best way to do.

                                        It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)

                                        S 1 Reply Last reply 16 Nov 2020, 16:03
                                        0
                                        • K KroMignon
                                          16 Nov 2020, 15:39

                                          @sitesv said in Executing QProcess in QThread: memory leak:

                                          Are there any methods to do this correctly?

                                          I don't say this is incorrect.
                                          I depends what you want to achieve.
                                          If you don't need an event queue, it works. If you are using signals/slots, this will not work.

                                          I do not know what is the purpose of this code extract, so maybe it is the best way to do.

                                          S Offline
                                          S Offline
                                          sitesv
                                          wrote on 16 Nov 2020, 16:03 last edited by
                                          #22

                                          @KroMignon said in Executing QProcess in QThread: memory leak:

                                          If you don't need an event queue, it works. If you are using signals/slots, this will not work.

                                          I don't know, but from this qthread in my main app I emit signal... and it is emitted... slot is from the main thread. QThread starts by start() method.
                                          The problem for me is memory leak only.

                                          K 1 Reply Last reply 16 Nov 2020, 16:59
                                          0

                                          12/79

                                          15 Nov 2020, 18:03

                                          67 unread
                                          • Login

                                          • Login or register to search.
                                          12 out of 79
                                          • First post
                                            12/79
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved