Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Detecting QStack overflow
Forum Updated to NodeBB v4.3 + New Features

Detecting QStack overflow

Scheduled Pinned Locked Moved Solved General and Desktop
15 Posts 5 Posters 1.1k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A Offline
    A Offline
    Alvein
    wrote on 13 Jul 2020, 17:58 last edited by
    #1

    Hello,

    According to the parent class, QVector, there's a limit of 2GB but that's not a safe guess. So I'd like to detect when there's no more available memory for QStack::push().

    What's the right way of doing this?

    And if you ask...simulated recursive function. The "tree of calls" size cannot be estimated and can be HUGE. So I need a way of avoiding creating new "branches" and forcing backtrack.

    Thanks for your comments.

    1 Reply Last reply
    0
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 13 Jul 2020, 18:14 last edited by
      #2

      Hi,

      Based on the documentation, the worst case scenario is: (2^31-32)/sizeof(YourStoredClass).

      However, if you have that kind of limit consideration you might want to implement some known safe values that allows your application to recover in case of spike or stop gracefully if there's an overload.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      A 1 Reply Last reply 13 Jul 2020, 18:39
      1
      • S SGaist
        13 Jul 2020, 18:14

        Hi,

        Based on the documentation, the worst case scenario is: (2^31-32)/sizeof(YourStoredClass).

        However, if you have that kind of limit consideration you might want to implement some known safe values that allows your application to recover in case of spike or stop gracefully if there's an overload.

        A Offline
        A Offline
        Alvein
        wrote on 13 Jul 2020, 18:39 last edited by
        #3

        @SGaist said in Detecting QStack overflow:

        ...you might want to implement some known safe values that allows your application to recover...

        Such as?

        1 Reply Last reply
        0
        • S Offline
          S Offline
          SGaist
          Lifetime Qt Champion
          wrote on 13 Jul 2020, 18:41 last edited by
          #4

          It depends on what you store and what could happen that fills your queue faster than it can process.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          1 Reply Last reply
          0
          • C Offline
            C Offline
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on 13 Jul 2020, 18:43 last edited by Christian Ehrlicher
            #5

            @Alvein said in Detecting QStack overflow:

            Such as?

            You want to tell us that you've more than 8GB (*) of contiguous memory available and also use this?

            If you really want more, use std::vector or wait for Qt6

            (*): 2^31 * 4 bytes = 8589934592 ~ 8.5GB, assumption that you only want to store an integer in the vector.

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            1 Reply Last reply
            0
            • A Offline
              A Offline
              Alvein
              wrote on 13 Jul 2020, 19:13 last edited by
              #6

              Not sure if you people get my idea.

              It's not that I need more memory.

              It's that I want to detect when there's no more available memory for my QStack, to stop the recursive calls at that point.

              If QStack::push() returned a bool result, meaning allocation success or something alike, this would be easy.

              1 Reply Last reply
              0
              • C Offline
                C Offline
                Christian Ehrlicher
                Lifetime Qt Champion
                wrote on 13 Jul 2020, 19:16 last edited by
                #7

                @Alvein said in Detecting QStack overflow:

                It's that I want to detect when there's no more available memory for my QStack, to stop the recursive calls at that point.

                I would guess it's too late then but: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

                Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                Visit the Qt Academy at https://academy.qt.io/catalog

                A 2 Replies Last reply 13 Jul 2020, 20:46
                2
                • C Christian Ehrlicher
                  13 Jul 2020, 19:16

                  @Alvein said in Detecting QStack overflow:

                  It's that I want to detect when there's no more available memory for my QStack, to stop the recursive calls at that point.

                  I would guess it's too late then but: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

                  A Offline
                  A Offline
                  Alvein
                  wrote on 13 Jul 2020, 20:46 last edited by
                  #8

                  @Christian-Ehrlicher

                  OK. Will try that. Thanks. :)

                  1 Reply Last reply
                  0
                  • C Christian Ehrlicher
                    13 Jul 2020, 19:16

                    @Alvein said in Detecting QStack overflow:

                    It's that I want to detect when there's no more available memory for my QStack, to stop the recursive calls at that point.

                    I would guess it's too late then but: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

                    A Offline
                    A Offline
                    Alvein
                    wrote on 15 Jul 2020, 17:44 last edited by
                    #9

                    @Christian-Ehrlicher Hello. Tried std::bad_alloc but no exception is being thrown.

                    Can't find a clear way of enabling those (if something needs to be enabled, that is - because it shouldn't, by default, IMHO).

                    1 Reply Last reply
                    0
                    • mrjjM Offline
                      mrjjM Offline
                      mrjj
                      Lifetime Qt Champion
                      wrote on 15 Jul 2020, 18:35 last edited by mrjj
                      #10

                      Hi
                      It does throw something
                      as

                      class BIG
                      {
                          std::array<int, 100000> test;
                      
                      };
                      

                      alt text
                      does go to the catch part.

                      Hmm
                      Win 10, visual studio 32bit does throw bad_alloc

                      alt text

                      but app terminates right after it leaves the scope so i think its too late at this point to continue.

                      1 Reply Last reply
                      2
                      • A Offline
                        A Offline
                        Alvein
                        wrote on 16 Jul 2020, 06:35 last edited by
                        #11

                        Thanks.

                        Finally, I implemented something like this...

                        template <class itemType>
                        bool safePush(QStack<itemType> &stkStack,itemType stkItem) {
                            bool bSP=true;
                            try {
                                stkStack.push(stkItem);
                            }
                            catch (std::bad_alloc) {
                                bSP=false;
                            }
                            return bSP;
                        }
                        

                        ...and it works now.

                        Not sure what was causing the exception ignoring, tho.

                        JonBJ 1 Reply Last reply 16 Jul 2020, 09:39
                        1
                        • A Alvein
                          16 Jul 2020, 06:35

                          Thanks.

                          Finally, I implemented something like this...

                          template <class itemType>
                          bool safePush(QStack<itemType> &stkStack,itemType stkItem) {
                              bool bSP=true;
                              try {
                                  stkStack.push(stkItem);
                              }
                              catch (std::bad_alloc) {
                                  bSP=false;
                              }
                              return bSP;
                          }
                          

                          ...and it works now.

                          Not sure what was causing the exception ignoring, tho.

                          JonBJ Offline
                          JonBJ Offline
                          JonB
                          wrote on 16 Jul 2020, 09:39 last edited by
                          #12

                          @Alvein
                          Just to say: if you are so close to running out of memory that pushing one item to a stack causes std::bad_alloc, you may succeed in trapping the error and continuing as you have, but is it not likely that as you continue something else fails disastrously as there is no memory left to allocate?

                          A 1 Reply Last reply 16 Jul 2020, 17:39
                          0
                          • JonBJ JonB
                            16 Jul 2020, 09:39

                            @Alvein
                            Just to say: if you are so close to running out of memory that pushing one item to a stack causes std::bad_alloc, you may succeed in trapping the error and continuing as you have, but is it not likely that as you continue something else fails disastrously as there is no memory left to allocate?

                            A Offline
                            A Offline
                            Alvein
                            wrote on 16 Jul 2020, 17:39 last edited by
                            #13

                            @JonB Not exactly sure if the "no memory" exception means no memory for a single continuous block. But I reckon I'm just being paranoid. I'm handling the stack myself since originally the call stack was not enough, so I just added the check because it looks good (and for learning something in the middle).

                            In the end, even if I catch the exception, that's not a dead end, since as it's coded, the process can be restarted from where it was, with an empty stack again.

                            TBH, by design, not even a restart is required because the process backtracks when the memory is full, freeing private stack space. I'll have to see it running live. But the probability of seeing it reaching the exception is terribly low.

                            JonBJ 1 Reply Last reply 16 Jul 2020, 17:51
                            0
                            • A Alvein
                              16 Jul 2020, 17:39

                              @JonB Not exactly sure if the "no memory" exception means no memory for a single continuous block. But I reckon I'm just being paranoid. I'm handling the stack myself since originally the call stack was not enough, so I just added the check because it looks good (and for learning something in the middle).

                              In the end, even if I catch the exception, that's not a dead end, since as it's coded, the process can be restarted from where it was, with an empty stack again.

                              TBH, by design, not even a restart is required because the process backtracks when the memory is full, freeing private stack space. I'll have to see it running live. But the probability of seeing it reaching the exception is terribly low.

                              JonBJ Offline
                              JonBJ Offline
                              JonB
                              wrote on 16 Jul 2020, 17:51 last edited by
                              #14

                              @Alvein

                              because the process backtracks when the memory is full, freeing private stack space

                              Again, just an observation :) I think you might be being too optimistic! You seem to assume two things:

                              • Freed blocks will get re-used, at this point, for another allocation. May depend on whether it's an odd or even day of the week ;)

                              • The code executed for backtracking does not itself do any allocation, which might fail where you are. There are a lot of things you might call without realizing they do some allocation!

                              Best of luck.

                              A 1 Reply Last reply 16 Jul 2020, 20:13
                              1
                              • JonBJ JonB
                                16 Jul 2020, 17:51

                                @Alvein

                                because the process backtracks when the memory is full, freeing private stack space

                                Again, just an observation :) I think you might be being too optimistic! You seem to assume two things:

                                • Freed blocks will get re-used, at this point, for another allocation. May depend on whether it's an odd or even day of the week ;)

                                • The code executed for backtracking does not itself do any allocation, which might fail where you are. There are a lot of things you might call without realizing they do some allocation!

                                Best of luck.

                                A Offline
                                A Offline
                                Alvein
                                wrote on 16 Jul 2020, 20:13 last edited by
                                #15

                                Thank you very much for your advice.

                                • Freed blocks will get re-used, at this point, for another allocation. May depend on whether it's an odd or even day of the week ;)

                                Yes, I'm aware of that. My intention was simply avoiding my program to crash and tell the users what's happening, letting them to stop the process if they wish.

                                • The code executed for backtracking does not itself do any allocation, which might fail where you are. There are a lot of things you might call without realizing they do some allocation!

                                Backtracking, for me, means just popping out items from the private stack. Remember this is about simulating recursion, so such backtracking is also simulated. In the end, there's two options for the program, either using the recently freed memory, or behaving like if there was no more, in case that memory was immediately seized by other things. Each option has been considered and handled by code.

                                1 Reply Last reply
                                0

                                1/15

                                13 Jul 2020, 17:58

                                • Login

                                • Login or register to search.
                                1 out of 15
                                • First post
                                  1/15
                                  Last post
                                0
                                • Categories
                                • Recent
                                • Tags
                                • Popular
                                • Users
                                • Groups
                                • Search
                                • Get Qt Extensions
                                • Unsolved