Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Overcoming the memory limitation of QGraphicsView
Forum Updated to NodeBB v4.3 + New Features

Overcoming the memory limitation of QGraphicsView

Scheduled Pinned Locked Moved Unsolved General and Desktop
11 Posts 5 Posters 2.3k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J Offline
    J Offline
    j_omega
    wrote on last edited by
    #1

    I am working on a custom map project. My software currently uses QGraphicsScene to load the map data set into memory and then QGraphicsView to render the data. I have been very happy with this solution, but it has scaling problems. Right now, I am only rendering a small subset of my full data set. The full data set is >1GB and my target platform only has 512MB of RAM. Obviously this won't work due to RAM size. I am trying to figure out the best way to approach this problem.

    Is there a way to make QGraphicsView render from a database file instead of loading everything into memory?

    I am wondering if I have hit the limits of QGraphicsView and need to abandon it for a custom tiling approach. If so, are there any recommendations for this?

    1 Reply Last reply
    0
    • mrjjM Offline
      mrjjM Offline
      mrjj
      Lifetime Qt Champion
      wrote on last edited by
      #2

      Hi
      The limit is not QGraphicsView in terms of items possible -but the very low
      memory you have to work with.
      http://doc.qt.io/qt-5/qtwidgets-graphicsview-chip-example.html

      What is this data ?
      How much memory does each piece of data require ?

      J 1 Reply Last reply
      0
      • mrjjM mrjj

        Hi
        The limit is not QGraphicsView in terms of items possible -but the very low
        memory you have to work with.
        http://doc.qt.io/qt-5/qtwidgets-graphicsview-chip-example.html

        What is this data ?
        How much memory does each piece of data require ?

        J Offline
        J Offline
        j_omega
        wrote on last edited by j_omega
        #3

        @mrjj I am using data from the TIGER geodatabase. These files are translated from shapefiles into QPolygons which are then loaded into a QGraphicsScene. The size per dataset can be upwards of 100MB

        Is there a different approach I can take to limit the memory usage of QGraphicsScene?

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #4

          Hi,

          One thing that comes to mind is to limit the number of elements you present on screen.

          Do you really need to load them all ? Can you display only parts of them and then use some kind of window to always only show a subset of them ?

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          1 Reply Last reply
          2
          • BuckwheatB Offline
            BuckwheatB Offline
            Buckwheat
            wrote on last edited by
            #5

            @SGaist said in Overcoming the memory limitation of QGraphicsView:

            ally need to load them all ? Can you display only parts of them and then use some kind of window to always only show a subset of them ?

            Hi @j_omega!

            I agree with SGaist. I had a similar situation where I was loading navigational map databases. I could only keep a 9x9 (6x6 visible) matrix of images in memory at any given time. This presented a problem because at different ranges, the window size of the data changed.

            I ended up solving the problem by making a server style class that would monitor my position and feed me file/rectangle to load areas for each element of the matrix.

            Dave Fileccia

            J 1 Reply Last reply
            0
            • BuckwheatB Buckwheat

              @SGaist said in Overcoming the memory limitation of QGraphicsView:

              ally need to load them all ? Can you display only parts of them and then use some kind of window to always only show a subset of them ?

              Hi @j_omega!

              I agree with SGaist. I had a similar situation where I was loading navigational map databases. I could only keep a 9x9 (6x6 visible) matrix of images in memory at any given time. This presented a problem because at different ranges, the window size of the data changed.

              I ended up solving the problem by making a server style class that would monitor my position and feed me file/rectangle to load areas for each element of the matrix.

              J Offline
              J Offline
              j_omega
              wrote on last edited by
              #6

              @Buckwheat So maybe I could do something with QGraphicsItemGroup. I can think of my map as a matrix of "tiles". Then I would query my geo-database for items in nearby tiles. Each tile would loaded into its own QGraphicsItemGroup. Then these groups could be easily added or removed to the scenegraph as the map moves.

              1 Reply Last reply
              0
              • A Offline
                A Offline
                Asperamanca
                wrote on last edited by
                #7

                The question is: Are you ever in a situation where the user wants an overview over the whole data ("zoom out"), then see every detail in a certain spot ("zoom in")? In that case, a combination of tiles, level of detail (LOD) and smart data caching should work.

                • For tiling, I would create a custom QGraphicsItem that can draw a part of your scene.
                • For level of detail, see the 40000 chips demo.
                • Smart caching: Make the single items a lightweight shell that can draw a tile at different level of details, without the need to have all data stored in memory. Add a cache so the currently visible tiles have the data they need for the current level of detail.
                • If the user zooms in, the tiles that are no longer visible can empty their caches. The tiles that are now visible in higher detail need to load their data in higher resolution
                • If the user zooms out, visible tiles can reduce their memory load trimming data to the amount needed for the current LOD. New tiles become visible, they need to load and cache some data

                Don't always "new" and "delete" items. Instead, make a tile displaying no data very lightweight. That way, you can keep your scene structure fully intact, and just load or unload data as the user scroll and zooms.

                J 1 Reply Last reply
                1
                • BuckwheatB Offline
                  BuckwheatB Offline
                  Buckwheat
                  wrote on last edited by
                  #8

                  @j_omega , @Asperamanca gives a good algorithm. QGraphicsItemGroup can be used for "ALL" the tiles you wish to display but I would only do that if it was one piece of multiple display items. In my case it was the background map. There were waypoints and text and range lines as the primary pieces.

                  You would still need to manage the matrix. I created a class called TileItem that was derived from QSharedData and stored QSharedDataPointer (nice reference counted pointer). When No data could be found for the item I just had an empty QSharedDataPointer.

                  The reason to use them is to only have one item loaded but to be able to share it as needed without making copies. A lot of the underlying data items in Qt are done this way so it conserves memory. I used my own QGraphicsItem derived class to contain the pointer and it overrode paint and boundingRect. Positions were then set by the map item data.

                  My LOD maxed out at 360nm radius and started at 1.5nm radius so zooming was never too much of an issue.

                  Dave Fileccia

                  J 1 Reply Last reply
                  0
                  • A Asperamanca

                    The question is: Are you ever in a situation where the user wants an overview over the whole data ("zoom out"), then see every detail in a certain spot ("zoom in")? In that case, a combination of tiles, level of detail (LOD) and smart data caching should work.

                    • For tiling, I would create a custom QGraphicsItem that can draw a part of your scene.
                    • For level of detail, see the 40000 chips demo.
                    • Smart caching: Make the single items a lightweight shell that can draw a tile at different level of details, without the need to have all data stored in memory. Add a cache so the currently visible tiles have the data they need for the current level of detail.
                    • If the user zooms in, the tiles that are no longer visible can empty their caches. The tiles that are now visible in higher detail need to load their data in higher resolution
                    • If the user zooms out, visible tiles can reduce their memory load trimming data to the amount needed for the current LOD. New tiles become visible, they need to load and cache some data

                    Don't always "new" and "delete" items. Instead, make a tile displaying no data very lightweight. That way, you can keep your scene structure fully intact, and just load or unload data as the user scroll and zooms.

                    J Offline
                    J Offline
                    j_omega
                    wrote on last edited by j_omega
                    #9

                    @Asperamanca said in Overcoming the memory limitation of QGraphicsView:

                    The question is: Are you ever in a situation where the user wants an overview over the whole data ("zoom out"), then see every detail in a certain spot ("zoom in")? In that case, a combination of tiles, level of detail (LOD) and smart data caching should work.

                    Yes, the map needs to be able to handle a wide range of zoom levels.

                    • For tiling, I would create a custom QGraphicsItem that can draw a part of your scene.
                    • For level of detail, see the 40000 chips demo.
                    • Smart caching: Make the single items a lightweight shell that can draw a tile at different level of details, without the need to have all data stored in memory. Add a cache so the currently visible tiles have the data they need for the current level of detail.
                    • If the user zooms in, the tiles that are no longer visible can empty their caches. The tiles that are now visible in higher detail need to load their data in higher resolution
                    • If the user zooms out, visible tiles can reduce their memory load trimming data to the amount needed for the current LOD. New tiles become visible, they need to load and cache some data

                    Don't always "new" and "delete" items. Instead, make a tile displaying no data very lightweight. That way, you can keep your scene structure fully intact, and just load or unload data as the user scroll and zooms.

                    This approach seems very good except for one part. I do not know how the graphics item tile would know when it had gone out-of-view.

                    1 Reply Last reply
                    0
                    • BuckwheatB Buckwheat

                      @j_omega , @Asperamanca gives a good algorithm. QGraphicsItemGroup can be used for "ALL" the tiles you wish to display but I would only do that if it was one piece of multiple display items. In my case it was the background map. There were waypoints and text and range lines as the primary pieces.

                      You would still need to manage the matrix. I created a class called TileItem that was derived from QSharedData and stored QSharedDataPointer (nice reference counted pointer). When No data could be found for the item I just had an empty QSharedDataPointer.

                      The reason to use them is to only have one item loaded but to be able to share it as needed without making copies. A lot of the underlying data items in Qt are done this way so it conserves memory. I used my own QGraphicsItem derived class to contain the pointer and it overrode paint and boundingRect. Positions were then set by the map item data.

                      My LOD maxed out at 360nm radius and started at 1.5nm radius so zooming was never too much of an issue.

                      J Offline
                      J Offline
                      j_omega
                      wrote on last edited by
                      #10

                      @Buckwheat said in Overcoming the memory limitation of QGraphicsView:

                      @j_omega , @Asperamanca gives a good algorithm. QGraphicsItemGroup can be used for "ALL" the tiles you wish to display but I would only do that if it was one piece of multiple display items. In my case it was the background map. There were waypoints and text and range lines as the primary pieces.

                      When you talk about "multiple display items", are you referring to different layers/overlays of the map?

                      You would still need to manage the matrix. I created a class called TileItem that was derived from QSharedData and stored QSharedDataPointer (nice reference counted pointer). When No data could be found for the item I just had an empty QSharedDataPointer.

                      The reason to use them is to only have one item loaded but to be able to share it as needed without making copies. A lot of the underlying data items in Qt are done this way so it conserves memory. I used my own QGraphicsItem derived class to contain the pointer and it overrode paint and boundingRect. Positions were then set by the map item data.

                      My LOD maxed out at 360nm radius and started at 1.5nm radius so zooming was never too much of an issue.

                      Are you saying that each tile shared the same dataset (TileItem) and decided which part of that data to render?

                      1 Reply Last reply
                      0
                      • BuckwheatB Offline
                        BuckwheatB Offline
                        Buckwheat
                        wrote on last edited by
                        #11

                        @j_omega said in Overcoming the memory limitation of QGraphicsView:

                        you saying that each tile shared the same dataset (TileItem) and decided which part of that data to render?

                        Hi @j_omega

                        For the QDisplayItemGroup: If I had a complex graphics item. say a tiled-map, that would be in a QGraphicsItemGroup made up of QGraphicsPixmapItems so I could treat it as a single item and show/hide it as required easily as the bottom layer item. Waypoints, waypoint text, radio and airport symbols, battle area symbology, and custom zones would be part of their own QGraphicsItemGroup(s) as well.
                        For QSharedDataItem: I kept a tile manager around that loaded/unloaded tiles as I moved. The manager stored TileItems. These TIleItems could also be shared. I did not want to copy them so the QSharedDataPointer gave me nice reference counted copies. When an item was finally displayed, because it was shared, the manager could delete the item from its structure and it would not effect the drawing cycle until the render pass and then the tile would be updated in the display. Hence, only one copy of the item (which contains, row, column, pixmap [which could be a subset of a larger pixmap or a complete pixmap based on resolution]). These TileItems could be EMITed as the management thread loaded new areas. Very low overhead for signal/slot handling.

                        Dave Fileccia

                        1 Reply Last reply
                        0

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved