Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Special Interest Groups
  3. C++ Gurus
  4. How to make code dealing with calculations based on data from large data sets faster in C++?
Forum Updated to NodeBB v4.3 + New Features

How to make code dealing with calculations based on data from large data sets faster in C++?

Scheduled Pinned Locked Moved Unsolved C++ Gurus
5 Posts 4 Posters 634 Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • Sachin BhattS Offline
    Sachin BhattS Offline
    Sachin Bhatt
    wrote on last edited by
    #1

    I'm composing a code in which I have input information of size 50000x20 lattice (50000 blocks, 20 properties each). I then estimate to conclude the block I would choose to move in view of certain computations for which I need to run the circle over all blocks. Then I roll out certain improvements to the characteristic Value (an alternate variable )for this block and rehash a similar cycle again and again. Do the computations for each block inside the circle. Is calling a capability a superior and quicker method for getting it done? This code is running without meeting (in view of some combination standards) for a long. How would I manage this? Is this way to deal with managing enormous informational indexes great? Could I at any point improve? Kindly assistance.

    1 Reply Last reply
    0
    • sierdzioS Offline
      sierdzioS Offline
      sierdzio
      Moderators
      wrote on last edited by
      #2

      It's a bit hard to understand what you are describing. Also, it's hard to help without knowing more details.

      So, I'll just list some general hints, maybe they will apply to your case:

      • if you are looping over the data, use standard loops (not ranges, not iterators) - they are the fastest
      • be very careful about constness, expecially of Qt containers (they can detach and with such amount of data it will lead to a lot of lost RAM and CPU cycles)
      • use const refs, views as much as possible
      • use tight containers (QVector) to improve CPU cache usage
      • if possible - parallelize using threads, SIMD instructions or GPGPU libs
      • when hashing, use streaming (additive) approach

      (Z(:^

      1 Reply Last reply
      3
      • Kent-DorfmanK Offline
        Kent-DorfmanK Offline
        Kent-Dorfman
        wrote on last edited by
        #3

        I suppose I'll risk being labeled a heretic and mention pointer access to elements instead of [] indexing...It's a loaded argument but I almost always have better performance with pointers in my world.

        Also, be aware of data cache misses on a 1000000 element structure, especially if you are not sequentially accessing elements.

        Christian EhrlicherC 1 Reply Last reply
        1
        • Kent-DorfmanK Kent-Dorfman

          I suppose I'll risk being labeled a heretic and mention pointer access to elements instead of [] indexing...It's a loaded argument but I almost always have better performance with pointers in my world.

          Also, be aware of data cache misses on a 1000000 element structure, especially if you are not sequentially accessing elements.

          Christian EhrlicherC Offline
          Christian EhrlicherC Offline
          Christian Ehrlicher
          Lifetime Qt Champion
          wrote on last edited by
          #4

          @Kent-Dorfman said in How to make code dealing with calculations based on data from large data sets faster in C++?:

          I suppose I'll risk being labeled a heretic and mention pointer access to elements instead of [] indexing...It's a loaded argument but I almost always have better performance with pointers in my world.

          Then you underestimate recent compilers... even it's a simple testcase the code is the same for the last three ones, and nearly the same like the first.

          Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
          Visit the Qt Academy at https://academy.qt.io/catalog

          1 Reply Last reply
          1
          • Kent-DorfmanK Offline
            Kent-DorfmanK Offline
            Kent-Dorfman
            wrote on last edited by
            #5

            @Christian-Ehrlicher said in How to make code dealing with calculations based on data from large data sets faster in C++?:

            Then you underestimate recent compilers

            there is a difference between underestimating and not being able to use something, especially when dealing with multiple CPU architectures...but as a I wrote, a loaded argument.

            1 Reply Last reply
            1

            • Login

            • Login or register to search.
            • First post
              Last post
            0
            • Categories
            • Recent
            • Tags
            • Popular
            • Users
            • Groups
            • Search
            • Get Qt Extensions
            • Unsolved