Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Update: Forum Guidelines & Code of Conduct

    Solved Waterfall Spectrogram UI for Massive Amounts of Data

    General and Desktop
    6
    10
    1869
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Y
      Yippiyak last edited by Yippiyak

      Hello,
      I am relatively new to Qt, but my first assignment is designing a UI for a Spectrogram. The UI receives a test file (eventually would receive live RF Signal data), that transmits approximately 4 gigabytes of data per second. The project is about 50 or so files and around 10,000 lines of code, so there won't be an easy fix. Like I said, I am relatively new to Qt, so I may not even be able to understand any answers you give me. That said, heres what I need help with;

      1. The processing speed of the waterfall graph is 1/20th of realtime, and this is the biggest issue. Not only does the data pour in slowly, but the UI tracks this data slowly as well. A Qt rep indicated that perhaps Qt 3d could render the data without issue, but as of right now I am not entirely sure how to use Qt 3d as anything other than a fancy version of solid works.
      2. Any time I resize the window all the Data in the waterfall chart resets, but the data itself does not loop (nor should it), so essentially all data is lost.
      3. Zoom crashes the UI. (Though this issue is likely a bug)

      So, I am here for any advice you can give me, even if it is as simple as "try googling this," because I currently have a working UI in C++ that is far too slow to be useful. The design challenge is that it has to be in Qt, and the quantity of data must remain at around this size. I have read that I may potentially be able to use CUDA in some manner to offload to the GPU, but I am not sure how feasible this is in Qt. Thank you for any help you can give me,

      Y

      Edit: My manager recommended trying OpenGL, and perhaps Qt DataVisualization, but that I should avoid qml. Anyone have experience with Qt and OpenGL for rendering.

      aha_1980 kshegunov 2 Replies Last reply Reply Quote 0
      • aha_1980
        aha_1980 Lifetime Qt Champion @Yippiyak last edited by

        @Yippiyak

        4 gigabyte per second

        how can you get data in your PC at this speed?

        anyhow, the first thing you need to do is to reduce the amount of data by appropriate filtering.

        the data is lost anyway later as your eyes cannot read it from the screen at this rate.

        and yes, hardware accelleration might speed up thimgs massively.

        Qt has to stay free or it will die.

        J.Hilk jsulm 2 Replies Last reply Reply Quote 2
        • J.Hilk
          J.Hilk Moderators @aha_1980 last edited by

          @aha_1980 said in Waterfall Spectrogram UI for Massive Amounts of Data:

          how can you get data in your PC at this speed?

          there are a couple of ways, I think.

          OC-3840 for example is a network line with transmission speeds of up to 200 Gbit/s

          or

          InfiniBand allows for transmission speeds of up to 600 Gbit/s (75 GB/s)

          Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct

          Qt Needs YOUR vote: https://bugreports.qt.io/browse/QTQAINFRA-4121


          Q: What's that?
          A: It's blue light.
          Q: What does it do?
          A: It turns blue.

          1 Reply Last reply Reply Quote 3
          • jsulm
            jsulm Lifetime Qt Champion @aha_1980 last edited by

            @aha_1980 My understanding is that the data is actually coming from a file:
            "The UI receives a test file (eventually would receive live RF Signal data), that transmits approximately 4 gigabytes of data per second."
            It is an unclear description. I think 4Gbps is the sample rate or something.

            https://forum.qt.io/topic/113070/qt-code-of-conduct

            1 Reply Last reply Reply Quote 2
            • kshegunov
              kshegunov Moderators @Yippiyak last edited by

              Check out this thread.

              @J.Hilk said in Waterfall Spectrogram UI for Massive Amounts of Data:

              InfiniBand allows for transmission speeds of up to 600 Gbit/s (75 GB/s)

              Do you know the price of an inifiniband interface and the switches, and the infrastructure? It's a pretty neat package for home ... if you can afford a third world country on your own ... ;)

              Read and abide by the Qt Code of Conduct

              J.Hilk 1 Reply Last reply Reply Quote 3
              • Y
                Yippiyak last edited by

                I suppose its worth making a little clearer, this is not a home project on a simple PC. The hardware is a couple hundred grand haha. Can't specify too much, my task is simply to figure out how to optimize the Qt software to run the UI smoothly. The data itself is a rather fixed input from the array. Its less about reading all of the data at that rate, but simply tracking it smoothly. The code runs well in C#, but Im having difficulty adapting the project to C++ in Qt.

                mrjj 1 Reply Last reply Reply Quote 0
                • mrjj
                  mrjj Lifetime Qt Champion @Yippiyak last edited by

                  @Yippiyak
                  Hi I would add some timing function to try to find out where the time is spend.
                  Do you use threads?

                  Y 1 Reply Last reply Reply Quote 0
                  • Y
                    Yippiyak @mrjj last edited by Yippiyak

                    @mrjj I do use some threading. And its less a matter of data processing so much as rendering, but I prefer not to rely on a client having a GPU. Just spoke with my project manager, and he said try avoiding qml, but that OpenGL would be helpful. Anyone have experience with this?
                    https://doc.qt.io/qt-5.11/qtdatavisualization-index.html

                    jsulm 1 Reply Last reply Reply Quote 0
                    • jsulm
                      jsulm Lifetime Qt Champion @Yippiyak last edited by

                      @Yippiyak "but I prefer not to rely on a client having a GPU" - a client for data visualisation without a GPU? What would that be? Even smartphones have a GPU.

                      https://forum.qt.io/topic/113070/qt-code-of-conduct

                      1 Reply Last reply Reply Quote 1
                      • J.Hilk
                        J.Hilk Moderators @kshegunov last edited by

                        @kshegunov said in Waterfall Spectrogram UI for Massive Amounts of Data:

                        Check out this thread.

                        @J.Hilk said in Waterfall Spectrogram UI for Massive Amounts of Data:

                        InfiniBand allows for transmission speeds of up to 600 Gbit/s (75 GB/s)

                        Do you know the price of an inifiniband interface and the switches, and the infrastructure? It's a pretty neat package for home ... if you can afford a third world country on your own ... ;)

                        😎 Don't we all want Jeff Bezos income?

                        @Yippiyak said in Waterfall Spectrogram UI for Massive Amounts of Data:

                        @mrjj I do use some threading. And its less a matter of data processing so much as rendering, but I prefer not to rely on a client having a GPU. Just spoke with my project manager, and he said try avoiding qml, but that OpenGL would be helpful. Anyone have experience with this?
                        https://doc.qt.io/qt-5.11/qtdatavisualization-index.html

                        If it's such a "high profile" case, where cost is a secundary concern, I would try reaching out to the Qt-Company for professional consulting!

                        Should get you further and quicker than a user forum 😉

                        Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct

                        Qt Needs YOUR vote: https://bugreports.qt.io/browse/QTQAINFRA-4121


                        Q: What's that?
                        A: It's blue light.
                        Q: What does it do?
                        A: It turns blue.

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post