Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. output a std::vector<ushort> to file.
Forum Updated to NodeBB v4.3 + New Features

output a std::vector<ushort> to file.

Scheduled Pinned Locked Moved Solved General and Desktop
10 Posts 6 Posters 2.9k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R Offline
    R Offline
    rafael
    wrote on 7 Feb 2018, 22:08 last edited by rafael 2 Jul 2018, 22:24
    #1

    I am using qdatastream to write to file it takes about 4 minutes to write that many data points. I was wondering if there is anyway to speed it up?

    QFile file("C:/Temp/file.dat");
    file.open(QIODevice::WriteOnly);
    QDataStream out(&file);
    
    for(int i=0; i < 691200000; i++)
    {
           out << data[i];
    }
    

    Is there a way to write all the contents of the vector to file without using a for loop using qdatastream?

    out << &data[0]
    

    the code below is faster only 58 seconds but its not qdatastream

    size_t sz = data.size();
    std::ofstream FILE("C:/Temp/example.dat", std::ios::out | std::ofstream::binary);
    FILE.write(reinterpret_cast<const char*>(&data[0]), sz * sizeof(data[0]));
    
    1 Reply Last reply
    0
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 7 Feb 2018, 22:29 last edited by
      #2

      Hi,

      What about QDataStream::writeRawData ?

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      1 Reply Last reply
      4
      • R Offline
        R Offline
        rafael
        wrote on 7 Feb 2018, 22:38 last edited by
        #3

        thanks! averages around 35 seconds now.

        1 Reply Last reply
        1
        • M Offline
          M Offline
          mrjj
          Lifetime Qt Champion
          wrote on 7 Feb 2018, 22:50 last edited by
          #4

          Hi
          That is really an improvement over 4 mins :)
          Do you really have 600 millions items?

          R 1 Reply Last reply 8 Feb 2018, 14:54
          1
          • M mrjj
            7 Feb 2018, 22:50

            Hi
            That is really an improvement over 4 mins :)
            Do you really have 600 millions items?

            R Offline
            R Offline
            rafael
            wrote on 8 Feb 2018, 14:54 last edited by
            #5

            @mrjj yes :) I'm trying to generate dummy data for a logging device.1000Hz * 4 bytes * 24 hours * 3600 seconds * 4 days.

            V 1 Reply Last reply 13 Feb 2018, 08:24
            1
            • ? Offline
              ? Offline
              A Former User
              wrote on 12 Feb 2018, 22:36 last edited by A Former User 2 Dec 2018, 23:56
              #6

              Code programmers say Assembler is only twice as fast, Assembly programmers say it's 5 times faster than C code.

              for(int i=0; i < 691200000; i++)
              {
              out << data[i];
              }

              Make a function and put it into Assembly Language, and import it as a module.

              I haven't noticed any ASM stuff. Does Qt allow this? Bring back Turbo C ! :-)

              J 1 Reply Last reply 13 Feb 2018, 05:38
              0
              • ? A Former User
                12 Feb 2018, 22:36

                Code programmers say Assembler is only twice as fast, Assembly programmers say it's 5 times faster than C code.

                for(int i=0; i < 691200000; i++)
                {
                out << data[i];
                }

                Make a function and put it into Assembly Language, and import it as a module.

                I haven't noticed any ASM stuff. Does Qt allow this? Bring back Turbo C ! :-)

                J Offline
                J Offline
                jsulm
                Lifetime Qt Champion
                wrote on 13 Feb 2018, 05:38 last edited by
                #7

                @Ian-Bray "Does Qt allow this?" - Qt isn't a programming language, it is a C++ framework. The question is: does C++ allow to use Assembler. Yes, it does. Depending on compiler you can use inline assembler or you write assembler code and use assembler to generate a binary which you then can link statically.
                But I'm wondering why you're bringing Assembler here? To benefit from Assembler you should really know what you are doing. How many programmers are able to use Assembler today? Also Assembler isn't portable. In most cases it does not make sense to use it. Especially in this case: it is an IO heavy use case where the limiting factor is IO and not CPU.

                https://forum.qt.io/topic/113070/qt-code-of-conduct

                1 Reply Last reply
                3
                • R rafael
                  8 Feb 2018, 14:54

                  @mrjj yes :) I'm trying to generate dummy data for a logging device.1000Hz * 4 bytes * 24 hours * 3600 seconds * 4 days.

                  V Offline
                  V Offline
                  VRonin
                  wrote on 13 Feb 2018, 08:24 last edited by VRonin
                  #8

                  @rafael said in output a std::vector<ushort> to file.:

                  I'm trying to generate dummy data for a logging device

                  Do you really mean to load 1.2GB in RAM for logging?!

                  "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
                  ~Napoleon Bonaparte

                  On a crusade to banish setIndexWidget() from the holy land of Qt

                  1 Reply Last reply
                  0
                  • ? Offline
                    ? Offline
                    A Former User
                    wrote on 13 Feb 2018, 22:23 last edited by
                    #9

                    Oh for fuck sake! Just trying to be helpful. I'll keep my mouth shut in future. AND it was only a suggestion AND it was only semi/serious as you can see by the smilie!

                    VRonin - maybe you should change that 286. I have a drum sampler that uses 6 gigs of memory!

                    V 1 Reply Last reply 14 Feb 2018, 08:41
                    0
                    • ? A Former User
                      13 Feb 2018, 22:23

                      Oh for fuck sake! Just trying to be helpful. I'll keep my mouth shut in future. AND it was only a suggestion AND it was only semi/serious as you can see by the smilie!

                      VRonin - maybe you should change that 286. I have a drum sampler that uses 6 gigs of memory!

                      V Offline
                      V Offline
                      VRonin
                      wrote on 14 Feb 2018, 08:41 last edited by
                      #10

                      @Guest said in output a std::vector<ushort> to file.:

                      I have a drum sampler that uses 6 gigs of memory

                      No, I agree, there are applications that need it. I would expect stuff like browsers, adobe CS and videogames to devour RAM but not a logger. You can probably use an SQLite DB to do that without any meaningful loss of speed

                      P.S.
                      For people coming here from Google, the writeRawData/reinterpret_cast solution only works with std::vector<T> ,QVector<T>, std::array<T> and only if T is Q_PRIMITIVE_TYPE

                      "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
                      ~Napoleon Bonaparte

                      On a crusade to banish setIndexWidget() from the holy land of Qt

                      1 Reply Last reply
                      2

                      1/10

                      7 Feb 2018, 22:08

                      • Login

                      • Login or register to search.
                      1 out of 10
                      • First post
                        1/10
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Users
                      • Groups
                      • Search
                      • Get Qt Extensions
                      • Unsolved