output a std::vector<ushort> to file.

rafael

I am using qdatastream to write to file it takes about 4 minutes to write that many data points. I was wondering if there is anyway to speed it up?

QFile file("C:/Temp/file.dat");
file.open(QIODevice::WriteOnly);
QDataStream out(&file);

for(int i=0; i < 691200000; i++)
{
       out << data[i];
}

Is there a way to write all the contents of the vector to file without using a for loop using qdatastream?

out << &data[0]

the code below is faster only 58 seconds but its not qdatastream

size_t sz = data.size();
std::ofstream FILE("C:/Temp/example.dat", std::ios::out | std::ofstream::binary);
FILE.write(reinterpret_cast<const char*>(&data[0]), sz * sizeof(data[0]));

SGaist

Hi,

What about QDataStream::writeRawData ?

rafael

thanks! averages around 35 seconds now.

mrjj

Hi
That is really an improvement over 4 mins :)
Do you really have 600 millions items?

rafael

@mrjj yes :) I'm trying to generate dummy data for a logging device.1000Hz * 4 bytes * 24 hours * 3600 seconds * 4 days.

? Offline

Code programmers say Assembler is only twice as fast, Assembly programmers say it's 5 times faster than C code.

for(int i=0; i < 691200000; i++)
{
out << data[i];
}

Make a function and put it into Assembly Language, and import it as a module.

I haven't noticed any ASM stuff. Does Qt allow this? Bring back Turbo C ! :-)

jsulm

@Ian-Bray "Does Qt allow this?" - Qt isn't a programming language, it is a C++ framework. The question is: does C++ allow to use Assembler. Yes, it does. Depending on compiler you can use inline assembler or you write assembler code and use assembler to generate a binary which you then can link statically.
But I'm wondering why you're bringing Assembler here? To benefit from Assembler you should really know what you are doing. How many programmers are able to use Assembler today? Also Assembler isn't portable. In most cases it does not make sense to use it. Especially in this case: it is an IO heavy use case where the limiting factor is IO and not CPU.

VRonin

@rafael said in output a std::vector<ushort> to file.:

I'm trying to generate dummy data for a logging device

Do you really mean to load 1.2GB in RAM for logging?!

? Offline

Oh for fuck sake! Just trying to be helpful. I'll keep my mouth shut in future. AND it was only a suggestion AND it was only semi/serious as you can see by the smilie!

VRonin - maybe you should change that 286. I have a drum sampler that uses 6 gigs of memory!

VRonin

@Guest said in output a std::vector<ushort> to file.:

I have a drum sampler that uses 6 gigs of memory

No, I agree, there are applications that need it. I would expect stuff like browsers, adobe CS and videogames to devour RAM but not a logger. You can probably use an SQLite DB to do that without any meaningful loss of speed

P.S.
For people coming here from Google, the writeRawData/reinterpret_cast solution only works with std::vector<T> ,QVector<T>, std::array<T> and only if T is Q_PRIMITIVE_TYPE