Copying two files in a third one...

[croussou]

Hi there,

I am interested in copying two files and combining them into a third file. Didn't manage to find that much information. Any suggestions would be appreciated.

Thank you in advance.

[Guest]

It is as simple as adding two binary data streams from the files together and outputting the result into the new file. If the files are bigger you might want to save yourself creating the third data stream and just append the two into the combined file.

Note that some file formats have headers and footers which means unless your file contains only raw data you may not get the result you expect.

Take a look at the QDataStream class:
http://developer.qt.nokia.com/doc/qt-4.8/qdatastream.html

[croussou]

Thank you for your prompt reply.

Could you please provide more information, I am not entirely familiar with that content.

Any chance for an example? What is considered raw data?

[JohnKaul]

A question: Are these "plain text" or "binary" files?

[croussou]

Both I guess, is it possible to combine?

Thank you.

[Guest]

For text specifically, this should work:

@QString text1, text2;

QFile file1("c:/test1.txt");
QTextStream in1;
if (file1.open(QIODevice::ReadOnly))
{
in1.setDevice(&file1);
}

QFile file2("c:/test2.txt");
QTextStream in2;
if (file2.open(QIODevice::ReadOnly))
{
in2.setDevice(&file2);
}

in1 >> text1;
in2 >> text2;

QFile result("c:/result.txt");
QTextStream res;
if (result.open(QIODevice::WriteOnly))
{
res.setDevice(&result);
}

res << text1 << text2;@

It basically creates two strings, two input files, two text streams, inputs the text streams from the files into the strings, and finally outputs the strings into the third text stream which is the output file.

[croussou]

Thank you.

I get QTextStream::No device...

[JohnKaul]

[quote author="croussou" date="1329947758"]Both I guess, is it possible to combine?
Thank you.[/quote]

Why don't you start with "plain text" and move on from there.

Here is a very simple (aka: basic) example. This example doesnt use any Qt, only standard C++ stuff.
@
int main(int argc, const char *argv[])
{
std::string InFile("InputOneFileTest.txt");
std::fstream inputOne(InFile.c_str(), std::fstream::in);

std::string OutFile("OutputFileTest.txt");
std::fstream output(OutFile.c_str(), std::fstream::out);

std::string str;
while (getline(inputOne, str))
    output << str << '\n';

InFile = "InputTwoFileTest.txt";
std::fstream inputTwo(InFile.c_str(), std::fstream::in);

while (getline(inputTwo, str))
    output << str << '\n';

return 0;

}
@

[Guest]

do you have the text1 and text2 files in your c:/ ?

you need to replace those with the files you actually want to combine

[croussou]

Nevermind, misspelled the file name, I think its getting late...

I have written something in text1 and something in text2...after execution result remains empty...

[Guest]

this is odd, it works for me, I had "123" and "456" in text1 and text2 respectively, and got "123456" in result.

Have you included QtCore and QFile?

ALso check if result is not open, also you may want to add:

@result.close();@

at the end to make sure your file doesn't stay open in which case the program will not be able to write in it

[croussou]

Here is the code...

@#include <QtCore/QCoreApplication>
#include <QFile>
#include <QTextStream>

int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);

QString text1, text2;

     QFile file1("/Users/croussou_dm4/Desktop/text1.txt");
     QTextStream in1;
     if (file1.open(QIODevice::ReadOnly))
     {
 in1.setDevice(&file1);
     }

     QFile file2("/Users/croussou_dm4/Desktop/text2.txt");
     QTextStream in2;
     if (file2.open(QIODevice::ReadOnly))
     {
 in2.setDevice(&file2);
     }

     QFile result("/Users/croussou_dm4/Desktop/result.txt");
     QTextStream res;
     if (result.open(QIODevice::WriteOnly))
     {
 res.setDevice(&result);
     }

     in1 >> text1;
     in2 >> text2;

     res << text1 << text2;

return a.exec(&#41;;

}
@

Text1 and Text2 are there, after I run the app the Result text file appears but empty... :S

[Guest]

Check my previous post, I edited it, maybe your result file stays open in which case the program will not write to it.

[croussou]

That's it, works perfectly now. Thanks a lot, both of you.

Now, can I do that with any type of file?

[Guest]

No responders, it seems like I will have to come to the rescue again. I hope I am not doing your homework ;)

This should work for all type of files as it does raw data copy:

@QDataStream in1, in2, out;
QFile file1("c:/test1.txt");
if (!file1.open(QIODevice::ReadOnly))
return EXIT_FAILURE;

in1.setDevice(&file1);

QFile file2("c:/test2.txt");
if (!file2.open(QIODevice::ReadOnly))
return EXIT_FAILURE;

in2.setDevice(&file2);

QFile result("c:/result.txt");
if (!result.open(QIODevice::WriteOnly))
return EXIT_FAILURE;

out.setDevice(&result);

char *data1 = new char[file1.size()];
char *data2 = new char[file2.size()];

in1.readRawData(data1, file1.size());
in2.readRawData(data2, file2.size());

out.writeRawData(data1, file1.size());
out.writeRawData(data2, file2.size());

result.close();

delete [] data1;
delete [] data2;@

To anyone more experienced, I am curious if there is a way to directly copy raw data from disk without using the dynamically allocated char[] to read in the two files. Thanks!

[croussou]

That code did the trick. I honestly appreciate your help, it was essential for proceeding with this project.

P.S This is an individual project for getting more familiar with Qt and file operations. Has nothing to do with school homework.

Again, thanks a bunch.

[Guest]

You are welcome, just note this implementation stores copies of both files in memory, so if you use it to merge really big files you might get very low performance or even an error.

[andre]

[quote author="ddriver" date="1329954109"]No responders, it seems like I will have to come to the rescue again. I hope I am not doing your homework ;)

This should work for all type of files as it does raw data copy:

@QDataStream in1, in2, out;
QFile file1("c:/test1.txt");
if (!file1.open(QIODevice::ReadOnly))
return EXIT_FAILURE;

in1.setDevice(&file1);

QFile file2("c:/test2.txt");
if (!file2.open(QIODevice::ReadOnly))
return EXIT_FAILURE;

in2.setDevice(&file2);

QFile result("c:/result.txt");
if (!result.open(QIODevice::WriteOnly))
return EXIT_FAILURE;

out.setDevice(&result);

char *data1 = new char[file1.size()];
char *data2 = new char[file2.size()];

in1.readRawData(data1, file1.size());
in2.readRawData(data2, file2.size());

out.writeRawData(data1, file1.size());
out.writeRawData(data2, file2.size());

result.close();

delete [] data1;
delete [] data2;@

To anyone more experienced, I am curious if there is a way to directly copy raw data from disk without using the dynamically allocated char[] to read in the two files. Thanks![/quote]

Yes, there is. It is never a good idea to just allocate a block of memory the size of a file you don't know. What if the file is 2GB, the other file is big as well? You'll run out of memory in no time flat. Instead, you read the file block by block, perhaps in blocks of 4KB or so. Note that Qt has a QByteArray class you can use for this purpose, instead of a raw char array.

Also, note that there really isn't a need to read in the first file at all. Simply copy the first file to the new file first (the OS can handle that more efficiently than you can), and then open the copy in append mode and add whatever is in the secons file to it.

Last, for croussou:

Note that if you concatenate binary files like this, the file is not likely to be valid any more. For instance, if you add two mp3 files together like this, the result will not be one mp3 file that just has both original songs, and combining two jpg images will not result in one bigger image that has both. There is no generic way to concatenate files in a meaningful way.

[Guest]

Yes that was what I had in mind last night but it was kind of late so I went with the "simpler" solution. BTW I already warned him about merging files that contain format headers, this is applicable ONLY for raw data.

For specific formats the concatenation must also be format specific, you must know how to parse that file, read in header tags and process them and only merge the raw data.

I was also thinking about pre-allocating the output file to the size of both inputs so that fragmentation can be avoided.

Since we are now talking BIG files, it would be wise to abort the operation early on in case there is not enough free space on disk, not merge the files all the way to 90% and fail just then, when time has been waster and the entire disk has been filled, which may have other negative implications.

[croussou]

What is a file with format headers?

I have tried some operations with .7z files and seems to work. The files I intend to concatenate are fairly small, so I guess I should not worry about memory?