Copying two files in a third one...



  • Hi there,

    I am interested in copying two files and combining them into a third file. Didn't manage to find that much information. Any suggestions would be appreciated.

    Thank you in advance.



  • It is as simple as adding two binary data streams from the files together and outputting the result into the new file. If the files are bigger you might want to save yourself creating the third data stream and just append the two into the combined file.

    Note that some file formats have headers and footers which means unless your file contains only raw data you may not get the result you expect.

    Take a look at the QDataStream class:
    http://developer.qt.nokia.com/doc/qt-4.8/qdatastream.html



  • Thank you for your prompt reply.

    Could you please provide more information, I am not entirely familiar with that content.

    Any chance for an example? What is considered raw data?



  • A question: Are these "plain text" or "binary" files?



  • Both I guess, is it possible to combine?

    Thank you.



  • For text specifically, this should work:

    @QString text1, text2;

    QFile file1("c:/test1.txt");
    QTextStream in1;
    if (file1.open(QIODevice::ReadOnly))
    {
    in1.setDevice(&file1);
    }

    QFile file2("c:/test2.txt");
    QTextStream in2;
    if (file2.open(QIODevice::ReadOnly))
    {
    in2.setDevice(&file2);
    }

    in1 >> text1;
    in2 >> text2;

    QFile result("c:/result.txt");
    QTextStream res;
    if (result.open(QIODevice::WriteOnly))
    {
    res.setDevice(&result);
    }

    res << text1 << text2;@

    It basically creates two strings, two input files, two text streams, inputs the text streams from the files into the strings, and finally outputs the strings into the third text stream which is the output file.



  • Thank you.

    I get QTextStream::No device...



  • [quote author="croussou" date="1329947758"]Both I guess, is it possible to combine?
    Thank you.[/quote]

    Why don't you start with "plain text" and move on from there.

    Here is a very simple (aka: basic) example. This example doesnt use any Qt, only standard C++ stuff.
    @
    int main(int argc, const char *argv[])
    {
    std::string InFile("InputOneFileTest.txt");
    std::fstream inputOne(InFile.c_str(), std::fstream::in);

    std::string OutFile&#40;"OutputFileTest.txt"&#41;;
    std::fstream output(OutFile.c_str(), std::fstream::out);
    
    std::string str;
    while (getline(inputOne, str))
        output << str << '\n';
    
    InFile = "InputTwoFileTest.txt";
    std::fstream inputTwo(InFile.c_str(), std::fstream::in);
    
    while (getline(inputTwo, str))
        output << str << '\n';
    
    return 0;
    

    }
    @



  • do you have the text1 and text2 files in your c:/ ?

    you need to replace those with the files you actually want to combine



  • Nevermind, misspelled the file name, I think its getting late...

    I have written something in text1 and something in text2...after execution result remains empty...



  • this is odd, it works for me, I had "123" and "456" in text1 and text2 respectively, and got "123456" in result.

    Have you included QtCore and QFile?

    ALso check if result is not open, also you may want to add:

    @result.close();@

    at the end to make sure your file doesn't stay open in which case the program will not be able to write in it



  • Here is the code...

    @#include <QtCore/QCoreApplication>
    #include <QFile>
    #include <QTextStream>

    int main(int argc, char *argv[])
    {
    QCoreApplication a(argc, argv);

    QString text1, text2;
    
         QFile file1("/Users/croussou_dm4/Desktop/text1.txt");
         QTextStream in1;
         if (file1.open(QIODevice::ReadOnly))
         {
     in1.setDevice(&file1);
         }
    
         QFile file2("/Users/croussou_dm4/Desktop/text2.txt");
         QTextStream in2;
         if (file2.open(QIODevice::ReadOnly))
         {
     in2.setDevice(&file2);
         }
    
         QFile result("/Users/croussou_dm4/Desktop/result.txt");
         QTextStream res;
         if (result.open(QIODevice::WriteOnly))
         {
     res.setDevice(&result);
         }
    
         in1 >> text1;
         in2 >> text2;
    
         res << text1 << text2;
    
    return a.exec(&#41;;
    

    }
    @

    Text1 and Text2 are there, after I run the app the Result text file appears but empty... :S



  • Check my previous post, I edited it, maybe your result file stays open in which case the program will not write to it.



  • That's it, works perfectly now. Thanks a lot, both of you.

    Now, can I do that with any type of file?



  • No responders, it seems like I will have to come to the rescue again. I hope I am not doing your homework ;)

    This should work for all type of files as it does raw data copy:

    @QDataStream in1, in2, out;
    QFile file1("c:/test1.txt");
    if (!file1.open(QIODevice::ReadOnly))
    return EXIT_FAILURE;

    in1.setDevice(&file1);

    QFile file2("c:/test2.txt");
    if (!file2.open(QIODevice::ReadOnly))
    return EXIT_FAILURE;

    in2.setDevice(&file2);

    QFile result("c:/result.txt");
    if (!result.open(QIODevice::WriteOnly))
    return EXIT_FAILURE;

    out.setDevice(&result);

    char *data1 = new char[file1.size()];
    char *data2 = new char[file2.size()];

    in1.readRawData(data1, file1.size());
    in2.readRawData(data2, file2.size());

    out.writeRawData(data1, file1.size());
    out.writeRawData(data2, file2.size());

    result.close();

    delete [] data1;
    delete [] data2;@

    To anyone more experienced, I am curious if there is a way to directly copy raw data from disk without using the dynamically allocated char[] to read in the two files. Thanks!



  • That code did the trick. I honestly appreciate your help, it was essential for proceeding with this project.

    P.S This is an individual project for getting more familiar with Qt and file operations. Has nothing to do with school homework.

    Again, thanks a bunch.



  • You are welcome, just note this implementation stores copies of both files in memory, so if you use it to merge really big files you might get very low performance or even an error.



  • [quote author="ddriver" date="1329954109"]No responders, it seems like I will have to come to the rescue again. I hope I am not doing your homework ;)

    This should work for all type of files as it does raw data copy:

    @QDataStream in1, in2, out;
    QFile file1("c:/test1.txt");
    if (!file1.open(QIODevice::ReadOnly))
    return EXIT_FAILURE;

    in1.setDevice(&file1);

    QFile file2("c:/test2.txt");
    if (!file2.open(QIODevice::ReadOnly))
    return EXIT_FAILURE;

    in2.setDevice(&file2);

    QFile result("c:/result.txt");
    if (!result.open(QIODevice::WriteOnly))
    return EXIT_FAILURE;

    out.setDevice(&result);

    char *data1 = new char[file1.size()];
    char *data2 = new char[file2.size()];

    in1.readRawData(data1, file1.size());
    in2.readRawData(data2, file2.size());

    out.writeRawData(data1, file1.size());
    out.writeRawData(data2, file2.size());

    result.close();

    delete [] data1;
    delete [] data2;@

    To anyone more experienced, I am curious if there is a way to directly copy raw data from disk without using the dynamically allocated char[] to read in the two files. Thanks![/quote]

    Yes, there is. It is never a good idea to just allocate a block of memory the size of a file you don't know. What if the file is 2GB, the other file is big as well? You'll run out of memory in no time flat. Instead, you read the file block by block, perhaps in blocks of 4KB or so. Note that Qt has a QByteArray class you can use for this purpose, instead of a raw char array.

    Also, note that there really isn't a need to read in the first file at all. Simply copy the first file to the new file first (the OS can handle that more efficiently than you can), and then open the copy in append mode and add whatever is in the secons file to it.

    Last, for croussou:

    Note that if you concatenate binary files like this, the file is not likely to be valid any more. For instance, if you add two mp3 files together like this, the result will not be one mp3 file that just has both original songs, and combining two jpg images will not result in one bigger image that has both. There is no generic way to concatenate files in a meaningful way.



  • Yes that was what I had in mind last night but it was kind of late so I went with the "simpler" solution. BTW I already warned him about merging files that contain format headers, this is applicable ONLY for raw data.

    For specific formats the concatenation must also be format specific, you must know how to parse that file, read in header tags and process them and only merge the raw data.

    I was also thinking about pre-allocating the output file to the size of both inputs so that fragmentation can be avoided.

    Since we are now talking BIG files, it would be wise to abort the operation early on in case there is not enough free space on disk, not merge the files all the way to 90% and fail just then, when time has been waster and the entire disk has been filled, which may have other negative implications.



  • What is a file with format headers?

    I have tried some operations with .7z files and seems to work. The files I intend to concatenate are fairly small, so I guess I should not worry about memory?



  • Well, specific file formats typically have a header, which defines properties, such as bitrate, color depth, resolution, after which follows the raw data.

    For example, if you want to join two images together, you have to read the resolution and color depth, figure out how you want to compose the two input images into the output image, do some conversion if the data is in a different format and so on.

    Take a look here for example, this is a wikipedia article on BMP image format and its header:
    http://en.wikipedia.org/wiki/BMP_file_format#Bitmap_file_header


  • Moderators

    Never assume that the user of your application will only ever use it for what you anticipate!

    If your application can concatenate two files some user will end up trying to concatenate two blueray films with it;-)



  • Aha, okay I get it now thanks. What about .zip files, that's what I am mostly interested about...

    Can I zip a file through my application? For example, there is a button for file creation, then another one checking if the file really exists and then a button called add to archive, where I put the file in question, in a .7z, .rar archive.



  • [quote author="Tobias Hunger" date="1329983973"]Never assume that the user of your application will only ever use it for what you anticipate!

    If your application can concatenate two files some user will end up trying to concatenate two blueray films with it;-)[/quote]

    Hehe, nice one indeed! The thing is that the user will not be able to select the files for concatenation. It would be predetermined in the application code, so that is not a big issue.



  • Well, it is not like something REALLY bad will happen, worst case scenario the procedure will fail in numerous possible ways. Luckily no one's foot will be literally shot off :)



  • How about the zipping files question? Any info on that one? :)



  • [quote author="croussou" date="1329984083"]Aha, okay I get it now thanks. What about .zip files, that's what I am mostly interested about...

    Can I zip a file through my application? For example, there is a button for file creation, then another one checking if the file really exists and then a button called add to archive, where I put the file in question, in a .7z, .rar archive.[/quote]

    No, it will not work with archives such as zip or 7z. Archives have headers as well. Those headers describe which files or folders are in the archive, and where their data can be found inside the file.



  • According to "wikipedia":http://en.wikipedia.org/wiki/Zip_(file_format)#File_headers, the .zip file format DOES feature a header that contains plenty of information:

    bq. 0 4 Local file header signature = 0x04034b50 (read as a little-endian number)
     4 2 Version needed to extract (minimum)
     6 2 General purpose bit flag
     8 2 Compression method
    10 2 File last modification time
    12 2 File last modification date
    14 4 CRC-32
    18 4 Compressed size
    22 4 Uncompressed size
    26 2 File name length (n)
    28 2 Extra field length (m)
    30 n File name

    It is pretty much safe to assume if you use this method to join two zip files you will get what MIGHT appear as a good file, but IT WILL BE CORRUPTED.



  • It was mostly a general question, if there is any way for an application to provide files in an archive :)
    Not about concatenation itself...just the zipping process.



  • You have no option but use the archive format API to merge two archives together.



  • [quote author="ddriver" date="1329984668"]You have no option but use the archive format API to merge two archives together.[/quote]

    I don't want to merge :) I just want to save a file that I read from my application in a .zip format. For example a text file, I want to be able to add it in an archive on the desktop through the app that is.



  • [quote author="croussou" date="1329984584"]It was mostly a general question, if there is any way for an application to provide files in an archive :)
    Not about concatenation itself...just the zipping process.[/quote]
    Grmbl!
    Then please, why do you only say that now? Why do you send us down a different road, trying to answer the question you asked instead of the question you have? Please spend a bit more time formulating your actual question in the future; describe what you want to achieve, not how you think you might achieve it. That will make us spend our time more efficiently.

    Anyway, working with ZIP files is a "FAQ":http://developer.qt.nokia.com/faq/answer/how_to_compress_data_with_qt on this site. There is "plenty":http://developer.qt.nokia.com/search?search=zip to be found on this site on this topic.



  • Again, you have to use the format API. Or reimplement everything yourself which is not really applicable.

    Luckily most archive formats have an open API so they can easily be integrated.



  • Well, I certainly apologize for misleading you. I was just trying to explain what I want to achieve...obviously did not come out in the right way.



  • Relax, we are all flawed by design - that is what life is for, polishing those flaws ;)

    Take a look here:
    http://www.7-zip.org/sdk.html

    bq. LZMA SDK includes:
    C++ source code for .7z compression and decompression (reduced version)

    This is no longer Qt territory, but you can easily integrate the 7zip library and use it in your Qt project.



  • Thank you.

    Looks interesting indeed. I will have a look, although I am not sure that I can use it that easily :)



  • You cannot expect others to write all your code for you, you have to learn it for yourself, and it does take time. Even if your get all the code snippets from others, without the proper skills you will not be able to put them together in a working program. Rome wasn't built overnight you know ;)



  • Definitely, no doubt about that. It's just the best way for me to learn is by reading theory but mostly examples. Then you can apply the similar functionality to your application :)



  • A final word of advice, don't think, not even for a moment, that you will be able to avoid the hard work of digging through the documentation, all the examples you see cover at best the very basics, learning to find what you need in the DOC is MANDATORY if you really intend to become a programmer.

    Good luck.



  • Thanks for all the help you provided, much appreciated.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.