Implement something like tar archive in windows environment

[Zgembo]

Hi all,
I am creating a Qt module for copying (archiving) some data files. Let’s say that I am exporting some files from database and some files from file system (on one PC). What I need is a way to group these files for an easier transfer and manipulation (something like tar archine on Linux systems), because I need to import them on another PC. Does anyone have any idea how to implement something like tar archive on windows or any other way of grouping files?

Tnx.

[sierdzio]

Just use zip with small (or even zero) compression. Or, you can use tar itself, too - works on Windows just like on unixes. Look for gtar libs, for example.

[SGaist]

Hi and welcome to devnet,

To add to @sierdzio, KDE's KArchive module looks like a good candidate for your use case.

[Zgembo]

Thank you both for your suggestions. I will investigate this approach.

[Zgembo]

@SGaist Hi, can you give me some guidance on how to include KArchive modul into windows aplication enviroment? I am using visual studio 2017 with Qt 5.9.4.

Thank you

[SGaist]

Did you already build the module itself ?

[Zgembo]

@SGaist No, I have just downloaded code from git, https://github.com/KDE/karchive but I do not know how to build it and include into my project.

[aha_1980]

@Zgembo

you need CMake and then follow the instructions here: https://github.com/KDE/karchive/blob/master/INSTALL

[Zgembo]

I have found this tutorial online. It works as a charm.
http://3adly.blogspot.com/2011/06/qt-folder-compression.html
If you do not need compression, as I do not, you can just add second argument to qCompression function 0-means no compression.

Tnx,
Zgembo

[VRonin]

@Zgembo said in Implement something like tar archive in windows environment:

No, I have just downloaded code from git, https://github.com/KDE/karchive but I do not know how to build it and include into my project.

https://forum.qt.io/post/377923

[Zgembo]

@Zgembo Can someone look at the code on this link. When I try to compress larger files, few GB nothing happens.
http://3adly.blogspot.com/2011/06/qt-folder-compression.html

[VRonin]

Yes, that post uses qCompress(file.readAll()); so it has to load the entire file in memory and then uses a non-zip-compliant compression.

My suggestion stands: use KArchive. You can find a pre-built version (updated as of 2018-12-14, will be deleted in 6months time) with all dependencies in the following links:

• MSVC 2015 64 bit
• MSVC 2015 32 bit
• MinGW 32 bit (does not support bzip2)

[Zgembo]

@VRonin said in Implement something like tar archive in windows environment:

Yes, that post uses qCompress(file.readAll()); so it has to load the entire file in memory and then uses a non-zip-compliant compression.

My suggestion stands: use KArchive. You can find a pre-built version (updated as of 2018-12-14, will be deleted in 6months time) with all dependencies in the following links:

• MSVC 2015 64 bit
• MSVC 2015 32 bit
• MinGW 32 bit (does not support bzip2)

Can I use it with MS Visual studio 2017, and if yes how can I include it in my project? Thank you.

[SGaist]

Yes you can. You have to build the module with VS2017 first.

As for how to include it in your project, it's shown on the bottom left of the module documentation.

[VRonin]

@SGaist said in Implement something like tar archive in windows environment:

You have to build the module with VS2017 first.

AFAIK MSVC2015 and MSVC2017 are binary compatible (for the first time ever in MSVC history) so you could try downloading and using the 2015 pre-compiled and go from there.

@Zgembo said in Implement something like tar archive in windows environment:

how can I include it in my project?

http://doc.qt.io/qt-5/third-party-libraries.html
http://doc.qt.io/qtcreator/creator-project-qmake-libraries.html

[Zgembo]

@SGaist Hi, I have tried to build KArchive with Cmake-GUI, but it looks like I am mission extra cmake modules (ECM). I will try @VRonin suggestion to use prebuild libraries from his link. Thank you.

[Zgembo]

@VRonin Hi, I have tried to add karchive libraries that I have downloaded from your link, but with out any success. I have used this link as a tutorial https://stackoverflow.com/questions/35008426/link-3rd-party-library-in-visual-studio .
When I try to compile my project I get the error: Cannot open include file: qplatformdefs.h: no such file or directory.

[VRonin]

@Zgembo said in Implement something like tar archive in windows environment:

qplatformdefs.h

That's an additional include.

In Project > Properties > C/C++->General > Additional Include Directories
Click Edit, and enter the path to the directory where the file "qplatformdefs.h" is located

It is in the mkspecs folder of qt, something like: C:\Qt\5.12\msvc2015_64\mkspecs\win32-msvc

[Zgembo]

@VRonin it compiles. Now I have to find some examples of how to use just KArchive to create archive file without any compression.

[VRonin]

from KArchive\examples\tarlocalfiles

/*  This file is part of the KDE project
    Copyright (C) 2013 Maarten De Meyer <de.meyer.maarten@gmail.com>
    You may use this file under the terms of the BSD license as follows:
    Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions
    are met:
    1. Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    2. Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
    IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
    OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
    IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
    INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
    NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
    THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
 * TarLocalFiles
 * This example shows how to add local files and directories to a KArchive
 *
 * api: addLocalFile(fileName, destName)
 * api: addLocalDirectory(dirName, destName)
 *
 * Usage: ./tarlocalfiles <file-1> <file-n>
*/
#include <QCoreApplication>
#include <QDir>
#include <QFileInfo>
#include <ktar.h>
int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);
    QStringList files(app.arguments());
    // Create or open an archive
    KTar archive(QStringLiteral("myFiles.tar.gz"));
    // Prepare the archive for writing.
    if (!archive.open(QIODevice::WriteOnly)) {
        // Failed to open file.
        return 1;
    }
    if (files.size() <= 1) {
        // No files given.
        qWarning("Usage: ./tarlocalfiles <file>");
        return 1;
    }
    for (int i = 1; i < files.size(); ++i) {
        QFileInfo localFileOrDir(files.at(i));
        if (localFileOrDir.isFile()) {
            QString name = localFileOrDir.fileName();
            archive.addLocalFile(name, name);
        } else if (localFileOrDir.isDir()) {
            QString name = QDir(files.at(i)).dirName();
            // Add this folder and all its contents
            archive.addLocalDirectory(name, name);
        }
    }
    archive.close();
    return 0;
}

If you want to compress it see KArchive\examples\bzip2gzip

[Zgembo]

@VRonin Thank you for the example. I do not need to compress files. I just need to group few files to single archive (no compression) and to move it to another pc.

[jsulm]

@Zgembo Take a look at documentation: https://api.kde.org/frameworks/karchive/html/classKTar.html
Especially "A class for reading / writing (optionally compressed) tar archives."

[Zgembo]

@jsulm said in Implement something like tar archive in windows environment:

@Zgembo Take a look at documentation: https://api.kde.org/frameworks/karchive/html/classKTar.html
Especially "A class for reading / writing (optionally compressed) tar archives."

Thank you for your replay. I am really stuck with this. I have examined this example https://github.com/KDE/karchive/blob/master/examples/tarlocalfiles/main.cpp but when I run this program it asks for bzip2.dll and two more.
I really do not need to compress files. There is no example of using just KTar without compression.

[JonB]

@Zgembo

KTar allows you to read and write tar archives, including those that are compressed using gzip, bzip2 or xz.

You may not need to use compression, but that indicates that because the code supports compression you still need to provide the extra DLLs it is asking for. Their code will not get used in your case, but they still need to be present at runtime.

[VRonin]

@Zgembo said in Implement something like tar archive in windows environment:

this program it asks for bzip2.dll and two more.

This is because KArchive can do much more than tar and my pre-compiled version allows you to do that linking to those libraries. They are available in the same link from which you downloaded the KArchive library.
Linking to those libraries does not mean your tar archive is getting compressed. it just means you have the option to compress it

[Zgembo]

@VRonin said in Implement something like tar archive in windows environment:

@Zgembo said in Implement something like tar archive in windows environment:

this program it asks for bzip2.dll and two more.

This is because KArchive can do much more than tar and my pre-compiled version allows you to do that linking to those libraries. They are available in the same link from which you downloaded the KArchive library.
Linking to those libraries does not mean your tar archive is getting compressed. it just means you have the option to compress it

Thank you @VRonin . I have managed to create, test and run my test program in qt creator. I have managed to create a tar archive. Now I am trying to figure out how to unpack this archive file from my program.

[VRonin]

@Zgembo said in Implement something like tar archive in windows environment:

Now I am trying to figure out how to unpack this archive file from my program.

That library uses the same interface for all "archives" so you can use the unzip example to extract your tar archive just by changing KZip to KTar

[Zgembo]

@VRonin said in Implement something like tar archive in windows environment:

@Zgembo said in Implement something like tar archive in windows environment:

Now I am trying to figure out how to unpack this archive file from my program.

That library uses the same interface for all "archives" so you can use the unzip example to extract your tar archive just by changing KZip to KTar

Thank you. Figured it out already :-)

[Zgembo]

I have moved my project from Qt Creator to Visual studio. I can compile the project in debug mode, but when I run it I get an error
"The application was unable to start correctly (0xc000007b). Click OK to close the application"

I have added lib path to Project->Properties->Linker->Additional library directories
and I have added all required dll files to Linker->Input section (bzip2.lib, liblzma.lib, zlib.lib, KF5Archive.lib, KF5Archived.lib)

I have added include path to Project->Properties->C/C++/General->Additional include directories

[VRonin]

That error usually comes up when you mix 32 and 64 bits version of the libraries.

[Zgembo]

@VRonin Yes you are right. I have copied wrong dll to debug folder, so on runtime my app tried to open these 32-bit dlls.
Where is the best way to put these .dlls for production enviroment? Should I copy all these dlls to c:\windows\system32?

[jsulm]

@Zgembo said in Implement something like tar archive in windows environment:

Should I copy all these dlls to c:\windows\system32

Please not! :-)
Just put them in same directory where you have your executable.

[Zgembo]

@jsulm OK, thank you. What about my include files? Now I have my KArchive folder that have "include" and "libs" subfolders in my C:\Downloads folder. I have included this two folders with their absolute path into my project (visual studio project).
Is there a better way to do this, because this is a large project that have a lot of developers.

[VRonin]

@Zgembo said in Implement something like tar archive in windows environment:

because this is a large project that have a lot of developers.

Every project will have its own convention. just ask how you guys handle 3rd party libraries locations

[Zgembo]

OK, thank you. My test program now works in debug mode. I am having problems compiling program in RELEASE mode. For some reason it can not see qplatformdefs.h . I have double checked that path to that file is included as @VRonin stated in some of the previous posts.

[VRonin]

Visual studio uses different "Additional library directories" and "Additional include directories" for different platforms (x86 and x64) and configurations (Debug/Release) so every time you have to set the same thing multiple times for each of them using the comboboxes in the top part of the properties widow of the project

[Zgembo]

Figured it out. Everything works now. Thank you.

[SGaist]

IIRC, qmake can generate Visual Studio Solutions that should handle that for you. Worth a try at least.

[Zgembo]

I have moved everything to my main project and it works. Now I can tar and untar files form my main project.

Right now I am trying to figure out best way to indicate to a user that archiving or unarchiving process is in progress. I was thinking of using qprogressbar but I do not see a way to send any signal from KArchive function to qprogressbar.
Lets say that I have for example three large files to unarchive (few gigabytes). What would be the best way to somehow indicate to a user what is going on?

[mrjj]

@Zgembo
In cases where you cannot know the length of the process/get status,
you can use a QLabel with a animated gif.
like