Shell commands on Android

Ferni

Hi!

I'm using QProcess to emulate linux shell and I'm facing some problems about missing commands. I started a hello world application and tried rm and gzip commands. They worked well but when I try to run tar it cannot be executed.

Is it possible to use TAR on Android? Should I do something different? I found that there is something called BusyBox, that includes many commands but I don't know if is possible to embedded that in my application or if I should install that as stand alone application.

    QString temp = QStandardPaths::writableLocation(QStandardPaths::CacheLocation);
    QString perm = QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);
    QString input = ":/test.txt.gz";
    QString output1 = temp + "/test.txt.gz";
    QString output2 = temp + "/test.txt";

    qDebug()<< "TEMP:" << temp;
    qDebug()<< "PERM:" << perm;

    QFile test(output2);
    qDebug()<<"EXIST? "<< test.exists();

    qDebug()<<QDir(temp).entryList();

    QFile::copy(input,output1);
    qDebug()<<"EXIST? "<< test.exists();

    qDebug()<<QDir(temp).entryList();

    QString program = "gzip";
    QStringList arguments;
    arguments << "-d" << output1;

    QProcess *myProcess = new QProcess();
    myProcess->start(program, arguments);
    bool finished = myProcess->waitForFinished();

    qDebug()<<"EXIST? "<< test.exists();
    qDebug()<<"END:" << finished <<QDir(temp).entryList();

    test.open(QIODevice::ReadOnly);
    qDebug()<< "OUTPUT:"<< test.readAll();


    input = ":/testFolder";
    output1 =temp +"/testFolder";

    qDebug()<<"EXIST DIR:" << QDir(input).exists();
    if(!QDir(output1).exists()){
        QDir().mkdir(output1);
    }
    foreach (QFileInfo var, QDir(input).entryInfoList()) {
        qDebug()<<"INPUT" <<input + "/" + var.fileName();
        qDebug()<<"OUTPUT"<<output1 + "/" + var.fileName();
        qDebug() << "Copy result:" << QFile::copy(input + "/" + var.fileName(),output1 + "/" + var.fileName());
    }
    qDebug()<<QDir(output1).entryList();

//here comes the problem :/
    program = "tar-zcvf";
    arguments.clear();
    arguments << output1;

    myProcess->setWorkingDirectory(temp);
    myProcess->start(program, arguments);
    finished = myProcess->waitForFinished();

    qDebug()<<"END: "<< finished << "OUTPUT" << QDir(temp).entryList();
    qDebug()<<myProcess->readAll();

Thanks!
Fernando

jsulm

@Ferni Did you check whether gzip command is available on Android?
You should connect http://doc.qt.io/qt-5/qprocess.html#errorOccurred and http://doc.qt.io/qt-5/qprocess.html#readyReadStandardError to slots and print out what you get there.

Ferni

@jsulm I could not find where are all the available shell commands for Android. About gzip, I'm sure that it's working because I could test it. The problem is (in this case) TAR command. I just need some basic commands like tar, zip, unzip.
I tried QuaZip but some part of the code is implemented with QProcess and it would be great if I can run the same lines on Android.

jsulm

@Ferni Did you check errorOccured and readyReadStandardError signals as I suggested?

Ferni

No I didn't. I just have a Qt Quick Application and only have my main.cpp
I wanted to test this in a minimal example and then continue with my real application.

jsulm

@Ferni It's not much work and could tell you what the problem is...

Ferni

I added a simple class with some slots to test what you suggest and I get this:

D libQGuiApplication.so: ../signalslot.cpp:23 (void SignalSlot::readyStandardError()): "tar: "
D libQGuiApplication.so: ../signalslot.cpp:24 (void SignalSlot::readyStandardError()): ""
D libQGuiApplication.so: ../signalslot.cpp:23 (void SignalSlot::readyStandardError()): "chown 1000:1000 'testFile.txt'"
D libQGuiApplication.so: ../signalslot.cpp:24 (void SignalSlot::readyStandardError()): ""
D libQGuiApplication.so: ../signalslot.cpp:23 (void SignalSlot::readyStandardError()): ": Operation not permitted"
D libQGuiApplication.so: ../signalslot.cpp:24 (void SignalSlot::readyStandardError()): ""
D libQGuiApplication.so: ../signalslot.cpp:23 (void SignalSlot::readyStandardError()): "\n"
D libQGuiApplication.so: ../signalslot.cpp:24 (void SignalSlot::readyStandardError()): ""

The errorOccurred signal was not triggered and QProcess finish ok

D libQGuiApplication.so: ../signalslot.cpp:61 (void SignalSlot::finished(int, QProcess::ExitStatus)): ExitCode:  1 ExitStatus:  QProcess::ExitStatus(NormalExit)

When I tried to create a TAR file, it seems to do it well but the problem comes when I want to unpack it.
I'm using this to pack things

tar -cvf output.tar filename

And this to unpack

tar -xvf output.tar

Any ideas?

SGaist

Hi,

You might be trying to untar in a read-only location.

Ferni

Hi @SGaist
I'm working in QStandardPaths::writableLocation(QStandardPaths::DataLocation) and I can create a tar.gz file. The problem comes when I want to unpack the file that I created.

SGaist

Wait... Is it tar that is failing or chown in this case ?

Ferni

How can I check that? I'm only using QProcess with these parameters

QString program = "tar";
QStringList args;
args << "-xvf" << outputTar;
proc->execute(program, args);

where proc is my QProcess and outputTar is my tar file(full path) made by QProcess with "-cvf" as arguments

Why can I create a tar file but not extract it? I tried also with a tar file created in my PC but the result is the same. I found that I have to set the working directory in QProcess and I set

proc->setWorkingDirectory(dir);

where dir is QStandardPaths::writableLocation(QStandardPaths::DataLocation), same location where tar file was created and where I want to unpack it.

EDIT:
@SGaist
I change the default location to QStandardPaths::writableLocation(QStandardPaths::DownloadLocation) and I found out that my tar file is empty. I tested my code in Linux and the only difference is the path that I use as working directory. Is it even possible to use TAR command in Android? Where can I find a list with the usable commands?

SGaist

Are you only interested in creating files archives ? If so you should rather take a look at KArchive.

AFAIK, Android doesn't use any standard Linux shell and even so, it might be different from one version to the other.

Ferni

Yeap, that would be my option. The problem is that I already have some code that was thought for Linux . If I want to implement that with a library I should process each command and its arguments and then do something to similar to the command with the specific arguments.

I found this and it seems that there is no TAR command so I will close this post :(

Also, I found a new approach. I compiled BusyBox and running commands with adb shell is working perfectly but I cannot make it work within my Qt application. I'm trying with these lines from java but I'm not sure if it's not working because the executable doesn't have the necessary permissions or if it's not even called. Could you point me where to look to run external executables from a Qt application? Using BusyBox will solve all my implementations problems

            String myExec = "/data/local/tmp/busybox touch /sdcard/fromBB.txt";
            Process process = Runtime.getRuntime().exec(myExec);

Ferni

@SGaist
Sorry to bother you but I decided to make a mixed solution: normal commands will run with console and compressed files will be run with a 3rd party library.

I tried to use KArchive as you suggested but I'm stuck, I can't even compile it for my pc (linux). Could you point me where to find a tutorial? As the library is written in Qt I was expected a .pro file but I found a CMakeList instedad :/

SGaist

You need one additional thing: the extra-cmake-modules

Use -DCMAKE_PREFIX_PATH=path/to/qtbase/lib/cmake/ to tell cmake where to find your Qt version and the do the same for karchive.

Ferni

Hey @SGaist thanks! Just adding that line solve the problem. I compiled and I'm testing the library. Do you know what are the changes that I need to compile KArchive for Android? I tried to make a .pro with the sources but I got a lot of missing files errors and I'm kind of lost with CMake.

Ferni

Oks... some missing paths for Android. After I added them I got a CMake error:

/usr/share/cmake-3.2/Modules/CMakeTestCXXCompiler.cmake:54: error: The C++ compiler "/opt/android-ndk-r12b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc-4.9.x" is not able to compile a simple test program. It fails with the following output: Change Dir: /home/ferni/QtBuild/QtKArchive/output/android/CMakeFiles/CMakeTmp

I run the standalone toolchain script from NDK and using it could run CMake.

/opt/android-ext/bin/arm-linux-androideabi-gcc

The problem now comes when I build. I left here the compile output

15:56:11: Running steps for project KArchive...
15:56:11: Persisting CMake state...
15:56:13: Starting: "/usr/bin/cmake" --build . --target all -- -j8
Scanning dependencies of target KF5Archive_automoc
[  7%] Automatic moc for target KF5Archive
[  7%] Built target KF5Archive_automoc
Scanning dependencies of target KF5Archive
[ 14%] [ 21%] [ 28%] [ 35%] Building CXX object src/CMakeFiles/KF5Archive.dir/kar.cpp.o
Building CXX object src/CMakeFiles/KF5Archive.dir/kcompressiondevice.cpp.o
[ 42%] Building CXX object src/CMakeFiles/KF5Archive.dir/karchive.cpp.o
Building CXX object src/CMakeFiles/KF5Archive.dir/kfilterbase.cpp.o
Building CXX object src/CMakeFiles/KF5Archive.dir/kfilterdev.cpp.o
[ 50%] [ 57%] [ 64%] Building CXX object src/CMakeFiles/KF5Archive.dir/klimitediodevice.cpp.o
Building CXX object src/CMakeFiles/KF5Archive.dir/knonefilter.cpp.o
Building CXX object src/CMakeFiles/KF5Archive.dir/kgzipfilter.cpp.o
[ 71%] Building CXX object src/CMakeFiles/KF5Archive.dir/ktar.cpp.o
[ 78%] Building CXX object src/CMakeFiles/KF5Archive.dir/kzip.cpp.o
[ 85%] Building CXX object src/CMakeFiles/KF5Archive.dir/krcc.cpp.o
[ 92%] Building CXX object src/CMakeFiles/KF5Archive.dir/loggingcategory.cpp.o
[100%] Building CXX object src/CMakeFiles/KF5Archive.dir/KF5Archive_automoc.cpp.o
Linking CXX shared library libKF5Archive.so
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::KArchive(QString const&): error: undefined reference to 'operator new(unsigned int)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::KArchive(QIODevice*): error: undefined reference to 'operator new(unsigned int)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::~KArchive(): error: undefined reference to 'operator delete(void*)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::~KArchive(): error: undefined reference to 'operator delete(void*)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::~KArchive(): error: undefined reference to 'operator delete(void*)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::createDevice(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator new(unsigned int)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchive::createDevice(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator new(unsigned int)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchiveEntry::~KArchiveEntry(): error: undefined reference to 'operator delete(void*)'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchiveDirectory::copyTo(QString const&, bool) const: error: undefined reference to '__dynamic_cast'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:function KArchiveDirectory::copyTo(QString const&, bool) const: error: undefined reference to '__dynamic_cast'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:vtable for KArchive: error: undefined reference to '__cxa_pure_virtual'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:vtable for KArchive: error: undefined reference to '__cxa_pure_virtual'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:vtable for KArchive: error: undefined reference to '__cxa_pure_virtual'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:vtable for KArchive: error: undefined reference to '__cxa_pure_virtual'
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:typeinfo for KArchiveDirectory: error: undefined reference to 'vtable for __cxxabiv1::__si_class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:typeinfo for KArchiveFile: error: undefined reference to 'vtable for __cxxabiv1::__si_class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:typeinfo for KArchiveEntry: error: undefined reference to 'vtable for __cxxabiv1::__class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/karchive.cpp.o:karchive.cpp:typeinfo for KArchive: error: undefined reference to 'vtable for __cxxabiv1::__class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:function KAr::openArchive(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator delete[](void*)'
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:function KAr::openArchive(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator delete[](void*)'
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:function KAr::openArchive(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator delete[](void*)'
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:function KAr::openArchive(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator new[](unsigned int)'
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:function KAr::openArchive(QFlags<QIODevice::OpenModeFlag>): error: undefined reference to 'operator delete[](void*)'
CMakeFiles/KF5Archive.dir/kar.cpp.o:kar.cpp:typeinfo for KAr: error: undefined reference to 'vtable for __cxxabiv1::__si_class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/kcompressiondevice.cpp.o:kcompressiondevice.cpp:typeinfo for KCompressionDevice: error: undefined reference to 'vtable for __cxxabiv1::__si_class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/kfilterbase.cpp.o:kfilterbase.cpp:typeinfo for KFilterBase: error: undefined reference to 'vtable for __cxxabiv1::__class_type_info'
/opt/android-ext3/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: the vtable symbol may be undefined because the class is missing its key function (see go/missingkeymethod)
CMakeFiles/KF5Archive.dir/kzip.cpp.o:kzip.cpp:function KZip::closeArchive(): error: undefined reference to 'operator new[](unsigned int)'
CMakeFiles/KF5Archive.dir/kzip.cpp.o:kzip.cpp:function KZip::doPrepareWriting(QString const&, QString const&, QString const&, long long, unsigned short, QDateTime const&, QDateTime const&, QDateTime const&): error: undefined reference to 'operator new[](unsigned int)'
CMakeFiles/KF5Archive.dir/loggingcategory.cpp.o:loggingcategory.cpp:function KArchiveLog(): error: undefined reference to '__cxa_guard_acquire'
CMakeFiles/KF5Archive.dir/loggingcategory.cpp.o:loggingcategory.cpp:function KArchiveLog(): error: undefined reference to '__cxa_guard_release'
collect2: error: ld returned 1 exit status
make[2]: *** [src/libKF5Archive.so.5.33.0] Error 1
make[1]: *** [src/CMakeFiles/KF5Archive.dir/all] Error 2
make: *** [all] Error 2
15:56:16: The process "/usr/bin/cmake" exited with code 2.
Error while building/deploying project KArchive (kit: Android)
When executing step "Make"
15:56:16: Elapsed time: 00:05.

What am I doing wrong?

SGaist

Android will require a bit of work. I'd start here: https://community.kde.org/Android

Ferni

I could finally compile KArchive for Android. I just had to make a .pro with the right files and include the sources to my project. I left it here! Don't forget to add config-compression.h, loggingcategory.h and karchive_export.h, some files need those headers.

I have tested for ZIP and TAR under Linux and Android and it's working perfectly!!

TARGET = KArchive
TEMPLATE = lib
QT       -= gui

INCLUDEPATH += ../../include

SOURCES += \
    ../../src/karchive.cpp \
    ../../src/kar.cpp \
    ../../src/kcompressiondevice.cpp \
    ../../src/kfilterbase.cpp \
    ../../src/kfilterdev.cpp \
    ../../src/kgzipfilter.cpp \
    ../../src/klimitediodevice.cpp \
    ../../src/knonefilter.cpp \
    ../../src/ktar.cpp \
    ../../src/kzip.cpp
    ../../src/krcc.cpp \

HEADERS += \
    ../../include/karchive.h \
    ../../include/kar.h \
    ../../include/kcompressiondevice.h \
    ../../include/kfilterbase.h \
    ../../include/kfilterdev.h \
    ../../include/kgzipfilter.h \
    ../../include/klimitediodevice_p.h \
    ../../include/knonefilter.h \
    ../../include/ktar.h \
    ../../include/kzip.h
    ../../include/krcc.h \