Converting QByteArray to QString

Sina Ranjkesh zade

Hi,
when I convert a QByteArray to QString, the length changes. I want to run a python script via QProcess. To do that, I should change QByteArray to QString and append it to a Qlist of QStrings.
But I lose some characters in this conversion.
These are the lengths:
QByteArray length: 3686400
QString length: 2805
The QByteArray variable was a opencv::mat variable converted to QByteArray without any problems.
Any help will be appreciated.

JonB

@Sina-Ranjkesh-zade
There is something wrong if 3.5MB of bytes is only 2.8K of string characters! So it depends what is in your bytes.

Are they indeed string characters.
Show you code for converting QByteArray to QString.

Actually, better, Google for qbytearray to qstring. Lots of ways, e.g. read through https://stackoverflow.com/questions/14131127/qbytearray-to-qstring.

Sina Ranjkesh zade

This is my function:

QByteArray PRID::dataProcess(QByteArray data, NkMetadataBag &metadata)
{

    auto path = QString::fromLocal8Bit(qgetenv(NK_HOME_PATH));

    QProcess p;
    QStringList params;
    qDebug() << "d data in processor: " << data.length() << endl;
    qDebug() << "data in processor: " << QString::fromUtf8(data.data(),data.size()).length() << endl;
    
    params << path + "/" + _scriptName + ".py" << QString::fromUtf8(data.data(),data.size());

    p.start("python", params);
    p.waitForFinished(-1);
    QString p_stdout = QString::fromLocal8Bit(p.readAll());
    p_stdout = p_stdout.remove("\r\n");
    return p_stdout.toLocal8Bit();
}

As you can see, I get an input as "data" which is QByteArray. I get this result in application output:

d data in processor: 3686400
data in processor: 3674351

But in python when I get input arguments using "sys.argv" command, it's length is 2805.
I forgot to say that I'm working on Ubuntu 20.04.

Christian Ehrlicher

As already told you on so - make sure there is no \0 char in your string - otherwise it might get ignored either by QString::fromLocal8Bit() (which is still wrong when you use utf8 in python) or by python arguments parser.

Sina Ranjkesh zade

@Christian-Ehrlicher
What should I do with them?
I searched the QString and it seems there isn't any \0.
Do you know how can I find them?
Each of the elements of QByteArray was a uint8 number.

SGaist

Hi,

Why do you want to dump a blob of binary data as argument to a Python script ?

ChrisW67

@Sina-Ranjkesh-zade said in Converting QByteArray to QString:

Each of the elements of QByteArray was a uint8 number.

The first one of these 3686400 bytes in the input QByteArray that is zero will terminate the string conversion. The zero byte is in the input, not the output QString.

What you are trying to do makes little sense. Even if you got a meaningful 3686400 character QString, and that's doubtful IMHO, I am fairly confident no operating system will accept such a long command line.

Seems far more likely that the python script is expecting binary data in a file, and the file name as an argument.

Sina Ranjkesh zade

@ChrisW67
Thank's for your response.
I want to pass an image from Qt to python. The image is high-resolution (1920*1080). I've converted the image data to QByteArray. To overcome the /0 problem, I convert to Base64 coding.
As you said, when I want to send it as an argument to python code, I got this error:
"execvp: Argument list too long"
(When I convert to Base64, data size is 4915200).
Do you have any suggestions on how to solve this problem?
(I'm working with video frames and I should do that for each of the frames. So the speed is important too)
Any help will be appreciated.

ChrisW67

What is the python program expecting, or are you writing that also?

Edit:
You certainly do not want to be starting a new python process for every frame of video (assuming 25 or 30 FPS) . Start one process and feed it frames through its standard input or perhaps a shared memory block (QSharedMemory)

Sina Ranjkesh zade

@ChrisW67
I'm writing that too.
For "standard input" if you mean input arguments of the process, I got this error:
"execvp: Argument list too long".
I also tried QSharedMemory too. I wrote data to memory in Qt successfully but I cannot read it from memory in python.

JonB

@Sina-Ranjkesh-zade said in Converting QByteArray to QString:

"execvp: Argument list too long".

As @ChrisW67 observed earlier

Even if you got a meaningful 3686400 character QString, and that's doubtful IMHO, I am fairly confident no operating system will accept such a long command line.

You cannot pass this sort of size literal string or byte-blob on the command-line to a process. And as @ChrisW67 also said, you will want just one external process (if at all) and pass data to it via shared memory or sockets or similar.

I do not know whether a QSharedMemory object is supposed to be sharable with a non-Qt process. Though I would have thought it was. Can't say anything about what might happen from Python accessing it, check out documentation.

SGaist

As was already asked, can you explain what exactly does your Python script expects as input ?

How does it handle it ?

ChrisW67

@Sina-Ranjkesh-zade said in Converting QByteArray to QString:

For "standard input" if you mean input arguments of the process, I got this error:

No, I meant standard input. That is , the python program reads from a terminal and accepts input just as if you typed it (except it is the Qt program sending that input). The Qt program can send a command to the python program, send the data it needs, and read the result (if there is one) back on the python program's standard output. Or you can used shared memory, or a socket, or files, or do whatever the python program is doing in the Qt program...