Error reading QByteArray

carpajr

Hello!

I'm trying to read data generated by a serial device, but I'm observing an issue that I did not find a way to avoid/solve.

I'll illustrate with one example:

// 02 00 35 06 00 00 00 03 40
static const char data[] = {
       '\x02', '\x00', '\x35', '\x06', '\x00', '\x00', '\x00', '\x03', '\x40'
};

   QByteArray test1;
   test1.resize(9);
   test1[0] = 0x02;
   test1[1] = 0x00;
   test1[2] = 0x35;
   test1[3] = 0x06;
   test1[4] = 0x00;
   test1[5] = 0x00;
   test1[6] = 0x00;
   test1[7] = 0x03;
   test1[8] = 0x40;

QByteArray test2 = QByteArray::fromRawData(data, sizeof(data));

qDebug() << test1.toHex() << test1[2];
qDebug() << test2.toHex() << test2[2];

"\x02\x00""5\x06\x00\x00\x00\x03" 5
"\x02\x00""5\x06\x00\x00\x00\x03" 5

Why '""5' and not \35 or 5 and not 35? What I have to consider to the charset be converted correctly?

Thanks!

SGaist

Hi,

How are you getting these readouts ?

carpajr

@SGaist said in Error reading QByteArray:

How are you getting these readouts ?

Using this:

 QByteArray l_receivedData{};
 while(m_serialPort->bytesAvailable()) {
   m_serialPort->waitForBytesWritten(READ_TIMEOUT);
   l_receivedData.append(m_serialPort->read(1));
 }

SGaist

My question was rather: how did you determine that you had the wrong content ?

Out of curiosity, why not use the port asynchronously ? And why read one char at a time ?

carpajr

@SGaist said in Error reading QByteArray:

My question was rather: how did you determine that you had the wrong content ?

Out of curiosity, why not use the port asynchronously ? And why read one char at a time ?

I need to compare the third byte to check if it is a specific status and the last to verify the checksum. Other bytes (3,4,5,6) stores information that must be interpreted bit by bit.

When I compare 3rd byte I expected 0x35, but I realized that it is only 5 even seeing that data in QByteArray is correct.

It seems that when I access the QByteArray position, some magic occurs and data is converted in an unknown way.

You are right, the data can be read asynchronously and it can fix the issue. Even so, why QByteArray has this behavior?... I'm curious to understand. :)

JonB

@carpajr
2 things.

The first one worries me. You wrote

   test1[7] = 0x03;
   test1t[8] = 0x40;

That implies to me you have not actually pasted your source code. Once that happens, I never know what to believe about anything else.

The second one. \35 is the digit 5. I wonder if the debug output format from qDebug() is trying to show you the content is as follows (I have put in a space)

"\x02\x00" "5\x06\x00\x00\x00\x03"

It's showing it as though it were two strings split at the only printable character there, the 5.

I suspect everything is as it should be, this is a qDebug() output format issue? [CLARIFICATION As in: how you understand the output you are looking at, not that it is wrong.]

hskoglund

Hi, indeed might be a qDebug output issue.

Just tested your code (I created a new, empty widgets app in Qt 5.14.1 MinGW), inserted #include "qbytearray.h" and "qdebug.h" on top of mainwindow.cpp , and then inserted your code after "ui->setupUi(this);"
(Also I changed "test1t[8] = 0x40;" to "test1[8] = 0x40;")

Running it I get:

"020035060000000340" 5
"020035060000000340" 5

Christian Ehrlicher

@hskoglund said in Error reading QByteArray:

Hi, indeed might be a qDebug output issue.

Why issue? qDebug() prints out non-printable characters as hex encoded values and printable characters as characters - so all is fine.

hskoglund

@Christian-Ehrlicher Of course, you nailed it, the actual code, the last 2 lines should be:

qDebug() << test1 << test1[2];
qDebug() << test2 << test2[2];

@jonB is correct, whenever the pasted code looks wrong...

carpajr

@JonB said in Error reading QByteArray:

The first one worries me. You wrote

Very sorry you and @hskoglund, I've just adapted the code to illustrate the issue.

Shame on me.

@JonB said in Error reading QByteArray:

I suspect everything is as it should be, this is a qDebug() output format issue?

@hskoglund said in Error reading QByteArray:

Hi, indeed might be a qDebug output issue.

Yes, could be a qDebug output issue.

@Christian-Ehrlicher said in Error reading QByteArray:

Why issue?

Could you tell us how to preserve the hex format?

The comparison is not affected.

JonB

@carpajr
I'm lost. You say your output is from

qDebug() << test1.toHex() << test1[2];

with that .toHex() (which is indeed what you want). As @hskoglund says, it looks more like the output from

qDebug() << test1 << test1[2];

without .toHex().

What are you actually showing is printed by what line of code? Can you please show output genuinely corresponding to code. I'm breaking my own rule in pursuing this!

carpajr

@JonB said in Error reading QByteArray:

What are you actually showing is printed by what line of code?

@JonB,
I used .toHex(), and the unexpected behavior was my reason to create the thread.

Christian Ehrlicher

@carpajr said in Error reading QByteArray:

and the unexpected behavior

There is nothing unexpected here. qDebug() is for debugging purposes and not for formatted output. So it tries to output in the best human-readable format it can which means it shows only hex values when there is no printable character for it.

It's event properly documented: https://doc.qt.io/qt-5/qdebug.html#operator-lt-lt-20

J.Hilk

@Christian-Ehrlicher said in Error reading QByteArray:

There is nothing unexpected here. qDebug() is for debugging purposes and not for formatted output.

I think this is unexpected. toHex is supposed to return a hex converted QByteArray, meaning 0x35 will be turned into 0x03 0x05 (doubling the size in essence)

and it does for me.
Handing over the unconverted QByteArray does indeed result in the "Most readable" qDebug output

// 02 00 35 06 00 00 00 03 40
    static const char data[] = {
           '\x02', '\x00', '\x35', '\x06', '\x00', '\x00', '\x00', '\x03', '\x40'
    };

       QByteArray test1;
       test1.resize(9);
       test1[0] = 0x02;
       test1[1] = 0x00;
       test1[2] = 0x35;
       test1[3] = 0x06;
       test1[4] = 0x00;
       test1[5] = 0x00;
       test1[6] = 0x00;
       test1[7] = 0x03;
       test1[8] = 0x40;

    QByteArray test2 = QByteArray::fromRawData(data, sizeof(data));

    qDebug() << test1.toHex() << "|" << test1 << "|" << test1[2];
    qDebug() << test2.toHex() << "|" << test2 << "|" << test2[2];

Result:

"020035060000000340" | "\x02\x00""5\x06\x00\x00\x00\x03@" | 5
"020035060000000340" | "\x02\x00""5\x06\x00\x00\x00\x03@" | 5

@carpajr what version of Qt do you use and what OS is it running on?

JonB

@carpajr said in Error reading QByteArray:

@JonB said in Error reading QByteArray:

What are you actually showing is printed by what line of code?

@JonB,
I used .toHex(), and the unexpected behavior was my reason to create the thread.

I'm sorry, but I just don't believe the output you show is from QByteArray::toHex(), that's the whole point. I believe it is direct from a QByteArray.

So far at least two other posters have shown you what the output from toHex() looks like, and you can see it does not look like what you claim. OTOH, what you show does match not using toHex(). What conclusion might one draw from that? You have not made a comment against either of these, why not? Has it not made you go look at your code again, since your results differ from everyone else's?

Can you please take the time to look closely at what @J-Hilk & @hskoglund have shown you. We know that the code you have shown is not what you actually have. Can you please look carefully at your own code in that light. If you really claim what you have shown is the case, then produce a minimal, compilable, runnable example, together with its output, which we can examine, not fragmentary bits of code & output.

It will be really annoying if it turns out that what you have been showing/claiming does not correspond to the code you paste, and in fact toHex() works fine as everybody is telling/showing you....

@Christian-Ehrlicher

There is nothing unexpected here.

There is! The "unexpected" is that the OP claims output "\x02\x00""5\x06\x00\x00\x00\x03" is produced from qDebug() << QByteArray::toHex(). Note the .toHex().

carpajr

@J-Hilk said in Error reading QByteArray:

meaning 0x35 will be turned into 0x03 0x05 (doubling the size in essence)
and it does for me.

The .hex() is ok, but not when one tries to extract the byte from qbytearray.

@J-Hilk said in Error reading QByteArray:

@carpajr what version of Qt do you use and what OS is it running on?

I'm using QT 5.11.2 GCC 64bit on Linux.

@JonB said in Error reading QByteArray:

I'm sorry, but I just don't believe ...

I had no intention to did that. Very sorry.
I got your point, I was not aware of my mistake at all.

I rewrote and share a reproducible example. The hex conversion it seems ok and again, sorry about this mess.

#include <QCoreApplication>
#include <QDebug>


int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    const char data[] = {
        '\x02', '\x00', '\x35', '\x01', '\x00', '\x01', '\x00', '\x03', '\x3c'
    };

    QByteArray test = QByteArray::fromRawData(data, sizeof(data));

    qDebug() << "\nQByteArray:" << test
             << "\nQByteArray Size:" << test.size()
             << "\nHexadecimal: " << test.toHex()
             << "\n\\x35 position:" << test.lastIndexOf(QByteArray("\x35"),0);


    // Trying to extract 3rd byte
    char byte = test.at(3);
    QString hexadecimal;
    hexadecimal.setNum(byte,16);
    char status[] = "x\35";


    qDebug() << "\n\nByte:" << test.at(3)
             << "\nByte from Local8bit:" << QString::fromLocal8Bit(&byte)
             << "\nByte from Latim1:" << QString::fromLatin1(&byte)
             << "\nByte to base 16:" << hexadecimal
             << "\n \\x35 = test.at(3)" << (*status == byte);

    return a.exec();
}


QByteArray: "\x02\x00""5\x01\x00\x01\x00\x03<" 
QByteArray Size: 9 
Hexadecimal:  "02003501000100033c" 
\x35 position: -1


Byte:  
Byte from Local8bit: "\u0001�\u0012y9pU" 
Byte from Latim1: "\u0001\u0080\u0012y9pU" 
Byte to base 16: "1" 
 \x35 = test.at(3) false

J.Hilk

@carpajr
at(3) is the forth byte, whitch is 01 of course that comparison fails 🙃

JonB

@carpajr
First, as @J-Hilk says, test.at(3) is the 4th byte, or the "3rd" byte if you start counting from the "0th" byte, as you please.

Second

char status[] = "x\35";

Oh dear! What's this? Earlier/later you have written '\x35' and "\x35", so you do know. You seem like a nice chap (or chapess), attention to detail, please, especially if you are posting code for something you think is wrong (when it shouldn't be), take time to look at your code in detail!