Problem converting QByteArray to long

nico162

I have a Java server application on a first side which send byteBuff message. For the moment I send from this application a long, convert in bytes:

Here is my java code:

// ConvertTools is a class I use to convert byte[] to other type (int,long, string ..) and to convert from these types o byte[].
long h = -4537561759989730831L;
fullMessage=ConvertTools.long2Byte(h);
client.messageToIP(fullMessage);
System.out.println("Message sent");


//I check here the bytes send and I do the conversion on the other side to check it returns the correct long.
for(int i=0;i<fullMessage.lenght();i++){
System.out.println(fullMessage[0]);
System.out.println(fullMessage.length);
System.out.println(ConvertTools.byte2Long(fullMessage));
}

So I know the content of my message.

In Qt then, I am listening a socket to recover the message sent and then I save it in a QbyteArray
Here is my Qt Code that recover the QByteArray:

void ClientTcp::readyRead(){
    qInfo()<<"[Debug]: Reading ...";
    QByteArray receivedByteArray = socket->readLine();
    qInfo()<<"FIN ByteBuff";
    assocaitedReceiver->on_receivingByteBuffMessage(receivedByteArray);
}

Then it launch this method (method I'll use for analyzing) :

void Receiver::on_receivingByteBuffMessage(QByteArray msg){
    QByteArray data(msg);
    QByteArray hashKeyValue;
    long value = -1L;
    hashKeyValue[0]= data[0];
    hashKeyValue[1]= data[1];
    hashKeyValue[2]= data[2];
    hashKeyValue[3]= data[3];
    hashKeyValue[4]= data[4];
    hashKeyValue[5]= data[5];
    hashKeyValue[6]= data[6];
    hashKeyValue[7]= data[7];
    for(int i=0;i<8;i++){
        qInfo()<<(qint64)hashKeyValue[0];
    }
    bool ok;
    value = hashKeyValue.toLong(&ok,10);
    qInfo()<<QString::number(value,10);
    qInfo()<<ok;
}

The values of the byte are the same at the reception than at the sending and in the correct order.

But the returned value is 0 and false for the bool ok . Why does the conversion failed, I test a lot of way to recover the data and convert it in another way than using QByteArray, but the returned values are never correct.

Can anyone help me? I am new in Qt and C++ :/..

JonB

@nico162
First let's start by having your actual code. Do you really have:

    for(int i=0;i<8;i++){
        qInfo()<<(qint64)hashKeyValue[0];
    }

or would you like to correct that in either code or pasting here?

Second, have a look at the docs & example for http://doc.qt.io/qt-5/qbytearray.html#toLong. It's not doing anything like you're expecting it to do: it's for parsing a byte-array-string to a long, not creating a long out of a number already sitting there in binary bytes!

VRonin

First of all, you need to understand endianness

Matching endianness between java and C++


void Receiver::on_receivingByteBuffMessage(const QByteArray& msg){
Q_ASSERT(msg.size()==sizeof(qint64));
qint64 value = *reinterpret_cast<const qint64*>(msg.constData());
}

Mismatching endianness between java and C++

void Receiver::on_receivingByteBuffMessage(const QByteArray& msg){
Q_ASSERT(msg.size()==sizeof(qint64));
QByteArray reversed(sizeof(qint64),0);
std::copy(msg.crbegin(),msg.crend(),reversed.begin());
qint64 value = *reinterpret_cast<const qint64*>(reversed.constData());
}

JonB

@VRonin
Your "reversing" code involves actually copying the bytes in reverse order into a temporary area to "convert" it to a native int64. This is hideously inefficient if you have a file with billions of numbers in it :) Isn't there a solution which swaps bytes directly (e.g. so could be done on a register directly)?

VRonin

@JonB That's a very good question but I'm not smart enough to know the answer

JonB

@VRonin
LOL, of course you're smart enough!

I was thinking of e.g. https://linux.die.net/man/3/htonl. These functions are "byte-swappers", and they were defined in macros with lots of << & >>s :) But they only go up to int32 size :(

EDIT I found this sample:

unsigned long int Endian_DWord_Conversion(unsigned long int dword)
 {
   return ((dword>>24)&0x000000FF) | ((dword>>8)&0x0000FF00) | ((dword<<8)&0x00FF0000) | ((dword<<24)&0xFF000000);
}

It's a question of expanding that approach from 4 to 8 bytes, and marking it inline!

mpergand

What about using QDataStream:

QDataStream data(msg);
qint64 val;
data>>val;

QDataStream is smart enough to convert to the host endianess automaticaly.
By defaut, DataStream is set to big endian, see setByteOrder().

Christian Ehrlicher

@mpergand said in Problem converting QByteArray to long:

What about using QDataStream

Since QDataStream is a stream format it will not only store the 4 or 8 bytes but also some information to know what is saved - therefore it's totally useless here.

SGaist

Hi,

Another important point, you don't check that you received all your data. The readyRead doesn't signal that the socket got the complete payload, only there's something that arrived. Your payload might be small enough now but it might very well change in the near future.

nico162

@JonB

my bad, no it's
for(int i=0;i<8;i++){
qInfo()<<(qint64)hashKeyValue[i];
}

Ah okay.. Thank you

JonB

@nico162
That's why it's always best to copy & paste code, not type it in, when you want help! :)

nico162

@VRonin
Yes I though it could be that, but the bytes sent from Java to C++, are of same value and in the same order at the sending and receiving (I print the content in java and in C++, so I know the structure of the byte array).
So I admit that it wasn't an endianness error (maybe I am wrong ..).

nico162

@VRonin
Ah no okay I misunderstood... Now I see what you mean :D! And it works fine

nico162

@nico162 said in Problem converting QByteArray to long:

@VRonin
Ah no okay I misunderstood... Now I see what you mean :D! And it works fine

So as @JonB explained the QByteArray::toLong() is not adapted to my need (to convert a byte array to long).
@VRonin gave a solution to interpret the byteArray in the correct endianness mode (from Java to C++):

void Receiver::on_receivingByteBuffMessage(const QByteArray& msg){
Q_ASSERT(msg.size()==sizeof(qint64));
QByteArray reversed(sizeof(qint64),0);
std::copy(msg.crbegin(),msg.crend(),reversed.begin());
qint64 value = *reinterpret_cast<const qint64*>(reversed.constData());
}

Maybe better solution could exist to avoid create a temp array to copy the byte in the reverse order as mentioned @JonB .
I personally have little bytebuffer received, so the solution that @VRonin gave it's enough efficient in my use case, that's why I close this topic.
@SGaist I take in account your remark thank you :).
I thank you all for your help :)! First time I posted a topic, and I am happy to see that there is an helpful and comprehensive community.

See ya !

JohanSolo

@JonB said in Problem converting QByteArray to long:

Isn't there a solution which swaps bytes directly (e.g. so could be done on a register directly)?

Would by chance _byteswap_uint64 or __builtin_bswap64 do the trick?

JonB

@JohanSolo
They would indeed, but each of those is MSVC/GCC specific ... :(

JohanSolo

@JonB
I know, that's why I gave the two of them... Not the first time a #ifdef WIN32 would help.

JonB

@JohanSolo
Yeah, but I think (for Qt) we still have to support MinGW and whatever on MacOS and ...

Unlike the functions for swapping 16 or 32 bytes, I couldn't find standard ones for 64 bytes anywhere when I searched. Shame!