How to send arabic string over Qserialport

Kamal Badi

hi,
I have problem in sending Arabic characters through serialport
am using qt 4.8 in pda, and I want to send string to C# application running in windows Desktop,
the problem is want to convert the text into utf 8 and the QByteArray, i used .toutf8(); but the string is not appear correct!

koahnig

@Kamal-Badi

I am certainly not an UTF-8 expert.
You seem to connect the problem already with the serial transfer. The implication is that you checked the string conversion on the sending machine and there everything is correct. Only on the other machine the string is not correctly represented?

One thing you may want to check is the byte order representation is the same on both machines. I do not know if this has been covered by the definition of UTF-8, but in general the byte order plays a significant role in those transfers.

Kamal Badi

yes, actually when I printed using QmessageBox it represent well, in the other hand when I printed using Qdebug as like

qDebug()<<QString::fromUtf8("عربي");
//the output is: ????

I assumed when it can be represented well in the qmessagebox well then it must be the problem within qserialport, but really am totally lost !

aha_1980

Hi @Kamal-Badi In Addition to what @koahnig said (the byte order is a good hint!), I'd suggest to print the byte array as hex output on the sender side directly before sending, a la:.

QString str = "your string";
QByteArray data = str.toUtf8();
qDebug() << data.toHex();
serial.write(data); // send serial data

Then you print an debug output on the receiver side and compare both hexadecimal datas. Take an UTF-8 code table and verify what's within the QByteArrays is correct. That should give you an indication where to look further.

Kamal Badi

hi @aha_1980, thanks for replying ..
i tried what you said,
print the ByteArray to hex, then i converted to text using online tool, and the result was the same it gives me "????" for string "كمال" the rest of the string which contain numbers is converted fine,
the result was as follow
the original text

$5,1,21,8,كمال,0,0,0,0,0,False,True,0,False,0,0,,1,55557,55666,8550659$

the output of hex

24352c312c32312c382c3f3f3f3f2c302c302c302c302c302c46616c73652c547275652c302c46616c73652c302c302c2c312c35353535372c35353636362c3835353036353924

when i converted using online tool it gives me

$5,1,21,8,????,0,0,0,0,0,False,True,0,False,0,0,,1,55557,55666,8550659$

when convert the original text using the online tool to hex

24352c312c32312c382cd983d985d8a7d9842c302c302c302c302c302c46616c73652c547275652c302c46616c73652c302c302c2c312c35353535372c35353636362c3835353036353924

comparing the two results the first hex the comes from my app can't represent the Arabic characters well so it gives me "????" instead of "كمال" so the receiver has nothing to do with this, its all about how to convert the Arabic to utf8 without corrupting the data?

koahnig

@Kamal-Badi

Probably you should print same arabic characters on both sides and analyse the hex stream. Also it will help when simply print arabic character by character. This one is a bit long and you get confused when trying compare. Also something is already different with the "0,0,0,.." string which is in another place compared to the sent string.

Kamal Badi

The zeros appear like that due to RTL and LTR, because Arabic written from the right to left yet English is from Left to right, that is not an issue, my concern now is getting the right format,
I'm already tried to send it to terminal on the receiver side it give me the exact result with questions mark instead of Arabic chars.
Now by converting the string into hex its so clearly that the problem with converting the Arabic into utf8 encode, But the confusion part is why the Qmessagebox is getting the string right!
And how can I convert the text into utf8 correctly?

koahnig

@Kamal-Badi

I would write a small application with only a message box outputting one letter in the string. Compile and start the application both machines. There you can confirm the proper handling of Arabic letters.

Than, when all is correct, add to the application the hex output for the one character string. You can basically check the sequence of bytes for every Arabic character separately. By doing that you can ensure that you are running into an issue with endianness, which is quite common when you are simply using serial communication.

Kamal Badi

finally,

    QTextCodec::setCodecForCStrings(QTextCodec::codecForName("UTF-8"));

solve any thing ... ^^