Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Decoding QByteArray into UTF-16 and UInt32

Decoding QByteArray into UTF-16 and UInt32

Scheduled Pinned Locked Moved Unsolved General and Desktop
13 Posts 4 Posters 2.6k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J Offline
    J Offline
    jars121
    wrote on 9 Aug 2019, 00:26 last edited by
    #1

    I'm trying to read data from a UDP connection. I'm able to bind to the IP address and port successfully, and have the readyRead() callback function all operating as expected.

    As part of the initiation, I send a packet to the UDP server, which responds with some handshake data. The handshake data is 408 bytes (per the UDP server documentation), which I've verified. The 408 bytes are specified as follows:

    • 50 char UTF-16 (50*2 bytes)
    • 50 char UTF-16 (50*2 bytes)
    • Int (4 bytes)
    • Int (4 bytes)
    • 50 char UTF-16 (50*2 bytes)
    • 50 char UTF-16 (50*2 bytes)

    All bytes are Little Endian. The issue I'm having is in decoding the bytes into strings (for the UTF-16 bytes) and integers (for the ints). Below is my code (I've removed some of the UDPSocket setup as it's not relevant).

    while (udpSocket->hasPendingDatagrams()) {
        QHostAddress sender = QHostAddress(IPAddr);
        quint16 port = udpPort;
    
        QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
        udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
    
        qDebug() << buf.mid(0, 100).toHex();
        qDebug() << buf.mid(200, 4).toHex();
    }
    

    How can I modify the second last line to print the first UTF-16 packet as a string and the last line to print the first int packet as an int? I've got toHex() there as an example.

    J A 2 Replies Last reply 9 Aug 2019, 04:29
    0
    • J jars121
      9 Aug 2019, 00:26

      I'm trying to read data from a UDP connection. I'm able to bind to the IP address and port successfully, and have the readyRead() callback function all operating as expected.

      As part of the initiation, I send a packet to the UDP server, which responds with some handshake data. The handshake data is 408 bytes (per the UDP server documentation), which I've verified. The 408 bytes are specified as follows:

      • 50 char UTF-16 (50*2 bytes)
      • 50 char UTF-16 (50*2 bytes)
      • Int (4 bytes)
      • Int (4 bytes)
      • 50 char UTF-16 (50*2 bytes)
      • 50 char UTF-16 (50*2 bytes)

      All bytes are Little Endian. The issue I'm having is in decoding the bytes into strings (for the UTF-16 bytes) and integers (for the ints). Below is my code (I've removed some of the UDPSocket setup as it's not relevant).

      while (udpSocket->hasPendingDatagrams()) {
          QHostAddress sender = QHostAddress(IPAddr);
          quint16 port = udpPort;
      
          QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
          udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
      
          qDebug() << buf.mid(0, 100).toHex();
          qDebug() << buf.mid(200, 4).toHex();
      }
      

      How can I modify the second last line to print the first UTF-16 packet as a string and the last line to print the first int packet as an int? I've got toHex() there as an example.

      J Online
      J Online
      jsulm
      Lifetime Qt Champion
      wrote on 9 Aug 2019, 04:29 last edited by
      #2

      @jars121
      https://doc.qt.io/qt-5/qstring.html#fromUtf16
      https://doc.qt.io/qt-5/qbytearray.html#toInt

      https://forum.qt.io/topic/113070/qt-code-of-conduct

      1 Reply Last reply
      4
      • H Offline
        H Offline
        hskoglund
        wrote on 9 Aug 2019, 05:02 last edited by
        #3

        Hmm, first link looks good, but 2nd link: using toInt() will most likely fail, because it assumes that the QByteArray contents was generated with something like QByteArray::number().

        I'm guessing the handshake integer data is raw (not formatted as a string), here's an example:

        QByteArray s("1234"); // this works with toInt()
        QByteArray n("\u0001\u0002\u0003\u0004");  // but I'm guessing this flavor
        
        int i1 = s.toInt();
        int i2 = n.toInt();
        
        qDebug() << i1 << i2;
        
        memcpy(&i2,n.data(),4);  // so, use this style of conversion
        qDebug() << i2;
        
        // which is the same as this (1,2,3,4 Little Endian four bytes)
        qDebug() << 4 * 256*256*256 + 3 * 256*256 + 2 * 256 + 1;
        
        J 2 Replies Last reply 9 Aug 2019, 05:10
        0
        • H hskoglund
          9 Aug 2019, 05:02

          Hmm, first link looks good, but 2nd link: using toInt() will most likely fail, because it assumes that the QByteArray contents was generated with something like QByteArray::number().

          I'm guessing the handshake integer data is raw (not formatted as a string), here's an example:

          QByteArray s("1234"); // this works with toInt()
          QByteArray n("\u0001\u0002\u0003\u0004");  // but I'm guessing this flavor
          
          int i1 = s.toInt();
          int i2 = n.toInt();
          
          qDebug() << i1 << i2;
          
          memcpy(&i2,n.data(),4);  // so, use this style of conversion
          qDebug() << i2;
          
          // which is the same as this (1,2,3,4 Little Endian four bytes)
          qDebug() << 4 * 256*256*256 + 3 * 256*256 + 2 * 256 + 1;
          
          J Online
          J Online
          jsulm
          Lifetime Qt Champion
          wrote on 9 Aug 2019, 05:10 last edited by
          #4

          @hskoglund @jars121 Better/cleaner solution would be to use https://doc.qt.io/qt-5/qdatastream.html and overload needed QDataStream &QDataStream::operator>>(...) operators.

          https://forum.qt.io/topic/113070/qt-code-of-conduct

          1 Reply Last reply
          3
          • H hskoglund
            9 Aug 2019, 05:02

            Hmm, first link looks good, but 2nd link: using toInt() will most likely fail, because it assumes that the QByteArray contents was generated with something like QByteArray::number().

            I'm guessing the handshake integer data is raw (not formatted as a string), here's an example:

            QByteArray s("1234"); // this works with toInt()
            QByteArray n("\u0001\u0002\u0003\u0004");  // but I'm guessing this flavor
            
            int i1 = s.toInt();
            int i2 = n.toInt();
            
            qDebug() << i1 << i2;
            
            memcpy(&i2,n.data(),4);  // so, use this style of conversion
            qDebug() << i2;
            
            // which is the same as this (1,2,3,4 Little Endian four bytes)
            qDebug() << 4 * 256*256*256 + 3 * 256*256 + 2 * 256 + 1;
            
            J Online
            J Online
            jsulm
            Lifetime Qt Champion
            wrote on 9 Aug 2019, 05:11 last edited by
            #5

            @hskoglund said in Decoding QByteArray into UTF-16 and UInt32:

            memcpy(&i2,n.data(),4); // so, use this style of conversion

            Don't forget about endianness :-)

            https://forum.qt.io/topic/113070/qt-code-of-conduct

            1 Reply Last reply
            3
            • J jars121
              9 Aug 2019, 00:26

              I'm trying to read data from a UDP connection. I'm able to bind to the IP address and port successfully, and have the readyRead() callback function all operating as expected.

              As part of the initiation, I send a packet to the UDP server, which responds with some handshake data. The handshake data is 408 bytes (per the UDP server documentation), which I've verified. The 408 bytes are specified as follows:

              • 50 char UTF-16 (50*2 bytes)
              • 50 char UTF-16 (50*2 bytes)
              • Int (4 bytes)
              • Int (4 bytes)
              • 50 char UTF-16 (50*2 bytes)
              • 50 char UTF-16 (50*2 bytes)

              All bytes are Little Endian. The issue I'm having is in decoding the bytes into strings (for the UTF-16 bytes) and integers (for the ints). Below is my code (I've removed some of the UDPSocket setup as it's not relevant).

              while (udpSocket->hasPendingDatagrams()) {
                  QHostAddress sender = QHostAddress(IPAddr);
                  quint16 port = udpPort;
              
                  QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
                  udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
              
                  qDebug() << buf.mid(0, 100).toHex();
                  qDebug() << buf.mid(200, 4).toHex();
              }
              

              How can I modify the second last line to print the first UTF-16 packet as a string and the last line to print the first int packet as an int? I've got toHex() there as an example.

              A Offline
              A Offline
              aha_1980
              Lifetime Qt Champion
              wrote on 9 Aug 2019, 05:32 last edited by
              #6

              @jars121 as @jsulm already said, your problem sounds like an excellent case for QDataStream.

              Regards

              Qt has to stay free or it will die.

              1 Reply Last reply
              1
              • J Offline
                J Offline
                jars121
                wrote on 9 Aug 2019, 05:42 last edited by
                #7

                Thanks everyone, much appreciated as always.

                I 've got the below amended QDataStream approach, but I've not used this method before so am not quite getting it.

                while (udpSocket->hasPendingDatagrams()) {
                    QHostAddress sender = QHostAddress(IPAddr);
                    quint16 port = udpPort;
                
                    QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
                    QDataStream str(&buf, QIODevice::ReadOnly);
                    udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
                
                    str.setByteOrder(QDataStream::LittleEndian);
                
                    //How do I get the first UTF-16 value as a string here?
                    quint16 firstString;
                    str >> firstString;
                }
                

                I get the logic to the above, but I don't see how I can break the QDataStream into its constituent packets (i.e. 4x 100 byte 50-character UTF-16s and 2x 32-bit ints) like I can do with the buf.mid() approach. Do I need to str.readBytes() or str.readRawData() beforehand?

                1 Reply Last reply
                0
                • J Offline
                  J Offline
                  jars121
                  wrote on 9 Aug 2019, 07:30 last edited by jars121 8 Sept 2019, 07:42
                  #8

                  Ok I've been playing with this, and seem to have worked out a solution using the original QByteArray approach which I've listed below. I'm really looking for the optimal approach however, so if the QDataStream approach is better I'm more than happy to use that, but I'll need some guidance as I've not yet had any success.

                  while (udpSocket->hasPendingDatagrams()) {
                      QHostAddress sender = QHostAddress(IPAddr);
                      quint16 port = udpPort;
                  
                      QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
                      udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
                  
                      QByteArray first;
                      QByteArray second;
                  
                      first.append(buf.mid(0, 100));
                      second.append(buf.mid(100, 100));
                  
                      QString strFirst = QString::fromUtf16(reinterpret_cast<const ushort*>(first.constData());
                      QString strSecond = QString::fromUtf16(reinterpret_cast<const ushort*>(second.constData());
                  }
                  

                  The above seems to work nicely, but I can't help but feel that using a QByteArray for each parameter in the packet is exactly what the QDataStream approach would be doing for me :D

                  Edit: Also, I can't seem to get the same approach to work for the ints. The QByteArray::toInt method always results in 0.

                  1 Reply Last reply
                  0
                  • J Offline
                    J Offline
                    jars121
                    wrote on 9 Aug 2019, 07:53 last edited by
                    #9

                    Ok, I'm able to decode all 6 parameters, but I can't imagine the below approach is the optimal way to do so.

                    while (udpSocket->hasPendingDatagrams()) {
                        QHostAddress sender = QHostAddress(IPAddr);
                        quint16 port = udpPort;
                    
                        QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
                        udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
                    
                        QByteArray first;
                        QByteArray second;
                        QByteArray third;
                        QByteArray fourth;
                        QByteArray fifth;
                        QByteArray sixth;
                    
                        first.append(buf.mid(0, 100));
                        second.append(buf.mid(100, 100));
                        third.append(buf.mid(200, 4));
                        fourth.append(buf.mid(204, 4));
                        fifth.append(buf.mid(208, 100));
                        sixth.append(buf.mid(308, 100));
                    
                        QString stringFirst = QString::fromtUtf16(reinterpret_cast<const ushort*>(first.constData()));
                        QString stringSecond = QString::fromtUtf16(reinterpret_cast<const ushort*>(second.constData()));
                    
                        int intThird = third.toInt();
                        memcpy(&intThird, third.data(), 4);
                    
                        int intFourth = fourth.toInt();
                        memcpy(&intFourth, fourth.data(), 4);
                    
                        QString stringFifth = QString::fromtUtf16(reinterpret_cast<const ushort*>(fifth.constData()));
                        QString stringSixth = QString::fromtUtf16(reinterpret_cast<const ushort*>(sixth.constData()));
                    }
                    

                    I'll have another play with a QDataStream approach now, as I'm sure the above can be simplified with a single QDataStream object.

                    A 1 Reply Last reply 9 Aug 2019, 19:47
                    0
                    • J jars121
                      9 Aug 2019, 07:53

                      Ok, I'm able to decode all 6 parameters, but I can't imagine the below approach is the optimal way to do so.

                      while (udpSocket->hasPendingDatagrams()) {
                          QHostAddress sender = QHostAddress(IPAddr);
                          quint16 port = udpPort;
                      
                          QByteArray buf(udpSocket->pendingDatagramSize(), Qt::Uninitialized);
                          udpSocket->readDatagram(buf.data(), buf.size(), &sender, &udpPort);
                      
                          QByteArray first;
                          QByteArray second;
                          QByteArray third;
                          QByteArray fourth;
                          QByteArray fifth;
                          QByteArray sixth;
                      
                          first.append(buf.mid(0, 100));
                          second.append(buf.mid(100, 100));
                          third.append(buf.mid(200, 4));
                          fourth.append(buf.mid(204, 4));
                          fifth.append(buf.mid(208, 100));
                          sixth.append(buf.mid(308, 100));
                      
                          QString stringFirst = QString::fromtUtf16(reinterpret_cast<const ushort*>(first.constData()));
                          QString stringSecond = QString::fromtUtf16(reinterpret_cast<const ushort*>(second.constData()));
                      
                          int intThird = third.toInt();
                          memcpy(&intThird, third.data(), 4);
                      
                          int intFourth = fourth.toInt();
                          memcpy(&intFourth, fourth.data(), 4);
                      
                          QString stringFifth = QString::fromtUtf16(reinterpret_cast<const ushort*>(fifth.constData()));
                          QString stringSixth = QString::fromtUtf16(reinterpret_cast<const ushort*>(sixth.constData()));
                      }
                      

                      I'll have another play with a QDataStream approach now, as I'm sure the above can be simplified with a single QDataStream object.

                      A Offline
                      A Offline
                      aha_1980
                      Lifetime Qt Champion
                      wrote on 9 Aug 2019, 19:47 last edited by
                      #10

                      @jars121,

                      I think your problem are the 50 char UTF-16. I guess these are no QStrings, therefore you cannot directly serialize/deserialize them. For the int that's no problem, QDataStream can handle that.

                      I wonder, if you can set up a type using UnicodeArray = std::array<QChar, 50>; and use that to read from the data stream?

                      Untested example:

                      QDataStream in(&udpSocket);  
                      UnicodeArray arr0;
                      UnicodeArray arr1;
                      qint32 a;
                      qint32 b;
                      UnicodeArray arr2;
                      UnicodeArray arr3;
                      in >> arr0 >> arr1 >> a >> b >> arr2 >> arr3; 
                      

                      Now you would just need to convert the UnicodeArray into QString.

                      Regards

                      Qt has to stay free or it will die.

                      J 1 Reply Last reply 9 Aug 2019, 22:01
                      0
                      • A aha_1980
                        9 Aug 2019, 19:47

                        @jars121,

                        I think your problem are the 50 char UTF-16. I guess these are no QStrings, therefore you cannot directly serialize/deserialize them. For the int that's no problem, QDataStream can handle that.

                        I wonder, if you can set up a type using UnicodeArray = std::array<QChar, 50>; and use that to read from the data stream?

                        Untested example:

                        QDataStream in(&udpSocket);  
                        UnicodeArray arr0;
                        UnicodeArray arr1;
                        qint32 a;
                        qint32 b;
                        UnicodeArray arr2;
                        UnicodeArray arr3;
                        in >> arr0 >> arr1 >> a >> b >> arr2 >> arr3; 
                        

                        Now you would just need to convert the UnicodeArray into QString.

                        Regards

                        J Offline
                        J Offline
                        jars121
                        wrote on 9 Aug 2019, 22:01 last edited by
                        #11

                        @aha_1980 I'm afraid I'm getting the following error:

                        invalid operands to binary expression ('QDataStream' and 'std::array<QChar, 50>')
                        

                        Your earlier assumption is correct, the 50 char UTF-16 are not QStrings. The reinterpret_cast approach I posted earlier does work for decoding, but I'd rather use QDataStream instead of having a separate QByteArray for each parameter.

                        I feel like I've tried every permutation with the QDataStream approach but I'm not getting the expected output. Even sizeof() testing of the various components doesn't return the correct result for some reason.

                        1 Reply Last reply
                        0
                        • J Offline
                          J Offline
                          jars121
                          wrote on 9 Aug 2019, 22:59 last edited by
                          #12

                          Ok so I've made some progress; the below correctly prints the first packet from the QDataStream:

                          ushort packet1[50];
                          ushort packet2[50];
                          quint32 packet3;
                          quint32 packet4;
                          ushort packet5[50];
                          ushort packet6[50];
                          
                          uint16_t i = 0;
                          while (str.atEnd() == false) {
                              str >> packet1[i];
                              i++;
                          }
                          

                          qDebug() << QString::fromUtf16(reinterpret_cast<const ushort*>(packet1));

                          How can I expand the while loop operation(s) to put the QDataStream (str) data into the remain ushorts and quint32s? I really don't understand why a simple str >> packet1 >> packet2 >> etc. operation doesn't work?

                          H 1 Reply Last reply 10 Aug 2019, 01:44
                          0
                          • J jars121
                            9 Aug 2019, 22:59

                            Ok so I've made some progress; the below correctly prints the first packet from the QDataStream:

                            ushort packet1[50];
                            ushort packet2[50];
                            quint32 packet3;
                            quint32 packet4;
                            ushort packet5[50];
                            ushort packet6[50];
                            
                            uint16_t i = 0;
                            while (str.atEnd() == false) {
                                str >> packet1[i];
                                i++;
                            }
                            

                            qDebug() << QString::fromUtf16(reinterpret_cast<const ushort*>(packet1));

                            How can I expand the while loop operation(s) to put the QDataStream (str) data into the remain ushorts and quint32s? I really don't understand why a simple str >> packet1 >> packet2 >> etc. operation doesn't work?

                            H Offline
                            H Offline
                            hskoglund
                            wrote on 10 Aug 2019, 01:44 last edited by
                            #13

                            @jars121 said

                            I really don't understand why a simple str >> packet1 >> packet2 >> etc. operation doesn't work?

                            Because while QDataStream is powerful, it does not know about packets of 50 ushorts. A single ushort, no problem. But old C-style arrays, e.g. *ushort packet1[50] are unknown territory :-(

                            Instead, QDataStream supports arrays in the flavors of QList, QVector etc. If you use QVector, you can do str >> myVector, voila.

                            But (there's always a but) for you to be able use QVector instead of ushort packet1[50], it needs to arrive in the correct format over the wire/UDP: the 50 ushorts has to be prefixed with a length/count integer. If you cannot persuade whatever entity is handing you the ushorts over the UDP connection to prefix them with a length integer, you're a bit out of luck. But I would suggest you use QVectors anyway, say like this:

                            #include "qvector.h"
                            
                            // declare the packets
                            QVector<ushort> packet1(50);
                            QVector<ushort> packet2(50);
                            quint32 packet3;
                            quint32 packet4;
                            QVector<ushort> packet5(50);
                            QVector<ushort> packet6(50);
                            
                            str.setByteOrder(QDataStream::LittleEndian);
                            
                            for (auto &u : packet1)
                                str >> u;
                            QString s1 = QString::fromUtf16(packet1.data());
                            
                            for (auto &u : packet2)
                                str >> u;
                            QString s2 = QString::fromUtf16(packet2.data());
                            
                            str >> packet3;
                            str >> packet4;
                            
                            for (auto &u : packet5)
                                str >> u;
                            QString s5 = QString::fromUtf16(packet5.data());
                            
                            for (auto &u : packet6)
                                str >> u;
                            QString s6 = QString::fromUtf16(packet6.data());
                            

                            (I.e. instead of str >> packet1; you can use a for loop, only one extra line).

                            1 Reply Last reply
                            2

                            4/13

                            9 Aug 2019, 05:10

                            topic:navigator.unread, 9
                            • Login

                            • Login or register to search.
                            4 out of 13
                            • First post
                              4/13
                              Last post
                            0
                            • Categories
                            • Recent
                            • Tags
                            • Popular
                            • Users
                            • Groups
                            • Search
                            • Get Qt Extensions
                            • Unsolved