Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. QByteArray and QString does not match in byte size ?
Forum Updated to NodeBB v4.3 + New Features

QByteArray and QString does not match in byte size ?

Scheduled Pinned Locked Moved Unsolved General and Desktop
12 Posts 4 Posters 2.6k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • Kent-DorfmanK Offline
    Kent-DorfmanK Offline
    Kent-Dorfman
    wrote on last edited by
    #3

    Don't access results directly. Think of it as a memory backing for the stream. Since the purpose of QDataStream is mostly in platform independent serialization, some type metadata is also stored in the backing. Four bytes of extra length smells strongly like length of message and type metadata.

    Try accessing the stored byte array using the readBytes() method on the stream and see what you get out of it.

    Let us know!

    1 Reply Last reply
    3
    • D Offline
      D Offline
      Dariusz
      wrote on last edited by Dariusz
      #4

      Hmmm I did not get far with it... still struggling to solve the basics... here is the current code I try to run :

      client.cpp

          /// Small data object
          QString dataA("21451254 125 32534 tgre34 r3re4v34gt24w efv42ew v");
          /// large data object
          QString dataB;
          for (int x = 0; x < 10000; ++x) {
              dataB += "23tgvrefbv4iknw23fgopq234;enwfo;nw2gno34wg3eorw;gn3eorw;'gn3ew";
          }
      
          QByteArray result;
          QDataStream stream(&result, QIODevice::WriteOnly);
          stream << 10 /// Packet idtype
                 << dataA.size()
                 << dataA.toUtf8()
                 << dataB.size()
                 << dataB.toUtf8();
          result.insert(0, result.size()); /// this is missing the extra overhead size?
      
          char *d;
          char *b;
          int totalSize, type, sizeA, bSize, sizeB;
      
      
          QDataStream streamX(&result, QIODevice::ReadOnly);
          streamX >> totalSize >> type; // read data for test
          mSocket->write(result);
      
          qDebug() << totalSize << result.size();
                  ///620756992 620070 - why totalSize so much bigger than result.size() ?
      

      The client sends the debug data above, no idea why the totalSize does not equal or its slightly over result.size() :- ( I though that result.size() would be about 4-x bytes more than totalSize but its wrong.

      And here is server reading it :
      server.cpp

      if (!mReadingData) {
          mReadingData = true;
          QByteArray rec = socket->readAll(); /// read as much as we can - specify a limit in future maybe?
          mData += rec; /// add new data
          mTotalSize = 0; /// 0 out total size counter
          QDataStream streamX(&rec, QIODevice::ReadOnly);
          streamX >> mTotalSize;//// get the size of full file?
          mDataLoaded += rec.size(); /// increment what we added
          qInfo() << "Last MSG Size : " << rec.size(); /// how much bytes we got from last message?
          qInfo() << "Readed Total data size : " << mTotalSize; /// how much bytes we need in total ?
      
      } else {
          qInfo()<<"Adding data";
          QByteArray rec = socket->readAll(); /// read next chunk of data?
          mData += rec;
          mDataLoaded += rec.size(); /// increment what we added
          qInfo() << "Last MSG Size : " << rec.size(); /// how much bytes we got from last message?
          qInfo() << "Readed Total data size : " << mTotalSize; /// how much bytes we need in total ?
          if (mDataLoaded == mTotalSize) { /// we loaded all data we send, now process the packet and unload the data in to proper object?
              processData(mData);
              mReadingData = false;
              mData.clear(); /// clear old data
          }
      }
      //qInfo() << "Bytes loaded : " << mDataLoaded;
      qInfo() << " TCPConnection::bytesAvailable" << socket->bytesAvailable(); /// always returns 0 ?
      qInfo() << " TCPConnection::bytesToWrite" << socket->bytesToWrite(); /// always returns 0 ?
      

      on Server site when I receive 1st transfer I get
      Last MSG Size : 14600 < thi seems ok, 1st chunk of data.
      Readed Total data size : 2666130979571105792 < this is waaaaaaaaaaaaay off, it should have been 620756992 if I am to believe the totalSize from client side

      And I'm lost. I must be checking or measuring something incorrectly here :- (

      I also tried to read data size via streamer but that returns 0

          char* raw;
          uint length;
          streamX.readBytes(raw, length);
          qDebug() << "length = " << length;
      
      1 Reply Last reply
      0
      • Christian EhrlicherC Offline
        Christian EhrlicherC Offline
        Christian Ehrlicher
        Lifetime Qt Champion
        wrote on last edited by
        #5

        You're writing raw bytes into a stream which is encoded with QDataStream and expect you don't break the datastream? Strange...

        Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
        Visit the Qt Academy at https://academy.qt.io/catalog

        D 1 Reply Last reply
        1
        • Christian EhrlicherC Christian Ehrlicher

          You're writing raw bytes into a stream which is encoded with QDataStream and expect you don't break the datastream? Strange...

          D Offline
          D Offline
          Dariusz
          wrote on last edited by
          #6

          @Christian-Ehrlicher You mean that if I do

            result.insert(0, result.size());
          

          I break something and that I should use QDataStream to add in the size of the data ?

          I'm very new to QDataStreams/QIODevice/QByteArrays tbh. Still getting a hang of it.

          1 Reply Last reply
          0
          • Christian EhrlicherC Offline
            Christian EhrlicherC Offline
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on last edited by
            #7

            QDataStream writes the data with it's own format into a stream/QByteArray. You just write something directly into the stream generated by QDateStream - so yes.

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            1 Reply Last reply
            1
            • SGaistS Offline
              SGaistS Offline
              SGaist
              Lifetime Qt Champion
              wrote on last edited by
              #8

              @Dariusz said in QByteArray and QString does not match in byte size ?:

              QByteArray result;
              QDataStream stream(&result, QIODevice::WriteOnly);
              stream << 10 /// Packet idtype
              << dataA.size()
              << dataA.toUtf8()
              << dataB.size()
              << dataB.toUtf8();
              result.insert(0, result.size()); /// this is missing the extra overhead size?

              dataA and dataB are QStrings which already has QDataStream operators so why are you trying to serialise them manually ?

              Interested in AI ? www.idiap.ch
              Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

              D 1 Reply Last reply
              2
              • SGaistS SGaist

                @Dariusz said in QByteArray and QString does not match in byte size ?:

                QByteArray result;
                QDataStream stream(&result, QIODevice::WriteOnly);
                stream << 10 /// Packet idtype
                << dataA.size()
                << dataA.toUtf8()
                << dataB.size()
                << dataB.toUtf8();
                result.insert(0, result.size()); /// this is missing the extra overhead size?

                dataA and dataB are QStrings which already has QDataStream operators so why are you trying to serialise them manually ?

                D Offline
                D Offline
                Dariusz
                wrote on last edited by Dariusz
                #9

                @SGaist I think because I don't know that :- ) even tho I read about it. I think I'm being overwhelmed by qt features and working against the flow of water. For examples I decided to go back to basic "structures" say :

                    unsigned char *ar;
                    unsigned int n = 123123;
                    ar[0] = (n >> 24) & 0xFF;
                    ar[1] = (n >> 16) & 0xFF;
                    ar[2] = (n >> 8) & 0xFF;
                    ar[3] = n & 0xFF;
                
                    qDebug() << ar[0] << ar[1] << ar[2] << ar[3];
                
                
                    int num = 0;
                    for (int i = 0; i < 4; i++) {
                        num <<= 8;
                        num |= ar[i];
                    }
                
                    qDebug() << num;
                

                This serialize/deserialize properly.

                But when I do

                    QByteArray ar;
                    ar.reserve(4);
                    unsigned int n = 123123;
                   .....
                

                Instead, then it does not work and I'm at loss. Perhaps I really miss understanding how I am meant to use QByteArray somehow. :- (

                Edit. doing more read on it, yeah it seems like QT does a lot more to help with my task than I expected and I'm pushing against the design...

                1 Reply Last reply
                0
                • Kent-DorfmanK Offline
                  Kent-DorfmanK Offline
                  Kent-Dorfman
                  wrote on last edited by Kent-Dorfman
                  #10
                  This post is deleted!
                  1 Reply Last reply
                  0
                  • Kent-DorfmanK Offline
                    Kent-DorfmanK Offline
                    Kent-Dorfman
                    wrote on last edited by Kent-Dorfman
                    #11

                    Here's a complete example for you, including passing an array.

                    #include <iostream>
                    #include <iomanip>
                    
                    #include <QByteArray>
                    #include <QDataStream>
                    #include <QString>
                    
                    // =============================================================================
                    int main(int argc, char** argv) {
                    
                        using namespace std;
                    
                        QByteArray buffer;                      // serialization backing buffer
                        QString aStr("No Way.....");            // a test string
                        QString bStr("Way, dude");              // second test string
                        int j(32768);                           // throw an integer in, too
                        const char* str("an array");
                        int l(strlen(str));
                    
                        cout << "--- input ---" << endl <<      // show stream input
                                aStr.toStdString() << endl <<
                                bStr.toStdString() << endl <<
                                str << endl <<
                                j << endl;
                    
                        // assume this end is the transmitter
                        QDataStream serialize(&buffer, QIODevice::WriteOnly);
                        serialize << aStr << bStr;
                        serialize.writeBytes(str, l);
                        serialize << j;         // serialize the data
                    
                    
                        // SEND "buffer" as a network packet somewhere else, then read it from the socket
                    
                        // assume this end is on the receiver
                        QDataStream deserialize(&buffer, QIODevice::ReadOnly);
                        QString outStrA, outStrB;
                        int i;
                        unsigned outL;
                        char* ptr;
                    
                        // pull the data out of the stream in FIFO order
                        deserialize >> outStrA >> outStrB;
                        deserialize.readBytes(ptr, outL);
                        deserialize >> i;
                    
                         // show the output
                        cout << endl << endl << "--- output ---" << endl <<
                                outStrA.toStdString() << endl <<
                                outStrB.toStdString() << endl <<
                                ptr << "    --- length=" << outL << endl <<
                                i << endl;
                    
                        delete [] ptr;
                    
                        return 0;
                    
                    }
                    
                    
                    D 1 Reply Last reply
                    1
                    • Kent-DorfmanK Kent-Dorfman

                      Here's a complete example for you, including passing an array.

                      #include <iostream>
                      #include <iomanip>
                      
                      #include <QByteArray>
                      #include <QDataStream>
                      #include <QString>
                      
                      // =============================================================================
                      int main(int argc, char** argv) {
                      
                          using namespace std;
                      
                          QByteArray buffer;                      // serialization backing buffer
                          QString aStr("No Way.....");            // a test string
                          QString bStr("Way, dude");              // second test string
                          int j(32768);                           // throw an integer in, too
                          const char* str("an array");
                          int l(strlen(str));
                      
                          cout << "--- input ---" << endl <<      // show stream input
                                  aStr.toStdString() << endl <<
                                  bStr.toStdString() << endl <<
                                  str << endl <<
                                  j << endl;
                      
                          // assume this end is the transmitter
                          QDataStream serialize(&buffer, QIODevice::WriteOnly);
                          serialize << aStr << bStr;
                          serialize.writeBytes(str, l);
                          serialize << j;         // serialize the data
                      
                      
                          // SEND "buffer" as a network packet somewhere else, then read it from the socket
                      
                          // assume this end is on the receiver
                          QDataStream deserialize(&buffer, QIODevice::ReadOnly);
                          QString outStrA, outStrB;
                          int i;
                          unsigned outL;
                          char* ptr;
                      
                          // pull the data out of the stream in FIFO order
                          deserialize >> outStrA >> outStrB;
                          deserialize.readBytes(ptr, outL);
                          deserialize >> i;
                      
                           // show the output
                          cout << endl << endl << "--- output ---" << endl <<
                                  outStrA.toStdString() << endl <<
                                  outStrB.toStdString() << endl <<
                                  ptr << "    --- length=" << outL << endl <<
                                  i << endl;
                      
                          delete [] ptr;
                      
                          return 0;
                      
                      }
                      
                      
                      D Offline
                      D Offline
                      Dariusz
                      wrote on last edited by Dariusz
                      #12

                      Hey @Kent-Dorfman, thanks for the example, it looks interesting. But I'm still lost :- ) Sorry...

                      In any case, after testing the char *ar example above, I went back to re-read more on QDataStream/QTcpSocket/QByteArrays, Seems that in latest Qt that entire system is more "connected", ie if I send large file, I no longer need to handle the size of the file/packet/data I want to send as 1st argument when sending it across QTcp as that is now handled by Qt. - unless I read incorrectly.

                      I've figured out this part lately, but now I moved to reading the data from QTcpSocket and that's where I seem to be stuck..

                          QByteArray ar;
                          QDataStream sendStream(&ar, QIODevice::ReadWrite);
                          sendStream << quint16(0) << dataA << dataB;
                          sendStream.device()->seek(0);  
                          sendStream << (quint16)(ar.size() - sizeof(quint16)); // replace the previously allocated quint16(0) value
                      

                      And now I'm stuck on socketReading it so

                      slot:

                      void networkTest::readyRead() {
                           QTcpSocket *socket = static_cast<QTcpSocket *>(sender());
                           if (!socket) return;
                           QDataStream in(socket);
                           in.startTransaction();
                           in >> mData; /// mData = QByteArray - we no longer need size of data because QT handles it (???) so just add data to mData .h memher
                           if (!in.commitTransaction()) { // check if end data, if no, return and wait for more data in next event loop 
                               active();
                               qDebug() << in->status(); // returns 1 every time ? :- (
                               return;
                          }
                          /// never gets called 
                          processReceivedData();
                      

                      }

                      TIA

                      Ok apparently, this does not work, it breaks entire data in QByteArray...

                          serialize.device()->seek(0);
                          serialize << buffer.size() - sizeof(quint16);
                      

                      ...

                      This is fun rollercoaster. 1 step forward, 2 steps backwards.

                      Ok perhaps the seek(0) does work, I just need to stick to proper types..

                          sendStream.device()->seek(0);
                          sendStream << (int) (block.size() - sizeof(int));
                      

                      cant use quint, and all that. Hmmm... 1 step forward... lets see what are the next 2 steps back :- )

                      EDIT 123123.
                      Ok I got it working, yeah esentially I need to ensure that whatever int I pass in int/quint/uint/uint8_t etc etc that they all match and are the same ints... say QByeteArray.size() return int, then I have to cast every other int in to (int) number to properly work. Even if updating existing value in array via stream/qiodevice seek...

                      Ahh this was fun. Thanks every one for help!

                      1 Reply Last reply
                      0

                      • Login

                      • Login or register to search.
                      • First post
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Users
                      • Groups
                      • Search
                      • Get Qt Extensions
                      • Unsolved