Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Trouble with serializing and deserializing structs

Trouble with serializing and deserializing structs

Scheduled Pinned Locked Moved Solved General and Desktop
8 Posts 3 Posters 978 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R Offline
    R Offline
    RobbieP
    wrote on 29 Mar 2022, 13:15 last edited by
    #1

    Hi all! I have a struct I'm trying to deal with...This program needs to primarily deserialize it from other sources, but for testign purposes I'm doing my own serialization in the program so I want to make sure I have it right both ways.

    the struct looks like this:

    #define SP_PACKET_SIZE 200
    #define NAME_SIZE 64
    struct SP_PacketStruct
    {
        int Size;                               ///< Number of items in the data packet
        char Name[SP_PACKET_SIZE][NAME_SIZE];   ///< Data labels
        double Value[SP_PACKET_SIZE];           ///< Data packet values
    };
    

    In my test code I am filling it with only one entry and setting size to 1 as such:

        SP_PacketStruct outPacket;
    
        std::string name = "Test";
        double value = 8.123;
    
        strncpy(outPacket.Name[0], name.c_str(), name.length());
        outPacket.Name[0][name.length()] = '\0';
        outPacket.Value[0] = value;
        outPacket.Size = 1;
    
        QByteArray buf;
        QDataStream s(&buf, QIODevice::WriteOnly);
        s << outPacket.Size << outPacket.Name << outPacket.Value;
        udpSocket->writeDatagram(buf, QHostAddress::LocalHost, 49435);
    }
    

    But when I go to deserialize I get all kinds of trouble with the char [200][64] bit.

        SP_PacketStruct inPacket;
        while (udpSocket->hasPendingDatagrams()) {
            QHostAddress ThisHost;
            quint16 ThisPort;
            QByteArray ThisPacket;
            ThisPacket.resize(udpSocket->pendingDatagramSize());
            udpSocket->readDatagram(ThisPacket.data(), ThisPacket.size(), &ThisHost, &ThisPort);
            QDataStream deserialize(ThisPacket);
            deserialize >> inPacket.Size
                    >> inPacket.Name
                    >> inPacket.Value;
    

    All the trouble begins on the >> inPacket.Name part. where it says:

    mainwindow.cpp:27: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
    /usr/include/qt/QtCore/qdatastream.h:162:18: note: candidate: ‘QDataStream& QDataStream::operator>>(qint8&)’ (near match)
      162 |     QDataStream &operator>>(qint8 &i);
          |                  ^~~~~~~~
    /usr/include/qt/QtCore/qdatastream.h:162:18: note:   conversion of argument 1 would be ill-formed:
    ../mainwindow.cpp:27:29: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
       27 |                 >> inPacket.Name
          |                    ~~~~~~~~~^~~~
          |                             |
          |                             char (*)[64]
    

    Whats the best way to proceed? Especially considering in the final product, QT will not be doing the serialization.

    K 1 Reply Last reply 29 Mar 2022, 13:27
    0
    • R RobbieP
      29 Mar 2022, 13:15

      Hi all! I have a struct I'm trying to deal with...This program needs to primarily deserialize it from other sources, but for testign purposes I'm doing my own serialization in the program so I want to make sure I have it right both ways.

      the struct looks like this:

      #define SP_PACKET_SIZE 200
      #define NAME_SIZE 64
      struct SP_PacketStruct
      {
          int Size;                               ///< Number of items in the data packet
          char Name[SP_PACKET_SIZE][NAME_SIZE];   ///< Data labels
          double Value[SP_PACKET_SIZE];           ///< Data packet values
      };
      

      In my test code I am filling it with only one entry and setting size to 1 as such:

          SP_PacketStruct outPacket;
      
          std::string name = "Test";
          double value = 8.123;
      
          strncpy(outPacket.Name[0], name.c_str(), name.length());
          outPacket.Name[0][name.length()] = '\0';
          outPacket.Value[0] = value;
          outPacket.Size = 1;
      
          QByteArray buf;
          QDataStream s(&buf, QIODevice::WriteOnly);
          s << outPacket.Size << outPacket.Name << outPacket.Value;
          udpSocket->writeDatagram(buf, QHostAddress::LocalHost, 49435);
      }
      

      But when I go to deserialize I get all kinds of trouble with the char [200][64] bit.

          SP_PacketStruct inPacket;
          while (udpSocket->hasPendingDatagrams()) {
              QHostAddress ThisHost;
              quint16 ThisPort;
              QByteArray ThisPacket;
              ThisPacket.resize(udpSocket->pendingDatagramSize());
              udpSocket->readDatagram(ThisPacket.data(), ThisPacket.size(), &ThisHost, &ThisPort);
              QDataStream deserialize(ThisPacket);
              deserialize >> inPacket.Size
                      >> inPacket.Name
                      >> inPacket.Value;
      

      All the trouble begins on the >> inPacket.Name part. where it says:

      mainwindow.cpp:27: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
      /usr/include/qt/QtCore/qdatastream.h:162:18: note: candidate: ‘QDataStream& QDataStream::operator>>(qint8&)’ (near match)
        162 |     QDataStream &operator>>(qint8 &i);
            |                  ^~~~~~~~
      /usr/include/qt/QtCore/qdatastream.h:162:18: note:   conversion of argument 1 would be ill-formed:
      ../mainwindow.cpp:27:29: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
         27 |                 >> inPacket.Name
            |                    ~~~~~~~~~^~~~
            |                             |
            |                             char (*)[64]
      

      Whats the best way to proceed? Especially considering in the final product, QT will not be doing the serialization.

      K Offline
      K Offline
      KroMignon
      wrote on 29 Mar 2022, 13:27 last edited by KroMignon
      #2

      @RobbieP said in Trouble with serializing and deserializing structs:

      Whats the best way to proceed? Especially considering in the final product, QT will not be doing the serialization.

      My way would be to add into SP_PacketStruct a method the serialize and deserialize the messages.
      The way you are doing it is not "clean".
      It is also more C and not C++ style.

      More C++ coding style would be, for me, something like this:

      struct DataPacket
      {
          std::string Name;
          double Value;
      };
      
      typedef std::vector<DataPacket> SP_DataPacket;
      

      It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)

      1 Reply Last reply
      1
      • R Offline
        R Offline
        RobbieP
        wrote on 29 Mar 2022, 13:46 last edited by
        #3

        I am familiar with that...the problem is the struct format is already predefined in other systems and can't change. Some of those systems are actually C based and one of them we don't even have the source code for...so I am unfortunately going to have to maintain that format. Am I screwed?

        K 1 Reply Last reply 29 Mar 2022, 14:18
        0
        • R RobbieP
          29 Mar 2022, 13:46

          I am familiar with that...the problem is the struct format is already predefined in other systems and can't change. Some of those systems are actually C based and one of them we don't even have the source code for...so I am unfortunately going to have to maintain that format. Am I screwed?

          K Offline
          K Offline
          KroMignon
          wrote on 29 Mar 2022, 14:18 last edited by
          #4

          @RobbieP said in Trouble with serializing and deserializing structs:

          I am familiar with that...the problem is the struct format is already predefined in other systems and can't change.

          I understand.
          To come back to the serialization issue, I would do it like this:

          // serialize
          QByteArray buf;
          QDataStream s(&buf, QIODevice::WriteOnly);
          s << outPacket.Size;
          if(outPacket.Size > 0)
          {
              s.writeRawData(outPacket.Name, NAME_SIZE * outPacket.Size);
              s.writeRawData(outPacket.Value, sizeof(double) * outPacket.Size);
          }
          ...
          // de-serialize
          SP_PacketStruct inPacket;
          QDataStream deserialize(buf);
          deserialize >> inPacket.Size;
          if(inPacket.Size > 0)
          {
              deserialize.readRawData(inPacket.Name, NAME_SIZE * inPacket.Size);
              deserialize.readRawData(inPacket.Value, sizeof(double) * inPacket.Size);
          }
          
          

          It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)

          1 Reply Last reply
          1
          • R Offline
            R Offline
            RobbieP
            wrote on 29 Mar 2022, 14:56 last edited by
            #5

            @KroMignon that looks solid, but it's complaining still:

            /mainwindow.cpp:30: error: cannot convert ‘char [200][64]’ to ‘char*’
            ../mainwindow.cpp: In member function ‘void MainWindow::readPendingDatagrams()’:
            ../mainwindow.cpp:30:46: error: cannot convert ‘char [200][64]’ to ‘char*’
               30 |             deserialize.readRawData(inPacket.Name, NAME_SIZE * inPacket.Size);
                  |                                     ~~~~~~~~~^~~~
                  |                                              |
                  |                                              char [200][64]
            /mainwindow.cpp:31: error: cannot convert ‘double [200]’ to ‘char*’
            ../mainwindow.cpp:31:46: error: cannot convert ‘double [200]’ to ‘char*’
               31 |             deserialize.readRawData(inPacket.Value, sizeof(double) * inPacket.Size);
                  |                                     ~~~~~~~~~^~~~~
                  |                                              |
                  |                                              double [200]
            

            I think also it's important to note that there will ALWAYS be 200 entries in the packet, the size entry just tells the processor how many entries are actually filled with usable data.

            K 1 Reply Last reply 29 Mar 2022, 15:03
            0
            • R RobbieP
              29 Mar 2022, 14:56

              @KroMignon that looks solid, but it's complaining still:

              /mainwindow.cpp:30: error: cannot convert ‘char [200][64]’ to ‘char*’
              ../mainwindow.cpp: In member function ‘void MainWindow::readPendingDatagrams()’:
              ../mainwindow.cpp:30:46: error: cannot convert ‘char [200][64]’ to ‘char*’
                 30 |             deserialize.readRawData(inPacket.Name, NAME_SIZE * inPacket.Size);
                    |                                     ~~~~~~~~~^~~~
                    |                                              |
                    |                                              char [200][64]
              /mainwindow.cpp:31: error: cannot convert ‘double [200]’ to ‘char*’
              ../mainwindow.cpp:31:46: error: cannot convert ‘double [200]’ to ‘char*’
                 31 |             deserialize.readRawData(inPacket.Value, sizeof(double) * inPacket.Size);
                    |                                     ~~~~~~~~~^~~~~
                    |                                              |
                    |                                              double [200]
              

              I think also it's important to note that there will ALWAYS be 200 entries in the packet, the size entry just tells the processor how many entries are actually filled with usable data.

              K Offline
              K Offline
              KroMignon
              wrote on 29 Mar 2022, 15:03 last edited by
              #6

              @RobbieP said in Trouble with serializing and deserializing structs:

              I think also it's important to note that there will ALWAYS be 200 entries in the packet, the size entry just tells the processor how many entries are actually filled with usable data.

              I understand this, but it was only to reduce transfer size to only send useful data.

              To solve the issue, you have to cast:

              // serialize
              QByteArray buf;
              QDataStream s(&buf, QIODevice::WriteOnly);
              s << outPacket.Size;
              if(outPacket.Size > 0)
              {
                  s.writeRawData(reinterpret_cast<char *>(outPacket.Name), NAME_SIZE * outPacket.Size);
                  s.writeRawData(reinterpret_cast<char *>(outPacket.Value), sizeof(double) * outPacket.Size);
              }
              ...
              // de-serialize
              SP_PacketStruct inPacket;
              QDataStream deserialize(buf);
              deserialize >> inPacket.Size;
              if(inPacket.Size > 0)
              {
                  deserialize.readRawData(reinterpret_cast<char *>(inPacket.Name), NAME_SIZE * inPacket.Size);
                  deserialize.readRawData(reinterpret_cast<char *>(inPacket.Value), sizeof(double) * inPacket.Size);
              }
              

              It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)

              1 Reply Last reply
              1
              • R Offline
                R Offline
                RobbieP
                wrote on 29 Mar 2022, 15:25 last edited by
                #7

                @KroMignon Oh man...that worked beautifully! And I think I learned a few things here too. Thank you so much for the help! I'll mark this as solved for now but when I start receiving packets from the real systems I may have to bug the forum again ;) Good stuff!

                1 Reply Last reply
                0
                • S Offline
                  S Offline
                  SimonSchroeder
                  wrote on 30 Mar 2022, 08:31 last edited by
                  #8

                  I know that reinterpret_cast works quite well. IIRC the standard does not give you that guarantee, though. Shouldn't we, at least starting with C++20, be using bit_cast instead? I have not switched over to C++20, so I am a little shady on the details. Can anyone chime in if bit_cast would work here (and be possibly more correct)?

                  1 Reply Last reply
                  0

                  4/8

                  29 Mar 2022, 14:18

                  • Login

                  • Login or register to search.
                  4 out of 8
                  • First post
                    4/8
                    Last post
                  0
                  • Categories
                  • Recent
                  • Tags
                  • Popular
                  • Users
                  • Groups
                  • Search
                  • Get Qt Extensions
                  • Unsolved