Trouble with serializing and deserializing structs

RobbieP

Hi all! I have a struct I'm trying to deal with...This program needs to primarily deserialize it from other sources, but for testign purposes I'm doing my own serialization in the program so I want to make sure I have it right both ways.

the struct looks like this:

#define SP_PACKET_SIZE 200
#define NAME_SIZE 64
struct SP_PacketStruct
{
    int Size;                               ///< Number of items in the data packet
    char Name[SP_PACKET_SIZE][NAME_SIZE];   ///< Data labels
    double Value[SP_PACKET_SIZE];           ///< Data packet values
};

In my test code I am filling it with only one entry and setting size to 1 as such:

    SP_PacketStruct outPacket;

    std::string name = "Test";
    double value = 8.123;

    strncpy(outPacket.Name[0], name.c_str(), name.length());
    outPacket.Name[0][name.length()] = '\0';
    outPacket.Value[0] = value;
    outPacket.Size = 1;

    QByteArray buf;
    QDataStream s(&buf, QIODevice::WriteOnly);
    s << outPacket.Size << outPacket.Name << outPacket.Value;
    udpSocket->writeDatagram(buf, QHostAddress::LocalHost, 49435);
}

But when I go to deserialize I get all kinds of trouble with the char [200][64] bit.

    SP_PacketStruct inPacket;
    while (udpSocket->hasPendingDatagrams()) {
        QHostAddress ThisHost;
        quint16 ThisPort;
        QByteArray ThisPacket;
        ThisPacket.resize(udpSocket->pendingDatagramSize());
        udpSocket->readDatagram(ThisPacket.data(), ThisPacket.size(), &ThisHost, &ThisPort);
        QDataStream deserialize(ThisPacket);
        deserialize >> inPacket.Size
                >> inPacket.Name
                >> inPacket.Value;

All the trouble begins on the >> inPacket.Name part. where it says:

mainwindow.cpp:27: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
/usr/include/qt/QtCore/qdatastream.h:162:18: note: candidate: ‘QDataStream& QDataStream::operator>>(qint8&)’ (near match)
  162 |     QDataStream &operator>>(qint8 &i);
      |                  ^~~~~~~~
/usr/include/qt/QtCore/qdatastream.h:162:18: note:   conversion of argument 1 would be ill-formed:
../mainwindow.cpp:27:29: error: invalid conversion from ‘char (*)[64]’ to ‘qint8’ {aka ‘signed char’} [-fpermissive]
   27 |                 >> inPacket.Name
      |                    ~~~~~~~~~^~~~
      |                             |
      |                             char (*)[64]

Whats the best way to proceed? Especially considering in the final product, QT will not be doing the serialization.

KroMignon

@RobbieP said in Trouble with serializing and deserializing structs:

Whats the best way to proceed? Especially considering in the final product, QT will not be doing the serialization.

My way would be to add into SP_PacketStruct a method the serialize and deserialize the messages.
The way you are doing it is not "clean".
It is also more C and not C++ style.

More C++ coding style would be, for me, something like this:

struct DataPacket
{
    std::string Name;
    double Value;
};

typedef std::vector<DataPacket> SP_DataPacket;

RobbieP

I am familiar with that...the problem is the struct format is already predefined in other systems and can't change. Some of those systems are actually C based and one of them we don't even have the source code for...so I am unfortunately going to have to maintain that format. Am I screwed?

KroMignon

@RobbieP said in Trouble with serializing and deserializing structs:

I am familiar with that...the problem is the struct format is already predefined in other systems and can't change.

I understand.
To come back to the serialization issue, I would do it like this:

// serialize
QByteArray buf;
QDataStream s(&buf, QIODevice::WriteOnly);
s << outPacket.Size;
if(outPacket.Size > 0)
{
    s.writeRawData(outPacket.Name, NAME_SIZE * outPacket.Size);
    s.writeRawData(outPacket.Value, sizeof(double) * outPacket.Size);
}
...
// de-serialize
SP_PacketStruct inPacket;
QDataStream deserialize(buf);
deserialize >> inPacket.Size;
if(inPacket.Size > 0)
{
    deserialize.readRawData(inPacket.Name, NAME_SIZE * inPacket.Size);
    deserialize.readRawData(inPacket.Value, sizeof(double) * inPacket.Size);
}

RobbieP

@KroMignon that looks solid, but it's complaining still:

/mainwindow.cpp:30: error: cannot convert ‘char [200][64]’ to ‘char*’
../mainwindow.cpp: In member function ‘void MainWindow::readPendingDatagrams()’:
../mainwindow.cpp:30:46: error: cannot convert ‘char [200][64]’ to ‘char*’
   30 |             deserialize.readRawData(inPacket.Name, NAME_SIZE * inPacket.Size);
      |                                     ~~~~~~~~~^~~~
      |                                              |
      |                                              char [200][64]
/mainwindow.cpp:31: error: cannot convert ‘double [200]’ to ‘char*’
../mainwindow.cpp:31:46: error: cannot convert ‘double [200]’ to ‘char*’
   31 |             deserialize.readRawData(inPacket.Value, sizeof(double) * inPacket.Size);
      |                                     ~~~~~~~~~^~~~~
      |                                              |
      |                                              double [200]

I think also it's important to note that there will ALWAYS be 200 entries in the packet, the size entry just tells the processor how many entries are actually filled with usable data.

KroMignon

@RobbieP said in Trouble with serializing and deserializing structs:

I think also it's important to note that there will ALWAYS be 200 entries in the packet, the size entry just tells the processor how many entries are actually filled with usable data.

I understand this, but it was only to reduce transfer size to only send useful data.

To solve the issue, you have to cast:

// serialize
QByteArray buf;
QDataStream s(&buf, QIODevice::WriteOnly);
s << outPacket.Size;
if(outPacket.Size > 0)
{
    s.writeRawData(reinterpret_cast<char *>(outPacket.Name), NAME_SIZE * outPacket.Size);
    s.writeRawData(reinterpret_cast<char *>(outPacket.Value), sizeof(double) * outPacket.Size);
}
...
// de-serialize
SP_PacketStruct inPacket;
QDataStream deserialize(buf);
deserialize >> inPacket.Size;
if(inPacket.Size > 0)
{
    deserialize.readRawData(reinterpret_cast<char *>(inPacket.Name), NAME_SIZE * inPacket.Size);
    deserialize.readRawData(reinterpret_cast<char *>(inPacket.Value), sizeof(double) * inPacket.Size);
}

RobbieP

@KroMignon Oh man...that worked beautifully! And I think I learned a few things here too. Thank you so much for the help! I'll mark this as solved for now but when I start receiving packets from the real systems I may have to bug the forum again ;) Good stuff!

SimonSchroeder

I know that reinterpret_cast works quite well. IIRC the standard does not give you that guarantee, though. Shouldn't we, at least starting with C++20, be using bit_cast instead? I have not switched over to C++20, so I am a little shady on the details. Can anyone chime in if bit_cast would work here (and be possibly more correct)?