Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Still having trouble deserializing binary data from another UDP source (non-qt)
Forum Update on Tuesday, May 27th 2025

Still having trouble deserializing binary data from another UDP source (non-qt)

Scheduled Pinned Locked Moved Unsolved General and Desktop
29 Posts 4 Posters 2.7k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J JonB
    10 May 2022, 14:26

    @RobbieP said in Still having trouble deserializing binary data from another UDP source (non-qt):

    Can I just read the data in directly to a struct and assume it would work if the byteorder is the same on both ends?

    Nope, because of possible packing/data alignment issues....

    At the end of the day you must know, somehow, how the data was sent/formatted. You will also need to know if it sends things like ints, and what endianness it does that in. As in: without that you can't read the data back sensibly.

    I don't know whether others would say you can still use QDataStream::readRawData() safely, or if you did whether you are then getting much benefit out of QDataStream anyway.

    If it were me I would just write code to read chunks of bytes as per whatever the sender's format is. Then I know where I am for sure. Qt/C++/C have functions to transform between endian integers without having to use QDataStream. Suggest do not read into a struct directly, read into each member separately as per what you were doing. Then alignment at receiving size won't be an issue. But without looking at sending code's side, can't be sure whether that might have padding/alignment. If we assume sender itself did not e.g. send any structs as a whole you may be good.

    Off you go, and do lots of testing on sender's packets!

    I've never deserialized without qt

    OK, if you really want to try sticking with it. Just tell your receiver QDataStream to be little-endian. See whether that sorts the current issue to your satisfaction. If so, try lots of others. If they all work, you can stick with QDataStream I guess....

    R Offline
    R Offline
    RobbieP
    wrote on 10 May 2022, 14:58 last edited by
    #20

    @JonB I'm getting the correct names and the size...the doubles are off though...does the second readRawData start where the previous readRawData left off? calling readRawData back to back like that doesn't reset the position does it?

    1 Reply Last reply
    0
    • J JonB
      10 May 2022, 14:26

      @RobbieP said in Still having trouble deserializing binary data from another UDP source (non-qt):

      Can I just read the data in directly to a struct and assume it would work if the byteorder is the same on both ends?

      Nope, because of possible packing/data alignment issues....

      At the end of the day you must know, somehow, how the data was sent/formatted. You will also need to know if it sends things like ints, and what endianness it does that in. As in: without that you can't read the data back sensibly.

      I don't know whether others would say you can still use QDataStream::readRawData() safely, or if you did whether you are then getting much benefit out of QDataStream anyway.

      If it were me I would just write code to read chunks of bytes as per whatever the sender's format is. Then I know where I am for sure. Qt/C++/C have functions to transform between endian integers without having to use QDataStream. Suggest do not read into a struct directly, read into each member separately as per what you were doing. Then alignment at receiving size won't be an issue. But without looking at sending code's side, can't be sure whether that might have padding/alignment. If we assume sender itself did not e.g. send any structs as a whole you may be good.

      Off you go, and do lots of testing on sender's packets!

      I've never deserialized without qt

      OK, if you really want to try sticking with it. Just tell your receiver QDataStream to be little-endian. See whether that sorts the current issue to your satisfaction. If so, try lots of others. If they all work, you can stick with QDataStream I guess....

      R Offline
      R Offline
      RobbieP
      wrote on 10 May 2022, 15:02 last edited by
      #21

      @JonB I ask because I'm curious if it's possible the second readRawData isn't tryign to convert the 200 64 character names into doubles.

      J 1 Reply Last reply 10 May 2022, 15:11
      0
      • R RobbieP
        10 May 2022, 15:02

        @JonB I ask because I'm curious if it's possible the second readRawData isn't tryign to convert the 200 64 character names into doubles.

        J Offline
        J Offline
        JonB
        wrote on 10 May 2022, 15:11 last edited by JonB 5 Oct 2022, 15:12
        #22

        @RobbieP
        No, your readRawData()s run sequentially, carrying on from where they got to.

        Possibilities:

        • Sender does have some pad between end of Name and start of Value.
        • Sender saves doubles as something other than write(&double, sizeof(double)).
        • Sender is different architecture which represents doubles differently. Though I kind of doubt this, I believe there is some standard for how doubles are represented which is cross-architecture.
        • QDataStream encodes double with some extra information. Though I doubt this.

        the doubles are off though

        How "off" is "off"? Slightly wrong, or way out there?

        Suggestions:

        • You really should put those checks in on return result of QDataStream::readRawData().
        • Can you get the sender to send just with inPacket.Size = 1 while you get it sorted out?
        • Try it with plain read()s and no QDataStream. Though if that does not work you're a bit stuck.
        • Do you have the source code of the sender's write()ing code to examine?
        R 1 Reply Last reply 10 May 2022, 15:30
        0
        • J JonB
          10 May 2022, 15:11

          @RobbieP
          No, your readRawData()s run sequentially, carrying on from where they got to.

          Possibilities:

          • Sender does have some pad between end of Name and start of Value.
          • Sender saves doubles as something other than write(&double, sizeof(double)).
          • Sender is different architecture which represents doubles differently. Though I kind of doubt this, I believe there is some standard for how doubles are represented which is cross-architecture.
          • QDataStream encodes double with some extra information. Though I doubt this.

          the doubles are off though

          How "off" is "off"? Slightly wrong, or way out there?

          Suggestions:

          • You really should put those checks in on return result of QDataStream::readRawData().
          • Can you get the sender to send just with inPacket.Size = 1 while you get it sorted out?
          • Try it with plain read()s and no QDataStream. Though if that does not work you're a bit stuck.
          • Do you have the source code of the sender's write()ing code to examine?
          R Offline
          R Offline
          RobbieP
          wrote on 10 May 2022, 15:30 last edited by
          #23

          @JonB

          /* This method writes data to the UDP socket and returns the number of
          * bytes written or -1 for an error.  Pass in the IP and port youare sending to. */
          
          int PCSBSocketUDP::WriteData(
          				void *				inBuf,
          				int				inBufLength,
          				unsigned int		inDstIP,
          				unsigned short		inDstPort)
          {
          	sockaddr_in destAddress;
          	int tolen = sizeof(destAddress);
          	destAddress.sin_family = AF_INET;
          	destAddress.sin_port = htons(inDstPort);
          	destAddress.sin_addr.s_addr = htonl(inDstIP);
          	int result = sendto(mWinSocket,
          						 (char*)inBuf,
          						 inBufLength,
          						 0,
          						 (sockaddr*)&destAddress,
          						 tolen);
          
          	if (result == -1)
          	{
          		return -1;
          	}
          	else
          	{
          		return result;
          	}
          }
          

          and I'm sending the struct from the top of my first post with that using:

          m_SendSocket->WriteData((char *)&m_Packet, sizeof (fennecPacketStruct), ntohl(inet_addr(m_ipaddress.c_str())), 46456);
          

          the sender is running on this same linux machine, but it's a plugin to a flight simulator. I'll see what the return value is for readRawData next.

          J 1 Reply Last reply 10 May 2022, 15:39
          0
          • R RobbieP
            10 May 2022, 15:30

            @JonB

            /* This method writes data to the UDP socket and returns the number of
            * bytes written or -1 for an error.  Pass in the IP and port youare sending to. */
            
            int PCSBSocketUDP::WriteData(
            				void *				inBuf,
            				int				inBufLength,
            				unsigned int		inDstIP,
            				unsigned short		inDstPort)
            {
            	sockaddr_in destAddress;
            	int tolen = sizeof(destAddress);
            	destAddress.sin_family = AF_INET;
            	destAddress.sin_port = htons(inDstPort);
            	destAddress.sin_addr.s_addr = htonl(inDstIP);
            	int result = sendto(mWinSocket,
            						 (char*)inBuf,
            						 inBufLength,
            						 0,
            						 (sockaddr*)&destAddress,
            						 tolen);
            
            	if (result == -1)
            	{
            		return -1;
            	}
            	else
            	{
            		return result;
            	}
            }
            

            and I'm sending the struct from the top of my first post with that using:

            m_SendSocket->WriteData((char *)&m_Packet, sizeof (fennecPacketStruct), ntohl(inet_addr(m_ipaddress.c_str())), 46456);
            

            the sender is running on this same linux machine, but it's a plugin to a flight simulator. I'll see what the return value is for readRawData next.

            J Offline
            J Offline
            JonB
            wrote on 10 May 2022, 15:39 last edited by JonB 5 Oct 2022, 15:45
            #24

            @RobbieP said in Still having trouble deserializing binary data from another UDP source (non-qt):

            m_SendSocket->WriteData((char *)&m_Packet, sizeof (fennecPacketStruct), ntohl(inet_addr(m_ipaddress.c_str())), 46456);

            Since this sends a fennecPacketStruct in one chunk of bytes, why in the world are you trying to deserialize field-by-field? Assuming same machine architecture and same compiler and same definition of fennecPacketStruct with same constants for sizes, you want to just do the opposite read, like read(&inPacket, sizeof(fennecPacketStruct)).

            Meanwhile I am "unconvinced" your sender size really has an identical declaration of struct fennecPacketStruct as you showed? Or does it? That uses a fixed size SP_PACKET_SIZE constant. But earlier your code read inPacket.Size from the data stream, and used that to affect how much it read where. Something is fishy...

            1 Reply Last reply
            0
            • R Offline
              R Offline
              RobbieP
              wrote on 10 May 2022, 15:43 last edited by
              #25

              That's what I was originally thinking a long time ago, but I can't seem to do it...it gives me this error:

              /home/robbie/Documents/Programming/Fennec/mainwindow.cpp:36: error: no matching function for call to ‘QUdpSocket::read(MainWindow::fennecPacketStruct*, long unsigned int)’
              ../Fennec/mainwindow.cpp: In member function ‘void MainWindow::readPendingDatagrams()’:
              ../Fennec/mainwindow.cpp:36:24: error: no matching function for call to ‘QUdpSocket::read(MainWindow::fennecPacketStruct*, long unsigned int)’
              36 | udpSocket->read(&inPacket, sizeof(fennecPacketStruct));
              | ~^~~~~~~~~~~~~~~~~~~~~~~~~

              J 1 Reply Last reply 10 May 2022, 15:49
              0
              • R RobbieP
                10 May 2022, 15:43

                That's what I was originally thinking a long time ago, but I can't seem to do it...it gives me this error:

                /home/robbie/Documents/Programming/Fennec/mainwindow.cpp:36: error: no matching function for call to ‘QUdpSocket::read(MainWindow::fennecPacketStruct*, long unsigned int)’
                ../Fennec/mainwindow.cpp: In member function ‘void MainWindow::readPendingDatagrams()’:
                ../Fennec/mainwindow.cpp:36:24: error: no matching function for call to ‘QUdpSocket::read(MainWindow::fennecPacketStruct*, long unsigned int)’
                36 | udpSocket->read(&inPacket, sizeof(fennecPacketStruct));
                | ~^~~~~~~~~~~~~~~~~~~~~~~~~

                J Offline
                J Offline
                JonB
                wrote on 10 May 2022, 15:49 last edited by
                #26

                @RobbieP
                Of course that won't match. The writer code does not use fennecPacketStruct in WriteData() either. For qint64 QIODevice::read(char *data, qint64 maxSize) it expects a char *, so cast it....

                R 2 Replies Last reply 10 May 2022, 15:53
                1
                • J JonB
                  10 May 2022, 15:49

                  @RobbieP
                  Of course that won't match. The writer code does not use fennecPacketStruct in WriteData() either. For qint64 QIODevice::read(char *data, qint64 maxSize) it expects a char *, so cast it....

                  R Offline
                  R Offline
                  RobbieP
                  wrote on 10 May 2022, 15:53 last edited by
                  #27

                  @JonB the writer is using fennecPacketStruct though isn't it? m_Packet is a fennecPacketStruct, and the writer is sending sizeof(fennecPacketStruct).

                  Do I cast it in the read block using another reinterpret_cast? I'm not sure how that would work, because even casting it doesn't seem to like it. Also I need to use ReadDataGram, not Read because I need the host and port that it's coming from.

                  1 Reply Last reply
                  0
                  • J JonB
                    10 May 2022, 15:49

                    @RobbieP
                    Of course that won't match. The writer code does not use fennecPacketStruct in WriteData() either. For qint64 QIODevice::read(char *data, qint64 maxSize) it expects a char *, so cast it....

                    R Offline
                    R Offline
                    RobbieP
                    wrote on 10 May 2022, 15:57 last edited by
                    #28

                    @JonB got it!

                    the correct cast was (char *)&inPacket

                    All my values are now correct!

                    I'll do some sanity checking and add some value logs as well to make sure this is goign to stay consistent in the lab. Thanks much for your incredible patience and help.

                    J 1 Reply Last reply 10 May 2022, 16:04
                    0
                    • R RobbieP
                      10 May 2022, 15:57

                      @JonB got it!

                      the correct cast was (char *)&inPacket

                      All my values are now correct!

                      I'll do some sanity checking and add some value logs as well to make sure this is goign to stay consistent in the lab. Thanks much for your incredible patience and help.

                      J Offline
                      J Offline
                      JonB
                      wrote on 10 May 2022, 16:04 last edited by JonB 5 Oct 2022, 16:05
                      #29

                      @RobbieP said in Still having trouble deserializing binary data from another UDP source (non-qt):

                      the correct cast was (char *)&inPacket

                      Yeah. Which should have gone through as reinterpret_cast<char *>(&inPacket) ? Not that it's much different, but keeps the wolves at bay.

                      the writer is using fennecPacketStruct though isn't it?

                      But not in the definition of PCSBSocketUDP::WriteData(void *inBuf, ...), or sendTo(..., (char*)inBuf), which are the equivalent of your udpSocket->read(&inPacket, ...).

                      1 Reply Last reply
                      0

                      29/29

                      10 May 2022, 16:04

                      • Login

                      • Login or register to search.
                      29 out of 29
                      • First post
                        29/29
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Users
                      • Groups
                      • Search
                      • Get Qt Extensions
                      • Unsolved