QTCP and packet parsing



  • I am working on a server that is simply recvieing traffic from a client. I am using QT5.9.2 and MSVS 2017 64bit complier to recv the data and parse the data and read it and send it to the client.

    So this is my code so far..

    void Server::onReadyRead(QObject *socketObject)
    {
        QTcpSocket *socket = qobject_cast<QTcpSocket *>(socketObject);
        if (!socket || !socket->bytesAvailable())
        {
            qDebug() << "socket bytes unavailable";
            return;
        }
        qDebug() << "recvd signal data. bytesAvailable:" << socket->bytesAvailable();
    
        // Recvd the entire buffer from bytesAvailable()
        recvdbuffer = socket->readAll();
    
        if(recvdbuffer.isEmpty())
        {
    		qDebug() << "recvdbuffer has no bytes available";
            return;
        }
    
        qDebug() << "recvbuffer in hex and size:" << recvdbuffer.toHex() << recvdbuffer.size();
    
        if(recvdbuffer.length() > MAX_BUFFER )
        {
            socket->close();
            return;
        }
        recdPacket(recvbuffer.toHex(0), recvbuffer[1], recvbuffer.size());
    }
    

    so i have a packet that looks like this...

    ce06000038000000010000000000000068656c6c6f616c6c00
    

    How would i be able to split this data up into pieces. The recvbuffer.toHex(0) prints it out to hex do i need to convert it?

    how to convert hex to string?



  • @Jonnybravo said in QTCP and packet parsing:

    how to convert hex to string?

    QByteArray hex =  QByteArray::fromHex("ce06000038000000010000000000000068656c6c6f616c6c00");
    QString str2 = QString::fromUtf16((char16_t*)hex.data());
    


  • @Taz742 It doesn't do what you are expecting.

    QByteArray hex =  QByteArray::fromHex("ce06000038000000010000000000000068656c6c6f616c6c00");
    QString str2 = QString::fromUtf16((char16_t*)hex.data());
    qDebug() << str2;
    

    this returns "?"

    instead of "helloall" the string inside the hex..

    only thing i can think of is looping thru the array base on a postion and return the buffer..


  • Lifetime Qt Champion

    Hi and welcome to devnet,

    In what form are you supposed to receive your data ?
    Is there any protocol between your devices ?



  • @SGaist Thankyou. Its just a normal tcp connection the recvdbuffer = socket->readAll(); returns the entire packet in full i'm now just trying to extract the data from the array.

    ce
    06
    0000
    38
    00000001
    00000000000000
    68656c6c6f616c6c
    00
    

    So as you can see you can split up the packet into different type of values. You can normally just create a structure out of this and parse it to the structure but i was wondering if there are other ways.

    The packet would look like this...

    struct information 
    {
     qint8 header
    qint8 opcode
    qint16 data
    qint8 size
    qint32 data
    qint8 ID
    qint64 data[7]
    QString message
    qint end
    } ;
    
    Someting like that i guess....

  • Lifetime Qt Champion

    Sounds like you should be using QDataStream and implement the related streaming operators for your structure.



  • is there a way to do it via array or do i have to build my own template that will cast a vector buffer and loop thru the vector base on the size and location i give it



  • @Jonnybravo
    not need array.
    QDataStream are best way, for my point of view.
    Here you can see how to use QDataStream
    https://forum.qt.io/topic/84222/qdatastream-serialze-deserialize-myclass-pointer/2
    It would be nice if you read everything in the end.



  • Let's imagine your ce06000038000000010000000000000068656c6c6f616c6c00 is one 32bit signed integer and a QString.

    QTcpSocket *socket = qobject_cast<QTcpSocket *>(socketObject);
    if(!socket) return;
    QDataStream streamer(socket);
    streamer.startTransaction();
    qint32 val1;
    QString val2;
    streamer >> val1 >> val2;
    if(streamer.commitTransaction()){
    qDebug() << "Read successful: " << val1 << val2;
    }
    


  • The only issue with datastreams is that the data is always changing very rarely do you see the same data twice. So i understand that hes breaking up the data into variables.

    qint32 val1;
    QString val2;
    streamer >> val1 >> val2;
    

    Before in my C++ app i would create a unsigned char vector resize the vector to the recvd size of the socket create a template that would loop thru the position base on the value i sent to it.

    its mostly about being able to recv the data verity it on my end and send back what i need.



  • @Jonnybravo I don't think I understand what you mean


  • Lifetime Qt Champion

    Isn't the information structure what you are sending around ?



  • The information is not known i am just recving traffic from a tcp connection and trying to parse the data so that i can read it and check its validations on my end before replying back.

    //Something like this i can do
    //ce06000038000000010000000000000068656c6c6f616c6c00
    qint8 header = recvdbuffer[0];
    qint8 opcode = recvdbuffer[1];
    qint8 size = recvdbuffer[4];
    for(unsigned i = 0; i < recvdbuffer.length(); i++ )
    {
    // grab the string from the array
    }
    

  • Lifetime Qt Champion

    How do you know when you received a full frame of data ?



  • The client builds the packet into one big buffer and sends it. On the server side i know the size and most data in the frame.

    Qt just has a crap way of being able to have a dynamic way of extracting data from a socket.

    The only methods i can think of off the top of my head is use qdatastream that is built on top of the data structure of what you are reading then pass the infomation into arguments later on to be used for whatever you need it for.

    example:

    QTcpSocket *socket = qobject_cast<QTcpSocket *>(socketObject);
    if(!socket) return;
    QDataStream streamer(socket);
    streamer.startTransaction();
    
    streamer >> val1 >> val2 >> val3 >> val4;
    
    switch (val2)
    case: // do what you need with val2 with a function that takes arguments val4 and val5
    break;
    

  • Lifetime Qt Champion

    If your client is sending the information structure, why not implement the stream operators for that structure directly ? That way you'll have directly an information object to handle on the receiving end.



  • You can always use startTransaction on the socket without using QDataStream



  • Could you tell me what im doing wrong here.. (qbyte array to qvector)

    So ive created a class that gets data from a struct vector buffer

    struct Packet
    {
        public:
            QVector<quint8>Buffer;
            quint8 Header;
            quint8 OpCode;
    
    public:
        Packet( ) {}
        Packet( int size ) : Buffer(size){ }
    
    	/* Read values from packet */
        template <typename T> T Get( quint32 pos )
    	{
    		if( Buffer.size() < pos+sizeof(T) ) 
                          return 0;
    		return *((T*)&Buffer[pos]);
    	}
        quint8 GetByte( quint32 pos )
    	{
            return Get<quint8>( pos );
    	}
        quint16 GetWord( quint32 pos )
    	{
            return Get<quint16>( pos );
    	}
        unsigned GetDWord( quint32 pos )
    	{
    		return Get<unsigned>( pos );
    	}
        quint64 GetQWord( quint32 pos )
    	{
            return Get<quint64>( pos );
    	}
        float GetFloat( quint32 pos )
    	{
    		return Get<float>( pos );
    	}
        void GetString( quint32 pos, char* buffer )
    	{	  
    		unsigned strLen = GetDWord( pos );
    		for(unsigned i = 0; i < strLen; i++ )
    		{
                buffer[i] = Get<quint8>(pos+4+i);
    		}
    		buffer[strLen] = 0;
    	}
    };
    

    So i am now trying to convert the data from qbyte to vector.

    // Recv the entire buffer from bytesAvailable()
    recvbuffer = socket->readAll();
    Packet m_pak(recvbuffer.size());
    
    QDataStream out(&recvbuffer ,QIODevice::WriteOnly);
    out << m_pak.Buffer;
    
    //doesnt fill the buffer with the qbytearray..
    

    Now should i just instead of using datastream just resize it and insert the qbytearray into the vector.

    recvbuffer = socket->readAll();
    Packet m_pak(MAX_BUFFER);
    m_pak.Buffer.resize(m_pak.Buffer.size() + recvbuffer.size());
    memcpy(&m_pak.Buffer[m_pak.Buffer.size() - recvbuffer.size()], &recvbuffer[0], recvbuffer.size() * sizeof(int));
    

    doesnt work..

    recvbuffer = socket->readAll();
    Packet m_pak(MAX_BUFFER);
    m_pak.Buffer.reserve(m_pak.Buffer.size() + recvbuffer.size());
    qCopy(&recvbuffer[0], &recvbuffer[recvbuffer.size()], std::back_inserter(m_pak.Buffer));
    //crashes due to the inserter copy.
    

    Any ideas in how to fix this or what i should do...


  • Lifetime Qt Champion

    How do you know you received the full data ?



  • i do a checksum on the packet size and of that of the header. if the data matches to what it should be i parse it..

    any sugguestions how to fix my issue?


  • Moderators

    @Jonnybravo Why do you want to use QVector for your buffer instead of QByteArray?



  • @Jonnybravo

    I don't quite get why you're doing this, but this, should work:

    m_pak.Buffer.clear();
    for(int i(0); i < recvbuffer .size(); i++)
        m_pak.Buffer.append(static_cast<int>(recvbuffer.at(i));) 
    

    I'm not sure if you'll need the cast, but I added it, just to be sure ;-)


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.