[solved][tcp][debian] QTcpSocket read buffer size



  • Hi,

    I got a problem, I am trying to read data from tcp socket and it works well until I received a lot of data.

    My tcp socket buffer cannot hold more than 4096 bytes which is wired. I already check the value in QTcpSocket class and it is set to 0(unlimited). I thought it might be some kind of debian issue and tweaked the values in /proc/sys/net/ipv4/tcp_rmem(nothing changed).

    Any ideas? Can it be Qt related?(If it would be system related probably I woludn't be possible to load large images on my apache2 server but not sure)


  • Lifetime Qt Champion

    Hi,

    Can you show how you are using QTcpSocket ?



  • Sure, this is how I create it

    m_socket = new QTcpSocket(this);
        qDebug() << "Buffer size: " << m_socket->readBufferSize();
    
        // set the ID
        if(!m_socket->setSocketDescriptor(m_socketDescriptor))
        {
            // something's wrong, we just emit a signal
            emit error(m_socket->errorString());
            return;
        }
    
        //Connect signals
        connect(m_socket, SIGNAL(disconnected()),
                this, SLOT(disconnected()));
        connect(m_socket, SIGNAL(readyRead()),
                this, SLOT(readyRead()));
    
    

    Later in my readyRead slot I read first 4 bytes(it is message size and wait for rest of data to arrive)

    //Reset timer last command timer
        lastCommandTimer->start();
    
        //Check if waiting for data
        if(m_msgSize == 0){ //No data length
            qDebug() << "Bytes available:" << m_socket->bytesAvailable(); //In this line it is never bigger than 4096!!!
            //Message too short to get data length
            if(m_socket->bytesAvailable() < 4)
                return;
            //Read message length
            QByteArray msgSizeArray = m_socket->read(4);
            QDataStream ds(&msgSizeArray, QIODevice::ReadOnly);
            ds >> m_msgSize;
            qDebug() << "Ready read size:" << m_msgSize;
        }
    
        //Got message length, waiting for whole message
        if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){
            //Got whole message read it
            QByteArray msg = m_socket->read(m_msgSize);
            //Process command
            //...
    }
    

    EDIT: BTW I use Qt 5.9,
    Since bufferSize in maximum is 4096 when I send to my server bigger message my communication gets corrupted, I miss a lot of data.



  • It is quite strange tbh but, in any case, you can build the buffer yourself.

    • declare a private QByteArray
    • Every time you get the readyRead() just dump everything in the QByteArray (m_byteArray.append(m_socket.readAll());)
    • do whatever you are doing now but use the m_byteArray as source of data instead of the m_socket


  • Ok, posting the code here made me find the mistake. So thanks guys.

    There is a bug in a code

        if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){
    

    Should be

        if(m_socket->bytesAvailable() >= m_msgSize && m_msgSize > 0){
    

    Note the ">=" instead if "<="
    It worked for smaller commands since tcp layer did not split it.

    Thanks again :D



  • @kolegs
    4096 makes kind of sense, prob 4 Bytes reserved (for internal stuff) and 12 for data storage.

    IIRC, than the MTU (Maximum Transmission Unit) for Ethernet, is 1500 bytes. so you should be unable to run into overflow imediately. And can save the data in a Private/Protected QByteArray, like @VRonin suggested.

    additionally to that,
    you're reading parts of the buffer each cycle

    QByteArray msgSizeArray = m_socket->read(4);
    

    AFAIK that removes those bytes from the buffer and your data should get corrupted



  • @J-Hilk Notice I only read this 4 bytes if I does not have the size

    if(m_msgSize == 0){
    ...
            //Read message length
            QByteArray msgSizeArray = m_socket->read(4);
            QDataStream ds(&msgSizeArray, QIODevice::ReadOnly);
            ds >> m_msgSize;
    ...
    }
    

    If I got a message length I wait for the data to arrive. m_msgSize is a quint32 private class member, so it is not 0 each time I get into readyRead slot.


  • Lifetime Qt Champion

    @kolegs Unrelated but you can now mark the thread as solved using the "Topic Tools" button rather than modify your thread's title :)



  • Partially related.
    If you use QDataStream for all your read operations and Qt 5.7+ you can use transactions to avoid you doing the manual check of bytesAvailable()


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.