[solved][tcp][debian] QTcpSocket read buffer size

[kolegs]

Hi,

I got a problem, I am trying to read data from tcp socket and it works well until I received a lot of data.

My tcp socket buffer cannot hold more than 4096 bytes which is wired. I already check the value in QTcpSocket class and it is set to 0(unlimited). I thought it might be some kind of debian issue and tweaked the values in /proc/sys/net/ipv4/tcp_rmem(nothing changed).

Any ideas? Can it be Qt related?(If it would be system related probably I woludn't be possible to load large images on my apache2 server but not sure)

[kolegs]

Ok, posting the code here made me find the mistake. So thanks guys.

There is a bug in a code

    if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){

Should be

    if(m_socket->bytesAvailable() >= m_msgSize && m_msgSize > 0){

Note the ">=" instead if "<="
It worked for smaller commands since tcp layer did not split it.

Thanks again :D

[SGaist]

Hi,

Can you show how you are using QTcpSocket ?

[kolegs]

Sure, this is how I create it

m_socket = new QTcpSocket(this);
    qDebug() << "Buffer size: " << m_socket->readBufferSize();

    // set the ID
    if(!m_socket->setSocketDescriptor(m_socketDescriptor))
    {
        // something's wrong, we just emit a signal
        emit error(m_socket->errorString());
        return;
    }

    //Connect signals
    connect(m_socket, SIGNAL(disconnected()),
            this, SLOT(disconnected()));
    connect(m_socket, SIGNAL(readyRead()),
            this, SLOT(readyRead()));

Later in my readyRead slot I read first 4 bytes(it is message size and wait for rest of data to arrive)

//Reset timer last command timer
    lastCommandTimer->start();

    //Check if waiting for data
    if(m_msgSize == 0){ //No data length
        qDebug() << "Bytes available:" << m_socket->bytesAvailable(); //In this line it is never bigger than 4096!!!
        //Message too short to get data length
        if(m_socket->bytesAvailable() < 4)
            return;
        //Read message length
        QByteArray msgSizeArray = m_socket->read(4);
        QDataStream ds(&msgSizeArray, QIODevice::ReadOnly);
        ds >> m_msgSize;
        qDebug() << "Ready read size:" << m_msgSize;
    }

    //Got message length, waiting for whole message
    if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){
        //Got whole message read it
        QByteArray msg = m_socket->read(m_msgSize);
        //Process command
        //...
}

EDIT: BTW I use Qt 5.9,
Since bufferSize in maximum is 4096 when I send to my server bigger message my communication gets corrupted, I miss a lot of data.

[VRonin]

It is quite strange tbh but, in any case, you can build the buffer yourself.

• declare a private QByteArray
• Every time you get the readyRead() just dump everything in the QByteArray (m_byteArray.append(m_socket.readAll());)
• do whatever you are doing now but use the m_byteArray as source of data instead of the m_socket

[kolegs]

Ok, posting the code here made me find the mistake. So thanks guys.

There is a bug in a code

    if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){

Should be

    if(m_socket->bytesAvailable() >= m_msgSize && m_msgSize > 0){

Note the ">=" instead if "<="
It worked for smaller commands since tcp layer did not split it.

Thanks again :D

[J.Hilk]

@kolegs
4096 makes kind of sense, prob 4 Bytes reserved (for internal stuff) and 12 for data storage.

IIRC, than the MTU (Maximum Transmission Unit) for Ethernet, is 1500 bytes. so you should be unable to run into overflow imediately. And can save the data in a Private/Protected QByteArray, like @VRonin suggested.

additionally to that,
you're reading parts of the buffer each cycle

QByteArray msgSizeArray = m_socket->read(4);

AFAIK that removes those bytes from the buffer and your data should get corrupted

[kolegs]

@J-Hilk Notice I only read this 4 bytes if I does not have the size

if(m_msgSize == 0){
...
        //Read message length
        QByteArray msgSizeArray = m_socket->read(4);
        QDataStream ds(&msgSizeArray, QIODevice::ReadOnly);
        ds >> m_msgSize;
...
}

If I got a message length I wait for the data to arrive. m_msgSize is a quint32 private class member, so it is not 0 each time I get into readyRead slot.

[SGaist]

@kolegs Unrelated but you can now mark the thread as solved using the "Topic Tools" button rather than modify your thread's title :)

[VRonin]

Partially related.
If you use QDataStream for all your read operations and Qt 5.7+ you can use transactions to avoid you doing the manual check of bytesAvailable()