[solved][tcp][debian] QTcpSocket read buffer size
-
Hi,
I got a problem, I am trying to read data from tcp socket and it works well until I received a lot of data.
My tcp socket buffer cannot hold more than 4096 bytes which is wired. I already check the value in QTcpSocket class and it is set to 0(unlimited). I thought it might be some kind of debian issue and tweaked the values in /proc/sys/net/ipv4/tcp_rmem(nothing changed).
Any ideas? Can it be Qt related?(If it would be system related probably I woludn't be possible to load large images on my apache2 server but not sure)
-
Ok, posting the code here made me find the mistake. So thanks guys.
There is a bug in a code
if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){
Should be
if(m_socket->bytesAvailable() >= m_msgSize && m_msgSize > 0){
Note the ">=" instead if "<="
It worked for smaller commands since tcp layer did not split it.Thanks again :D
-
Hi,
Can you show how you are using QTcpSocket ?
-
Sure, this is how I create it
m_socket = new QTcpSocket(this); qDebug() << "Buffer size: " << m_socket->readBufferSize(); // set the ID if(!m_socket->setSocketDescriptor(m_socketDescriptor)) { // something's wrong, we just emit a signal emit error(m_socket->errorString()); return; } //Connect signals connect(m_socket, SIGNAL(disconnected()), this, SLOT(disconnected())); connect(m_socket, SIGNAL(readyRead()), this, SLOT(readyRead()));
Later in my readyRead slot I read first 4 bytes(it is message size and wait for rest of data to arrive)
//Reset timer last command timer lastCommandTimer->start(); //Check if waiting for data if(m_msgSize == 0){ //No data length qDebug() << "Bytes available:" << m_socket->bytesAvailable(); //In this line it is never bigger than 4096!!! //Message too short to get data length if(m_socket->bytesAvailable() < 4) return; //Read message length QByteArray msgSizeArray = m_socket->read(4); QDataStream ds(&msgSizeArray, QIODevice::ReadOnly); ds >> m_msgSize; qDebug() << "Ready read size:" << m_msgSize; } //Got message length, waiting for whole message if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){ //Got whole message read it QByteArray msg = m_socket->read(m_msgSize); //Process command //... }
EDIT: BTW I use Qt 5.9,
Since bufferSize in maximum is 4096 when I send to my server bigger message my communication gets corrupted, I miss a lot of data. -
It is quite strange tbh but, in any case, you can build the buffer yourself.
- declare a private
QByteArray
- Every time you get the
readyRead()
just dump everything in the QByteArray (m_byteArray.append(m_socket.readAll());
) - do whatever you are doing now but use the
m_byteArray
as source of data instead of them_socket
- declare a private
-
Ok, posting the code here made me find the mistake. So thanks guys.
There is a bug in a code
if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){
Should be
if(m_socket->bytesAvailable() >= m_msgSize && m_msgSize > 0){
Note the ">=" instead if "<="
It worked for smaller commands since tcp layer did not split it.Thanks again :D
-
Sure, this is how I create it
m_socket = new QTcpSocket(this); qDebug() << "Buffer size: " << m_socket->readBufferSize(); // set the ID if(!m_socket->setSocketDescriptor(m_socketDescriptor)) { // something's wrong, we just emit a signal emit error(m_socket->errorString()); return; } //Connect signals connect(m_socket, SIGNAL(disconnected()), this, SLOT(disconnected())); connect(m_socket, SIGNAL(readyRead()), this, SLOT(readyRead()));
Later in my readyRead slot I read first 4 bytes(it is message size and wait for rest of data to arrive)
//Reset timer last command timer lastCommandTimer->start(); //Check if waiting for data if(m_msgSize == 0){ //No data length qDebug() << "Bytes available:" << m_socket->bytesAvailable(); //In this line it is never bigger than 4096!!! //Message too short to get data length if(m_socket->bytesAvailable() < 4) return; //Read message length QByteArray msgSizeArray = m_socket->read(4); QDataStream ds(&msgSizeArray, QIODevice::ReadOnly); ds >> m_msgSize; qDebug() << "Ready read size:" << m_msgSize; } //Got message length, waiting for whole message if(m_socket->bytesAvailable() <= m_msgSize && m_msgSize > 0){ //Got whole message read it QByteArray msg = m_socket->read(m_msgSize); //Process command //... }
EDIT: BTW I use Qt 5.9,
Since bufferSize in maximum is 4096 when I send to my server bigger message my communication gets corrupted, I miss a lot of data.@kolegs
4096 makes kind of sense, prob 4 Bytes reserved (for internal stuff) and 12 for data storage.IIRC, than the MTU (Maximum Transmission Unit) for Ethernet, is 1500 bytes. so you should be unable to run into overflow imediately. And can save the data in a Private/Protected QByteArray, like @VRonin suggested.
additionally to that,
you're reading parts of the buffer each cycleQByteArray msgSizeArray = m_socket->read(4);
AFAIK that removes those bytes from the buffer and your data should get corrupted
-
@J-Hilk Notice I only read this 4 bytes if I does not have the size
if(m_msgSize == 0){ ... //Read message length QByteArray msgSizeArray = m_socket->read(4); QDataStream ds(&msgSizeArray, QIODevice::ReadOnly); ds >> m_msgSize; ... }
If I got a message length I wait for the data to arrive. m_msgSize is a quint32 private class member, so it is not 0 each time I get into readyRead slot.