[SOLVED]QTcpSocket problem reading large data.
-
Hi, I'm having strange difficulties to read large data.
Strangeness is that, when I send small data, it reads successfully, but when I send like 44kb of data, server crashes under release mode, but in debug mode it shows that I have to read 0 bytes data and when following to code, nothing was read, but fun part is that, it somehow already read data and showed in result.. I don't really understand why!I'm using custom protocol, sending length of data (qint64) followed by the data.
Here's the code what I'm using right now.
[code]#define chunk_size 1024
void Thread::process()
{
qDebug() << "Started receiving data.";QByteArray buffer; char temp[chunk_size]; qint64 bytesReceived, datalen; if((quint64)socket->bytesAvailable() < sizeof(quint64)) return; socket->read((char *)&datalen, sizeof(qint64)); // when sending small amount of data, datalen is OK, but when I tried ~44kb of data, it is always 0. why? while(datalen) { memset(temp, 0, sizeof(temp)); if(datalen >= chunk_size) { bytesReceived = socket->read(temp, chunk_size); } else if(datalen < chunk_size) { bytesReceived = socket->read(temp, datalen); } if(bytesReceived != -1) { buffer += temp; datalen -= bytesReceived; } else { qDebug() << "Error receiving data."; buffer.clear(); return; } } if(socket->bytesAvailable()) socket->readAll(); // ignore other packets. qDebug() << "Data:" << buffer;}[/code]
Server is written in Qt and client in C++. Client side doesn't matter, because I'm using only send(sockfd, buffer, 44 * 1024, 0);
Also, let me know if I'm doing something wrong.
Regards.EDIT:
Following error is shown: double free or corruption (!prev): 0x00007f49f40823c0 *** -
Yes, all data is sent successfully.
EDIT: am I doing it wrong?
[code]unsigned long long size = 44 * 1024;
char *buffer = (char *)malloc(size + 8);
if(buffer)
{
memmove(buffer, (void *)&size, sizeof(unsigned long long));
memmove(buffer + 8, 44kb_data, size);
send(sockfd, buffer, size + 8, 0);
}[/code]EDIT: This problem is related to this topic -> http://qt-project.org/forums/viewthread/25609 but whenever I'm doing it on QByteArray, it shows that result is always empty.
-
You MUST check what send() returns.
It can return less bytes you provided, in this case you must sent the remaining data.For example:
@
size_t dataToSend = size + 8;
size_t sent = 0;
char *pBuffer = buffer;
while (dataToSend > 0) {
sent = send (sockfd, pBuffer, dataToSend);
if (sent > 0) {
pBuffer += sent;
dataToSend -= sent;
}
else {
// ERROR
}
}
@ -
Sorry for long reply, I was traveling a bit..
About the problem, seems I can't debug the code, because as far as I guess, readyRead called many times (because TCP is streaming protocol). do I am missing something? :-([code]//Thread.h
#ifndef THREAD_H
#define THREAD_H#include <QObject>
#include "tcpsocket.h"class Thread : public QObject
{
Q_OBJECT
public:
explicit Thread(TcpSocket *client, QObject *parent = 0);signals:
void finished();public slots:
void process();private:
TcpSocket *socket; //inherits QTcpSocket.
qint64 blockSize;
QByteArray buffer;};
#endif // THREAD_H[/code]
[code]//Thread.cpp
#include "thread.h"#include <QDebug>
#define chunk_size 1024
Thread::Thread(TcpSocket *client, QObject *parent) :
QObject(parent), socket(client)
{
blockSize = 0;
}void Thread::process() {
qDebug() << "readyRead()";char temp[chunk_size]; qint64 bytesReceived; if(socket->bytesAvailable() < (qint64)sizeof(qint64)) return; if(blockSize == 0) { socket->read((char *)&blockSize, sizeof(qint64)); } while(blockSize) { memset(temp, 0, sizeof(temp)); bytesReceived = socket->read(temp, chunk_size); if(bytesReceived > 0) { buffer += temp; blockSize -= bytesReceived; } else { blockSize = 0; qDebug() << "Error receiving data."; } } qDebug() << "Data:" << buffer; buffer.clear(); emit finished();}[/code]
Sender part is exactly like that you posted above.
When sending large data, it stops on readyRead and after step over, it still stays on readyRead twice or more and then just jumps to return and data is received successfully (can't see the receiving part while debugging), but on release mode, it just don't receive all data.. WHY? can someone do some tests please? :-(
-
bq. Server is written in Qt and client in C++. Client side doesn’t matter, because I’m using only send(sockfd, buffer, 44 * 1024, 0);
So, the server is C++ and the client is C++. Qt is library not a language. I will assume the first 8 bytes in buffer are a 64-bit int in the correct byte order.
The general problem happens because you are subverting the normal flow of events in Qt.
The readyRead() signal will be emitted when there is some data, note that's not all data, to be read from the socket. If that is less than the 8 byte size field then you return, fine.
If you have more than 8 bytes you read the 8 byte size leaving the remainder. You then enter a loop trying to read chunks of 1024 bytes until the blocksize is exhausted. The thing is, blocksize will not be exhausted unless the entire packet was received and available when readyRead() was emitted. In general, blocks written in one send() will not be received in one packet because the network will fragment the data. For small payloads you may get away with it but you cannot rely on it.
In a single threaded application your loop would never allow the Qt event loop to resume, and therefore no further data will be received. Your code implies you have threading involved, possibly in an attempt to receive data after you enter this loop.
You do not require any form of threads to implement a network interfaces in Qt. Using them only adds potential places this can be coming unstuck.Here is a complete example that will receive single blocks in the form you describe. (No error handling etc.)
@
#include <QtCore>
#include <QTcpServer>
#include <QTcpSocket>class Receiver: public QObject
{
Q_OBJECT
public:
Receiver(QTcpSocket *socket, QObject *p = 0):
QObject(p),
sock(socket),
bytesExpected(0)
{
connect(socket, SIGNAL(readyRead()), SLOT(readyRead()));
}public slots:
void readyRead() { if (bytesExpected == 0 && sock->bytesAvailable() >= sizeof(bytesExpected)) { sock->read((char *)&bytesExpected, sizeof(bytesExpected)); qDebug() << "Expecting:" << &bytesExpected; } if (bytesExpected > 0 && sock->bytesAvailable() > 0) { QByteArray chunk = sock->read(qMin(bytesExpected, sock->bytesAvailable())); buffer += chunk; bytesExpected -= chunk.size(); qDebug() << "Received chunk of:" << chunk.size(); if (bytesExpected == 0) { qDebug() << "Received block of size:" << buffer.size(); qDebug() << "Bytes left in socket:" << sock->bytesAvailable(); sock->deleteLater(); deleteLater(); } } }private:
QTcpSocket *sock;
qint64 bytesExpected;
QByteArray buffer;
};class Server: public QObject
{
Q_OBJECT
public:
Server(QObject *p = 0): QObject(p)
{
server = new QTcpServer(this);
connect(server, SIGNAL(newConnection()), SLOT(handleConnection()));
server->listen(QHostAddress::Any, 12345);
}
public slots:
void handleConnection()
{
while(server->hasPendingConnections())
(void) new Receiver(server->nextPendingConnection(), this);
}
private:
QTcpServer *server;
};int main(int argc, char **argv)
{
QCoreApplication app (argc, argv);Server s; return app.exec();}
#include "main.moc"
@ -
Big thanks to you.. I was getting crazy day by day finding what caused the problem.. I finally got it that it was problem with threading, because after re-creating thread, blockSize would be always 0 (which constructor initializes).
Many many many many thanks to you!