[Solved] Problem using QTcpSocket::readAll() on large amounts of data

Melph

Hi,
I have a simple client-server communication using TCP. I send an XML-string from the client to the server. Within this XML-string there is binary data (of an image) encoded as hex-string.

Here are the snippets:

Client:
@QPicture pic;
pic.load(ui->txt_pic->text());
QByteArray hex_data = (QByteArray::fromRawData(pic.data(), pic.size())).toHex();
socket->write(hex_data);@

Server:
@QByteArray helper = socket->readAll(); //the program crashes here
buffer.append(helper);@

With small pictures (some KB) everything works fine. With bigger pictures (a few hundred KB), the application crashes sometimes at "QByteArray helper = socket->readAll();"
If run in the debugger, there is sometimes a "double free or corruption"-error and a backtrace (but not everytime it crashes) which looks always different. If run from console, there's a "segmentation fault (core dumped)"

I have also tried using QImage instead of QPicture, but there's no difference in behavior.

Any ideas/suggestions how to make it work?

edit: Tested on a Fedora Linux, but the application crashes on the target board as well (Freescale i.MX6)

3dh3dh.de

Hi,

there are 2 problem sources I see:

the way how the QImage data is streamed is wrong
usage of sockets inside threads

Just use QImage and a QDataStream to stream the image data like a file to the network socket, so the receiver can stream back the full image with all it's attributes (size, depth, color table...).

What you did is just writing a bunch of data converted to a numeric format (hex values) and thus destroying all informations about the original data format. Further no image attributes are send, so the receiver has no chance to build a new image based on the received pixel data.

Example:

@
QTcpSocket* socket = new QTcpSocket....
QImage img("test.png");
QByteArray buffer;

QDataStream out(&buffer);
out << int(0); // placeholder for info about bytes for the binary data
out << img; // binary representation of image (as PNG file)

out.device()->seek(0); // go back to start
out << buffer.size(); // info about bytes for the size value (int) and binary image data (image)

if (socket->write(buffer) < buffer.size()) {
qWarning("transmit error!");
}
@

Usage of sockets inside apps with multiple threads must be done with caution. A socket ONLY can be used inside the thread's run() method or context, where the socket was created originally.

If you create the socket by the main gui thread / inside the main() function and use a separate thread to process the socket's events, you'll get serious trouble.

Further "readAll()" isn't a safe way of ensuring that all data are received successfully - a safer way is to transmit the amount of bytes of the binary data and then waiting to receive them.

I extended the above example for sending the image by sending the buffer size, too. So we know how big the network message is and can wait for receiving it:

Example:

@
QDataStream in(socket);
int msgSize = -1;

if (socket->bytesAvailable() && msgSize = -1) {
in >> msgSize;
}

while (socket->bytesAvailable() < msgSize - sizeof(int)) {
if (!socket->waitForReadyRead(100)) {
socket->disconnectFromHost();
break;
}
}

QImage img;
in >> img;
@

This works with real big images (> 40 MB) under Windows + Linux since 7 years - tested with Qt 4.3 up to 4.8 (and MUST work with 5.0, too).

Melph

Hi Christian,
Thank you very much for the response, I think you solved my problem. Especially calling socket functions from another thread seems to be a problem (in some cases at least).

blackie

Christian, thanks for the code snippets.

Two observation wrt to the above code:

use qint32 for the length instead of int, to ensure that it works accross different platforms
When writting to the byte array you need a QBuffer instead of a QByteArray
QBuffer buffer;
buffer.open(QIODevice::WriteOnly);
...
socket->write(buffer.data())