[Solved] Problem using QTcpSocket::readAll() on large amounts of data



  • Hi,
    I have a simple client-server communication using TCP. I send an XML-string from the client to the server. Within this XML-string there is binary data (of an image) encoded as hex-string.

    Here are the snippets:

    Client:
    @QPicture pic;
    pic.load(ui->txt_pic->text());
    QByteArray hex_data = (QByteArray::fromRawData(pic.data(), pic.size())).toHex();
    socket->write(hex_data);@

    Server:
    @QByteArray helper = socket->readAll(); //the program crashes here
    buffer.append(helper);@

    With small pictures (some KB) everything works fine. With bigger pictures (a few hundred KB), the application crashes sometimes at "QByteArray helper = socket->readAll();"
    If run in the debugger, there is sometimes a "double free or corruption"-error and a backtrace (but not everytime it crashes) which looks always different. If run from console, there's a "segmentation fault (core dumped)"

    I have also tried using QImage instead of QPicture, but there's no difference in behavior.

    Any ideas/suggestions how to make it work?

    edit: Tested on a Fedora Linux, but the application crashes on the target board as well (Freescale i.MX6)



  • Hi,

    there are 2 problem sources I see:

    1. the way how the QImage data is streamed is wrong
    2. usage of sockets inside threads

    Just use QImage and a QDataStream to stream the image data like a file to the network socket, so the receiver can stream back the full image with all it's attributes (size, depth, color table...).

    What you did is just writing a bunch of data converted to a numeric format (hex values) and thus destroying all informations about the original data format. Further no image attributes are send, so the receiver has no chance to build a new image based on the received pixel data.

    Example:

    @
    QTcpSocket* socket = new QTcpSocket....
    QImage img("test.png");
    QByteArray buffer;

    QDataStream out(&buffer);
    out << int(0); // placeholder for info about bytes for the binary data
    out << img; // binary representation of image (as PNG file)

    out.device()->seek(0); // go back to start
    out << buffer.size(); // info about bytes for the size value (int) and binary image data (image)

    if (socket->write(buffer) < buffer.size()) {
    qWarning("transmit error!");
    }
    @

    Usage of sockets inside apps with multiple threads must be done with caution. A socket ONLY can be used inside the thread's run() method or context, where the socket was created originally.

    If you create the socket by the main gui thread / inside the main() function and use a separate thread to process the socket's events, you'll get serious trouble.

    Further "readAll()" isn't a safe way of ensuring that all data are received successfully - a safer way is to transmit the amount of bytes of the binary data and then waiting to receive them.

    I extended the above example for sending the image by sending the buffer size, too. So we know how big the network message is and can wait for receiving it:

    Example:

    @
    QDataStream in(socket);
    int msgSize = -1;

    if (socket->bytesAvailable() && msgSize = -1) {
    in >> msgSize;
    }

    while (socket->bytesAvailable() < msgSize - sizeof(int)) {
    if (!socket->waitForReadyRead(100)) {
    socket->disconnectFromHost();
    break;
    }
    }

    QImage img;
    in >> img;
    @

    This works with real big images (> 40 MB) under Windows + Linux since 7 years - tested with Qt 4.3 up to 4.8 (and MUST work with 5.0, too).



  • Hi Christian,
    Thank you very much for the response, I think you solved my problem. Especially calling socket functions from another thread seems to be a problem (in some cases at least).



  • Christian, thanks for the code snippets.

    Two observation wrt to the above code:

    • use qint32 for the length instead of int, to ensure that it works accross different platforms
    • When writting to the byte array you need a QBuffer instead of a QByteArray
      QBuffer buffer;
      buffer.open(QIODevice::WriteOnly);
      ...
      socket->write(buffer.data())

Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.