Testing QUdpSocket. Facing limit on the amount of received packets. Inconsistent results when using QThreads, issues with debug mode

Sergey Kolesnik

I have written a program for unit testing ща data transfer with QUdpSocket. It sends an arbitrary (provided by user) amount of Data elements in a sorted vector (Data has an index and a randomly generated number).
The program creates 2 QUdpSockets: one for sending, other for receiving data. It creates and sends a vector of Data, receives it and analyzes for consistency. Function for an analysis can be found here. There is also a repo with the complete project on GitHub.

I have faced very odd results.
When I use the main event loop to send and receive data, there is always a limit on how many bytes can be received: for Windows 10 it is 65536 bytes, for Windows 7 it is 8192. I tried to set buffer size of a QUdpSocket with different values (0 by default, docs say it means that buffer has no limit), but the result is always the same.
Wireshark though shows that all the packets are sent, no matter how much data I transfer.

When I use different QThreads for sending and receiving sockets I also get very inconsistent behavior.
Such that when I run on Windows 10 in release mode (compiled with either MSVC or MinGW) ALL the packets are received.
For debug mode:

just running the test compiled with MinGW goes fine and all the packets are received
running the test in Debug mode (with CLion) is dramatically different. The execution takes way more time and a lot of packets are lost or received out of order.
running the test compiled with MSVC yields invalid results - packets are lost or out of order no matter what.

struct Data
{
    uint64_t index,
            rndNumber;
    // uint64_t zero{0};
};

// helper function to ensure Data is not sent segmented
template<typename Iter, typename = typename
std::enable_if<
        std::is_same<typename std::iterator_traits<Iter>::iterator_category,
                std::random_access_iterator_tag>::value &&
        sizeof(typename std::iterator_traits<Iter>::value_type) <= 512>::type>
void WriteData(QUdpSocket &socket, Iter begin, Iter end)
{
    constexpr size_t sizeofData = sizeof(typename std::iterator_traits<Iter>::value_type);
    static_assert(sizeofData <= 512, "Size of data type must be less or equal 512 bytes");

    assert(std::distance(begin, end) >= 0 && "Invalid range");
    while (begin != end)
    {
        size_t elemsToSend = 512 / sizeofData;
        size_t elemsRemains = std::distance(begin, end);

        elemsToSend = std::min(elemsToSend, elemsRemains);
        size_t bytesToSend = elemsToSend * sizeofData;
        const char *pDatagram = reinterpret_cast<const char *>(&*begin);

        uint64_t bytesWritten = socket.write(pDatagram, bytesToSend);
        assert(bytesWritten == bytesToSend ||
               bytesWritten != -1 && "Failed to send datagram");

        begin = std::next(begin, elemsToSend);
    }
}


int main(int argc, char **argv)
{
    size_t elems = 32768;
    if (argc == 2)
    {
        try
        {
            elems = std::stoll(argv[1]);
        }
        catch (const std::invalid_argument &e)
        {
            std::cerr << "Invalid argument for elements number. Testing with " << elems <<
                      "elements." << std::endl;
        }
    }

    // form vector to send
    std::vector<Data> dataSent{elems},
            dataReceived{};
    {
        std::random_device rd;
        std::mt19937 rng(rd());
        std::uniform_int_distribution<std::mt19937::result_type> dist(0);
        for (size_t i = 0; i != elems; ++i)
        {
            dataSent[i].index = i;
            dataSent[i].rndNumber = dist(rng);
        }
    }

    QHostAddress address = QHostAddress::LocalHost;
    quint16 unicastPortSend = 8001;

    QCoreApplication app{argc, argv};

    QTimer receiveTimeout;
    QObject::connect(&receiveTimeout, &QTimer::timeout,
                     &app, &QCoreApplication::quit, Qt::QueuedConnection);
    receiveTimeout.start(2000);

    QUdpSocket receiver;
    receiver.bind(address, unicastPortSend);

#ifdef QTHREADS
    // receive vector in a separate thread
    QThread receiverThread;
    receiver.moveToThread(&receiverThread);
    receiveTimeout.moveToThread(&receiverThread);

    QObject::connect(&receiveTimeout, &QTimer::timeout,
                     &receiverThread, &QThread::quit);

    // move receiver and timer back to the main thread when finished
    QObject::connect(&receiverThread, &QThread::finished, &receiver,
                     std::bind(&QObject::moveToThread, &receiver,
                               app.thread()));

    QObject::connect(&receiverThread, &QThread::finished, &receiveTimeout,
                     std::bind(&QObject::moveToThread, &receiveTimeout,
                               app.thread()));

    receiverThread.start();
#endif

    // connect reading function
    size_t countSteps{};
    QObject::connect(&receiver, &QUdpSocket::readyRead, [&]
    {
        receiveTimeout.stop();
        ++countSteps;
        while (receiver.hasPendingDatagrams())
        {
            QByteArray array;

            int64_t pendingDatagramSize = receiver.pendingDatagramSize();
            assert(pendingDatagramSize != -1 && "Unexpected datagram size!");

            array.resize(pendingDatagramSize);

            uint16_t readSize = receiver.readDatagram(array.data(), pendingDatagramSize);
            assert(readSize == pendingDatagramSize && "Failed to read pending datagram");

            const Data *pData = reinterpret_cast<const Data *>(array.data());
            size_t elemsReceived = readSize / sizeof(Data);

            std::copy(pData, std::next(pData, elemsReceived), std::back_inserter(dataReceived));
        }
        receiveTimeout.start(200);
    });

    // sender socket
    QUdpSocket sender;
    sender.connectToHost(address, unicastPortSend);

#ifdef QTHREADS
    QThread senderThread;
    sender.moveToThread(&senderThread);

    // move sender back to main thread on finish
    QObject::connect(&senderThread, &QThread::finished,
                     &sender,
                     std::bind(&QObject::moveToThread, &sender, app.thread()));

    senderThread.start();
#endif

    QTimer::singleShot(2, &sender, [&]
    {
        WriteData(sender, dataSent.cbegin(), dataSent.cend());
#ifdef QTHREADS
        senderThread.quit();
#endif
    });

    app.exec();

    auto result = make_io_arrays_compare(std::move(dataSent), std::move(dataReceived),
                                         [](const Data &d1, const Data &d2)
                                         {
                                             return std::tie(d1.index, d1.rndNumber) <
                                                    std::tie(d2.index, d2.rndNumber);
                                         });

    std::cout << result;

    if (result)
        return 0;

    return -1;
}

To sum up, my questions are:

Why do I have a limitation on receiving data when sockets run in the main (same) thread (event loop)?
What is the reason for me not getting consistent results when sockets run in separate threads (event loops)?
Why running the test in debug mode takes incredibly long time and packets are lost or received out of order?
If QThreads (event loops) are unreliable, how can I overcome this problem without event loops?

JKSH

If you need to ensure that none of your data is lost and all of your data is received in the correct order, then UDP is not an appropriate choice. UDP is a lossy protocol by design.

Solution: Use TCP instead. It is designed to ensure that data always arrives complete, and in the correct order.

Anyway, the answer to all of your questions are related to timing. Debug mode has much slower timing than release mode. More bytes take more time to transmit. Slower timings increase the likelihood that UDP packets are lost or delayed.

Sergey Kolesnik

@JKSH I figured out timing is the case. I am sending just way more data in a limited time period. Sending less data solves the problem.
But using only main thread somehow yields rather low limit of data received regardless of configuration.

JKSH

@Sergey-Kolesnik said in Testing QUdpSocket. Facing limit on the amount of received packets. Inconsistent results when using QThreads, issues with debug mode:

But using only main thread somehow yields rather low limit of data received regardless of configuration.

Use TCP. You won't encounter these kinds of limits.

kshegunov

Do you realize you're quite possibly waiting more on the heap management than on the actual socket operations?

This is allocation inside a loop:

array.resize(pendingDatagramSize);

This is a superfluous data copy:

std::copy(pData, std::next(pData, elemsReceived), std::back_inserter(dataReceived));

And finally what's the point of doing benchmarks in debug mode, this makes no sense at all; debug mode is for debugging not for measuring performance.

Pablo J. Rogina

@kshegunov said in Testing QUdpSocket. Facing limit on the amount of received packets. Inconsistent results when using QThreads, issues with debug mode:

debug mode is for debugging not for measuring performance.

Superb!