Can we receive large (1064) udp datagram

[Sowjanya]

Hi
I want to use Qt UDP (not TCP) socket to recieve.I tried to recieve small packets like 15 bytes in qt .Small packets are recieving properly but while i tried to recieve 1064 packets data QT is not recieving properly ,However i can recieve full data in wireshark .
Here is my normal code

MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
socket = new QUdpSocket(this);
socket->bind(QHostAddress("192.168.10.1"),64826));
connect(socket,SLOT(readyread(),this,SLOT(readdata());
}
void MainWindow::readdata()
{
QHostAddress sender;
quint16 port;
QByteArray buffer;
QByteArray Data;
buffer.resize(socket1->pendingDatagramSize());
socket1->readDatagram(buffer.data(),buffer.size(),&sender,&senderport);
qDebug()<<"sender"<<sender;
qDebug()<<"data"<<buffer;
}

Can anyone help me ,Thanks.

[jsulm]

@Sowjanya In what way it does not work for big packets?

[Sowjanya]

@jsulm through fpga i tried to recieve packets

[jsulm]

@Sowjanya said in Can we recieve large(1064) udp datagram:

i tried to recieve packets

And what happened?
What exactly did not work?
Can you be more precise?

[Sowjanya]

@jsulm i sent data to fpga through that i recieved some thousands of packets ,which are in 1064(lenth) .In wireshark i can able to see ,but in qt only some packets are recieving
ex:12mb data
i am getting 1mb data

[J.Hilk]

@Sowjanya well it's udp so there is no guarantee that your datagram comes in one go (especially filtered through the OS and Qt interface)

You have to impose some kind of protocol on it, so that you know this chunk of bytes is one coherent datagram and the next one belongs to an other one.

You need error checking as well, your datagram may loos bytes

[jsulm]

@Sowjanya Please take a look at the example in https://doc.qt.io/qt-5/qudpsocket.html
In your slot you should read as long as there are still pending datagrams:

while (udpSocket->hasPendingDatagrams()) {
    QNetworkDatagram datagram = udpSocket->receiveDatagram();
    processTheDatagram(datagram);
}

In your current code you only read one datagram.

[Sowjanya]

This post is deleted!

[Sowjanya]

@jsulm
Hi i again tried like this,but i cant able to recieve properly
{
ui->setupUi(this);
socket = new QUdpSocket(this);
socket->bind(QHostAddress("192.168.10.1"),64826));
connect(socket,SLOT(readyread(),this,SLOT(readdata());
}
void MainWindow::readdata()
{
QHostAddress sender;
quint16 port;
QByteArray buffer;
QByteArray Data;
while (socket->hasPendingDatagrams())
{
QNetworkDatagram datagram = socket->receiveDatagram();
qDebug()<<"data"<<datagram.data();
// processTheDatagram(datagram);
}
}

[JonB]

@Sowjanya said in Can we receive large (1064) udp datagram:

but i cant able to recieve properly

Code looks about right to me. Please don't say just "but i cant able to recieve properly", you have debug statement there, why not tell people what it did/did not output? Or do you prefer people trying to help should just have to guess for themselves?

[Pablo J. Rogina]

@Sowjanya said in Can we receive large (1064) udp datagram:

i cant able to recieve properly

In addition to @JonB suggested, you may want to analyze network traffic to/from your Qt app by means of Wireshark to confirm what data is actually flowing back and forth in the connection...

[Joe von Habsburg]

@Sowjanya said in Can we receive large (1064) udp datagram:

I want to use Qt UDP (not TCP) socket to recieve.I tried to recieve small packets like 15 bytes in qt .Small packets are recieving properly but while i tried to recieve 1064 packets data QT is not recieving properly ,However i can recieve full data in wireshark .

Hello, I have same problem. When I check wireshark all packets are arrived, but when I try take on QUdpSocket, I could not take.

Also I tried on python yes I could not take all packages. But there is a some different thing, python faster than qt.

Now you will ask how you measure that,

Qt:

#include <QDateTime>
QDateTime time11 = QDateTime::currentDateTime();
int counterx = 0;
void Udp::parseData(QByteArray data)
{
    counterx++;
    QDataStream stream(data);
    stream.setByteOrder(QDataStream::BigEndian);
    stream.setFloatingPointPrecision(QDataStream::SinglePrecision);

    quint32 frameHeader;
    quint32 frameLenght;
    quint32 instruction;
    quint32 frameId;

    stream >> frameHeader >> frameLenght >> instruction >> frameId;

    quint64 abc = 0;
    quint64 def = 0;
    quint32 startIndex;
    quint32 count;
    quint32 size;

    stream >> abc >> def >> startIndex >> count >> size;

    vector<float> list;

    for (int j = 0; j < count; ++j)
    {
        if(stream.atEnd()){
            return;
        }

        float val;
        stream >>val;
        list.push_back(val);
    }

    if(size != _listSize){
        _listSize = size;
        _list.resize(_listSize);
    }

    for(int j = startIndex, i = 0; i < count; i++, j++){
        _list[j] = list[i];
    }

    _received += count;

    QDateTime time5 = QDateTime::currentDateTime();
    qDebug() << "Udp::parseData" << abc << def << startIndex << count << size << _received << counterx << time5 - time11;

   //LAST MESSAGE - Udp::parseData 2500000000 3000000 16975 175 65536 17150 98 5328ms
}

Python:

import time
start_time = time.time()
received_count = 0
def parseData(data: bytes):
    global received_size, received_count, my_list, start_time
    
    header = data[0:4]
    length = int.from_bytes(data[4:8], 'big') 
    instruction = int.from_bytes(data[8:12], 'big') 
    frameid = int.from_bytes(data[12:16], 'big') 
    abc = struct.unpack_from('>Q', data, 16)[0]  
    def = struct.unpack_from('>Q', data, 24)[0]  
    start_index = int.from_bytes(data[32:36], 'big') 
    count = int.from_bytes(data[36:40], 'big') 
    total_count = int.from_bytes(data[40:44], 'big') 
    
    received_size += count
    received_count += 1
    
    if len(my_list) != total_count:
        my_list = [0.0] * total_count
    
    list = []
    index = 44
    
    for _ in range(count):
        val = struct.unpack_from('>f', data, index)[0] 
        
        list.append(val)
        
        index += 4
    
    for i in range(count):
        my_list[start_index + i] = list[i]
    
    temp_time = time.time()
    time_diff = (temp_time - start_time) * 1000
    print(f"abc : {abc} - def : {def} - START_INDEX : {start_index} - COUNT : {count} - TOTAL_COUNT : {total_count} - RECEIVED_SIZE : {received_size} - RECEIVED_COUNT : {received_count} - TIME : {time_diff}")

#LAST MESSAGE - abc : 2500007629 - def : 3000000 - START_INDEX : 46025 - COUNT : 175 - TOTAL_COUNT : 65536 - RECEIVED_SIZE : 23625 - RECEIVED_COUNT : 135 - TIME : 15.902280807495117

what is yours opinion ?

[jsulm]

@Joe-von-Habsburg start_time initialization should be at the beginning of parseData, else you're also measuring the time between time11 initialization and parseData being called. Or what do you want to measure?

[J.Hilk]

@Joe-von-Habsburg

QDateTime time5 = QDateTime::currentDateTime();
    qDebug() << "Udp::parseData" << abc << def << startIndex << count << size << _received << counterx << time5 - time11; 

the creation of QDateTime::CurrentDateTme() alone is in order of 10's of ms magnitude. Also are you running release build and/or any optimisations, that you're comparing to probably auto jitted Python?

[Joe von Habsburg]

I solved my problem. Thank you for reply,

[Pablo J. Rogina]

@Joe-von-Habsburg would you mind sharing the solution you've found? Thanks

[Joe von Habsburg]

@Pablo-J.-Rogina I changed my code.
I parsed each datagram when they come, now I waiting for end, after parsing step by step. That solve my problem.

//Before

if(udpSocket.hasPendingDatagrams()){
        qint64 sz = _udpSocket.pendingDatagramSize();
        if (sz <= 0) { _udpSocket.readDatagram(nullptr, 0); return; }
        QByteArray chunk;
        chunk.resize(sz);
        _udpSocket.readDatagram(chunk.data(), chunk.size());
        _rxBuf.append(chunk);
        processIncomingDatagram();
}

//After
while(_udpSocket.hasPendingDatagrams()){
        qint64 sz = _udpSocket.pendingDatagramSize();
        if (sz <= 0) { _udpSocket.readDatagram(nullptr, 0); continue; }
        QByteArray chunk;
        chunk.resize(sz);
        _udpSocket.readDatagram(chunk.data(), chunk.size());
        _rxBuf.append(chunk);
}

processIncomingDatagram();

[Joe von Habsburg]

@J.Hilk said in Can we receive large (1064) udp datagram:

the creation of QDateTime::CurrentDateTme() alone is in order of 10's of ms magnitude

I did not know that. Thank you so much. This is be problem for measuring. Thanks again

[Joe von Habsburg]

@jsulm said in Can we receive large (1064) udp datagram:

start_time initialization should be at the beginning of parseData, else you're also measuring the time between time11 initialization and parseData being called. Or what do you want to measure?

My aim was measure to taking 1 frame which is sum of datagrams taking time, but I solved yesterday. Thank you so much.
I learned new thing about QDateTime from @J.Hilk . It is help to me for right measuring. Thanks for your reply

[Joe von Habsburg]

@Joe-von-Habsburg said in Can we receive large (1064) udp datagram:

Hello, I have same problem. When I check wireshark all packets are arrived, but when I try take on QUdpSocket, I could not take.

I understand the reason now. If I parse every incoming datagram, I can't receive all of them. But if I wait for it to finish, then I can receive them.
Probly @Sowjanya had some problem. I hope so, s/he solved that. :)

Great a good day :) @J.Hilk @jsulm @Pablo-J.-Rogina