QSerialPort parsing binary packets at a fast speed

lukutis222

Hello. I am writing QT Application to parse incoming serial data from the power analyzer device. The power analyzer sends serial packets every 1ms at baudrate 921600. The packet is 13 bytes length:
Byte 0: Header #1 (0xAA)
Byte 1: Header #2 (0x55)
Byte 2: Packet length
Byte 3: Channel byte
Byte 4,5,6,7: 4 bytes of voltage
Byte 8,9,10,11: 4 bytes of current
Byte 12: CRC

This is how the packets look when I check them via Salae logic analyzer:

The simillar pakcets are being sent approximately every 1ms:

In my QT application, I have created a serial worker class. See my class below:

#include "serialworker.h"
#include <qDebug>
#include <QDateTime>
#include <QThread>
#include <QMutexLocker>


SerialWorker::SerialWorker(SettingsWindow *settings_ptr, QObject *parent)
    : QObject{parent}
{
    settings_local = settings_ptr;

}

SerialWorker::~SerialWorker() {
    if (serial->isOpen()) {
        serial->close();
    }
    delete serial;
}




void SerialWorker::initialize() {
    qDebug("initialize method \n");

    serial = new QSerialPort(this);
    connect(serial, &QSerialPort::readyRead, this, &SerialWorker::enqueueData);
    // Initialize timer to process data from the queue periodically
    processTimer = new QTimer(this);
    connect(processTimer, &QTimer::timeout, this, &SerialWorker::processQueue);
    processTimer->start(1); // Process every 1 ms

    available_devices.resize(10);
    timer_detect = new QTimer(this);
    connect(timer_detect, &QTimer::timeout, this, &SerialWorker::detect_timeout);

}




void SerialWorker::enqueueData() {
    QByteArray data = serial->readAll();
    QMutexLocker locker(&queueMutex); // Lock the queue
    dataQueue.enqueue(data);          // Add data to the queue
}

void SerialWorker::processQueue() {
    QMutexLocker locker(&queueMutex); // Lock the queue for thread-safe access

    while (!dataQueue.isEmpty()) {
        QByteArray data = dataQueue.dequeue(); // Get the next data block
        locker.unlock();                       // Unlock while processing to improve efficiency
        // Process the data block
        processData(data);
        locker.relock(); // Re-lock for the next iteration
    }
}

void SerialWorker::processData(const QByteArray &data) {
    static QByteArray buffer; // Persistent buffer for incomplete packets
    buffer.append(data);

    while (buffer.size() >= 13) { // Minimum size of a packet
        if (static_cast<uint8_t>(buffer[0]) == 0xAA && static_cast<uint8_t>(buffer[1]) == 0x55) {
            if (static_cast<uint8_t>(buffer[2]) == 0x09) { // Validate payload length
                if (buffer.size() < 13) return;           // Wait for more data

                char channel = buffer[3];
                if (channel == static_cast<char>(0x41)) { // 'A' for channel A
                    float voltage, current;
                    memcpy(&voltage, buffer.mid(4, 4).data(), 4);
                    memcpy(&current, buffer.mid(8, 4).data(), 4);

                    uint8_t calculatedChecksum = 0;
                    for (int i = 2; i < 12; ++i) {
                        calculatedChecksum ^= static_cast<uint8_t>(buffer[i]);
                    }

                    if (calculatedChecksum == static_cast<uint8_t>(buffer[12])) {
                        // Emit the processed data to the main thread
                        emit dataProcessed(channel, voltage, current);

                        buffer.remove(0, 13); // Remove processed packet
                        continue;
                    } else {
                        qDebug() << "Invalid checksum. Skipping packet.";
                        buffer.remove(0, 13); // Remove invalid packet
                    }
                } else {
                    qDebug() << "Unknown channel. Skipping packet.";
                    buffer.remove(0, 13); // Remove unknown packet
                }
            } else {
                qDebug() << "Invalid payload length. Skipping packet.";
                buffer.remove(0, 13); // Remove invalid packet
            }
        } else {
            qDebug() << "Invalid header. Searching for next valid header.";

            int headerIndex = buffer.indexOf(QByteArray::fromHex("AA55"), 1);
            if (headerIndex == -1) {
                buffer.clear(); // No valid header found, clear the buffer
            } else {
                buffer.remove(0, headerIndex); // Move to the next valid header
            }
        }
    }
}


// Stop receiving serial data
void SerialWorker::stop() {
    stopRequested = true;
    processTimer->stop();
    if (serial->isOpen()) {
        serial->close();
    }
}




void SerialWorker::write_data(QByteArray data){
    int64_t bytes_written = serial->write(data);
}



bool SerialWorker::Serial_connect(){

    if(serial->open(QIODevice::ReadWrite)){
        qDebug("Connection is succesfull \n");
        return 1;
    }
    else
    {
        //error
        qDebug() << serial->errorString();
    }
    return 0;

}
bool SerialWorker::Serial_disconnect(){
    if(serial->isOpen()){
        serial->close();
        return 1;
    }
    else{
        return 0;
    }
}

bool SerialWorker::Check_if_open(){
    if(serial->isOpen()){
        return 1;
    }
    else{
        return 0;
    }

}

QString SerialWorker::Get_Connect_Port(){
    if(Check_if_open() == 1){
        return serial->portName();
    }
    return "null";
}

The logic is as following, the QSerialPort::readyRead is connected to enqueueData slot which adds serial data to the queue.

Then the processQueue is called every 1ms which dequeues the data and try to match with the expected header (0xAA and 0x55). If the expected header is matched, then we attempt to read voltage and current data and emit dataProcessed.

In my mainwindow.h, I have:

    QElapsedTimer timer_usec;

In my mainwindow.cpp, I call:


MainWindow::MainWindow(SettingsWindow *settings_ptr,SerialWorker *serial_worker_ptr, Logging *logging_ptr, CustomTimeTicker *ticker_ptr,Cursor *cursor_ptr,FW_update_window *fw_update_ptr, QWidget *parent)
    : QMainWindow(parent), ui(new Ui::MainWindow)
{
    serial_worker = serial_worker_ptr;
    workerThread = new QThread(this);

    serial_worker->moveToThread(workerThread);

    connect(workerThread, &QThread::started, serial_worker, &SerialWorker::initialize);
    connect(workerThread, &QThread::finished, workerThread, &QThread::deleteLater);

    workerThread->start();

    settings_local = settings_ptr;
    logging_local = logging_ptr;
    ticker_local = ticker_ptr;
    cursor_local = cursor_ptr;
    fw_update_local = fw_update_ptr;
    uCurrent_plot_enabled = 0;
    autoscroll_enabled = 0;
    time_elapsed_count = 0;

    range_reduced = 0;
    previous_visible_x_axis_lower = 0;
    previous_visible_x_axis_upper = 0;


    ui->setupUi(this);
    ui->version_edit->setText(("2.2.6"));

    timer_usec.start();
    connect(serial_worker, &SerialWorker::dataProcessed, this, &MainWindow::handleProcessedData);

}

and the handleProcessedData :

void MainWindow::handleProcessedData(char channel, float voltage, float current) {
    if (channel == 'A') {
        qint64 elapsedNanoseconds = timer_usec.nsecsElapsed();
        double relative_timestamp = elapsedNanoseconds / 1e9;
        QString logString = QString::number(relative_timestamp, 'f', 6);  // Format to 9 decimal places
        logging_local->Write_to_file(logString);  // Save to file
    }
}

So to summarize the logic:
QSerialPort::readyRead -> enqueueData -> processQueue -> dataProcessed -> handleProcessedData

Sinec I am receiving serial data packets approximately every 1ms, I am hoping to be able to parse them fast enough, but that does not happen unfortunately. In my handleProcessedData I am simply capturing the timestamp :

        qint64 elapsedNanoseconds = timer_usec.nsecsElapsed();
        double relative_timestamp = elapsedNanoseconds / 1e9;

and writing this timestamp to the log. I expect the timestamps but be approximately 1ms apart, but the log contains the following data:

I cannot wrap my head around this. It seems that the timestamps are very inconsistent and most of them are not 1ms apart. I would really like to get some advice regarding this. Perhaps what I am trying to do is not doable?

Which part of my code could be causing this bottleneck? Is i the logging part or processData ? Can I optimize this somehow to be able to parse the data at regular 1ms intervals?

JonB

@lukutis222
I don't know why you are doing the "1ms timer" stuff, but as per e.g. yesterday's https://forum.qt.io/topic/159883/timerevent-response-overtime we have apparently established that Windows does not maintain a 1ms accuracy timer and you cannot do much about that (you need to at least try making your QTimer use Qt::PreciseTimer). As I say, I am unsure why you need any such accurate timer. Your readyRead()/enqueueData() can do a little extra work to process once you have received 13+ bytes without needing the timer, or with a less frequent timer if you need to recognise more time passed without 13.

I don't think your code is doing very much itself to take up 1ms. If it's only 1 13 byte packet every 1ms I would have thought it would keep up. "Slow" would be the serial->readAll() (if bytes have not yet been physically read, which actually they may have been) and logging_local->Write_to_file(logString); // Save to file, other than those can't see much.

VentureLee

Well... I think maybe it would be helpful if you moved all your Process Functions into a separate thread-safe queue, and then had another processing thread request data from this queue.

I have a couple of points to make: 1. A 1.1ms timer is probably not as reliable as you think; I wouldn't trust a Qt timer to run stably at 1ms intervals. 2. If the thread that receives the readReady signal is being called this frequently, make sure you don't do any processing in this thread. It would be more appropriate to move all processing operations to a separate thread."

aha_1980

@lukutis222 I don't know why you would need a timer.

Your receive packets, and whenever a packet is received you can process it.

But before going that far I'd first try if you can receive all packets at that speed without loss. E.g. by just queing all received bytes for some time (one minute?) in a buffer and check afterwards that there are no missing bytes. As you have a CRC, that should be easy to verify.

If you can receive without loss, then you can push received packets to another process to handle them.

lukutis222

First of all. I would like to thank everyone for their input. I will reply to some statements:

@VentureLee
Regarding using seperate threads, this is exactly what I am doing is that not right? The SerialWorker class that is responsible for receiving and parsing data is on a seperate thread called workerThread whereas the the rest of the logic (in this case logging data to file) happens in main thread (handleProcessedData)

Regarding using the 1ms QTimer I agree that this is not the most optimal solution. I will rewrite the logic without relying on QTimer and post back the results here.

As you have seen from my code, to capture the timestamp (relative_timestamp) I use QElapsedTimer timer_usec that is started at the beggining of a reception.

Then everytime new packet is detected, before writing it to the log I do:


qint64 elapsedNanoseconds = timer_usec.nsecsElapsed(); 
double relative_timestamp = elapsedNanoseconds / 1e9;

Is that a good method for capturing timestamps? Is that guaranteed to have incremented correctly (assuming I am able to read data packets at 1ms intervals). Perhaps I am able to receive and parse data every 1ms but this timer is not to be trusted hence irregular timestamps printed in the logs?

@aha_1980
Thats good suggestion. Il try to send 1000 packets from my measurement device and see if I manage to receive all 1000 packets (without checking the time intervals).

aha_1980

@lukutis222 I can answer your third question: QElapsedTimer has the highest resolution you can get to measure time differences.

I think it will work perfectly for your case.

lukutis222

I have simplified my serialworker greatly:

#include "serialworker.h"
#include <qDebug>
#include <QDateTime>



SerialWorker::SerialWorker(SettingsWindow *settings_ptr, QObject *parent)
    : QObject{parent}
{
    settings_local = settings_ptr;

}

SerialWorker::~SerialWorker() {
    if (serial->isOpen()) {
        serial->close();
    }
    delete serial;
}




void SerialWorker::initialize() {
    qDebug("initialize method \n");

    serial = new QSerialPort(this);
    connect(serial, &QSerialPort::readyRead, this, &SerialWorker::processData);
}




void SerialWorker::processData() {
    static QByteArray buffer; // Persistent buffer for incomplete packets
    buffer.append(serial->readAll()); // Append the new data

    while (buffer.size() >= 13)// Minimum size of a packet
    {
        if (static_cast<uint8_t>(buffer[0]) == 0xAA && static_cast<uint8_t>(buffer[1]) == 0x55)
        {
            if (static_cast<uint8_t>(buffer[2]) == 0x09) { // Validate payload length
                if (buffer.size() < 13)
                    return;           // Wait for more data

                char channel = buffer[3];
                if (channel == static_cast<char>(0x41))
                {
                    float voltage, current;
                    memcpy(&voltage, buffer.mid(4, 4).data(), 4);
                    memcpy(&current, buffer.mid(8, 4).data(), 4);

                    uint8_t calculatedChecksum = 0;
                    for (int i = 2; i < 12; ++i)
                    {
                        calculatedChecksum ^= static_cast<uint8_t>(buffer[i]);
                    }

                    if (calculatedChecksum == static_cast<uint8_t>(buffer[12]))
                    {
                        // Emit the processed data to the main thread
                        emit dataProcessed(channel, voltage, current);
                        buffer.remove(0, 13); // Remove processed packet
                        continue;
                    } else
                    {
                        qDebug() << "Invalid checksum. Skipping packet.";
                        buffer.remove(0, 13); // Remove invalid packet
                    }
                }
                else
                {
                    qDebug() << "Unknown channel. Skipping packet.";
                    buffer.remove(0, 13); // Remove unknown packet
                }
            }
            else
            {
                qDebug() << "Invalid payload length. Skipping packet.";
                buffer.remove(0, 13); // Remove invalid packet
            }
        }
        else
        {
            qDebug() << "Invalid header. Searching for next valid header.";
            int headerIndex = buffer.indexOf(QByteArray::fromHex("AA55"), 1);
            if (headerIndex == -1)
            {
                buffer.clear(); // No valid header found, clear the buffer
            }
            else
            {
                buffer.remove(0, headerIndex); // Move to the next valid header
            }
        }
    }
}




void SerialWorker::write_data(QByteArray data){
    int64_t bytes_written = serial->write(data);
}



bool SerialWorker::Serial_connect(){

    if(serial->open(QIODevice::ReadWrite)){
        qDebug("Connection is succesfull \n");
        return 1;
    }
    else
    {
        //error
        qDebug() << serial->errorString();
    }
    return 0;

}
bool SerialWorker::Serial_disconnect(){
    if(serial->isOpen()){
        serial->close();
        return 1;
    }
    else{
        return 0;
    }
}

bool SerialWorker::Check_if_open(){
    if(serial->isOpen()){
        return 1;
    }
    else{
        return 0;
    }

}

QString SerialWorker::Get_Connect_Port(){
    if(Check_if_open() == 1){
        return serial->portName();
    }
    return "null";
}

I have configured my power analyzer device to send some test packets. This is pseudo code of my power analyzer:

        	   static int counter = 1;
        	   if (counter <= 10000) {
        	       struct DataPacket packet;
        	       uint8_t channel = 0x41; // ASCII 'A'

        	       // Replace voltage and current data with counter value
        	       float dummyVoltage = 0.0f; // Placeholder for voltage (can be set to 0)
        	       float dummyCurrent = (float)(counter); // Use the counter as the 'current' data

        	       // Encode the packet with counter value as the 'current'
        	       encodeDataPacket(channel, dummyVoltage, dummyCurrent, &packet);

        	       // Transmit the packet
        	       HAL_UART_Transmit(&huart4, (uint8_t *)&packet, sizeof(packet), 0xFF);

        	       counter++;
        	   }

It simply sends the packets that contains counter value. For example:

(The counter values are converted to single point floating precision)

0xAA 0x55 0x09 0x41 0x00 0x00 0x00 0x00 0x00 0x00 0x80 0x3F CRC
0xAA 0x55 0x09 0x41 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x40 CRC
0xAA 0x55 0x09 0x41 0x00 0x00 0x00 0x00 0x00 0x00 0x40 0x40 CRC

and etc...

When the processData detects a valid 13 byte packet, it calls:

emit dataProcessed(channel, voltage, current);

Then in my handleProcessedData I do the following:


void MainWindow::handleProcessedData(char channel, float voltage, float current) {
    if (channel == 'A') {
        static int counter = 1;  // Static counter to persist across function calls
                // Measure elapsed time in nanoseconds and calculate relative timestamp
        qint64 elapsedNanoseconds = timer_usec.nsecsElapsed();
        double relative_timestamp = elapsedNanoseconds / 1e9;
                // Format the log string to include counter, timestamp, voltage, and current
        QString logString = QString("%1, %2, %3 V, %4 mA")
                                .arg(counter)                                  // Counter value
                                .arg(QString::number(relative_timestamp, 'f', 6))  // Relative timestamp
                                .arg(QString::number(voltage, 'f', 2))        // Voltage
                                .arg(QString::number(current, 'f', 2));       // Current
                // Write the log string to the file
        logging_local->Write_to_file(logString);
        counter++;  // Increment the counter
    }
}

The purpose of this test is to simply send 10000 packets from my device and confirm if I am able to receive all 10000 packets on my QT application. After running the test, I can open the log file or plot the points to see what happened.

The results:

The QT application managed to receive all 10000 packets and it took roughly 10 seconds to do so which is correct. However, the time intervals between each packet are very inconsistent. Sometimes, it parse multiple packets within a couple of uS (micro seconds) and then some other times it parses some packets within 10's of ms (miliseconds), but the end result is that it received all 10000 packets within the expected time interval (10 seconds).

However, it is still not clear to me why the timestamps are so inconsistent. See a snippet of a log below.

1685, 2.452991, 0.00 V, 1685.00 mA
1686, 2.453939, 0.00 V, 1686.00 mA
1687, 2.454988, 0.00 V, 1687.00 mA
1688, 2.455882, 0.00 V, 1688.00 mA
1689, 2.456991, 0.00 V, 1689.00 mA
1690, 2.457946, 0.00 V, 1690.00 mA
1691, 2.469666, 0.00 V, 1691.00 mA
1692, 2.469721, 0.00 V, 1692.00 mA
1693, 2.469732, 0.00 V, 1693.00 mA
1694, 2.469741, 0.00 V, 1694.00 mA
1695, 2.469749, 0.00 V, 1695.00 mA
1696, 2.469756, 0.00 V, 1696.00 mA
1697, 2.469763, 0.00 V, 1697.00 mA
1698, 2.469770, 0.00 V, 1698.00 mA
1699, 2.469777, 0.00 V, 1699.00 mA
1700, 2.469784, 0.00 V, 1700.00 mA
1701, 2.472088, 0.00 V, 1701.00 mA
1702, 2.472121, 0.00 V, 1702.00 mA
1703, 2.472133, 0.00 V, 1703.00 mA
1704, 2.472141, 0.00 V, 1704.00 mA
1705, 2.507428, 0.00 V, 1705.00 mA
1706, 2.507476, 0.00 V, 1706.00 mA
1707, 2.507486, 0.00 V, 1707.00 mA
1708, 2.507494, 0.00 V, 1708.00 mA
1709, 2.507502, 0.00 V, 1709.00 mA
1710, 2.507509, 0.00 V, 1710.00 mA
1711, 2.507516, 0.00 V, 1711.00 mA
1712, 2.507523, 0.00 V, 1712.00 mA
1713, 2.507530, 0.00 V, 1713.00 mA
1714, 2.507537, 0.00 V, 1714.00 mA
1715, 2.507543, 0.00 V, 1715.00 mA
1716, 2.507551, 0.00 V, 1716.00 mA
1717, 2.507559, 0.00 V, 1717.00 mA
1718, 2.507566, 0.00 V, 1718.00 mA
1719, 2.507573, 0.00 V, 1719.00 mA
1720, 2.507580, 0.00 V, 1720.00 mA
1721, 2.507587, 0.00 V, 1721.00 mA
1722, 2.507594, 0.00 V, 1722.00 mA
1723, 2.507600, 0.00 V, 1723.00 mA
1724, 2.507607, 0.00 V, 1724.00 mA
1725, 2.507614, 0.00 V, 1725.00 mA
1726, 2.507621, 0.00 V, 1726.00 mA
1727, 2.507627, 0.00 V, 1727.00 mA
1728, 2.507634, 0.00 V, 1728.00 mA
1729, 2.507641, 0.00 V, 1729.00 mA
1730, 2.507648, 0.00 V, 1730.00 mA
1731, 2.507654, 0.00 V, 1731.00 mA
1732, 2.507661, 0.00 V, 1732.00 mA
1733, 2.507668, 0.00 V, 1733.00 mA
1734, 2.507674, 0.00 V, 1734.00 mA
1735, 2.507681, 0.00 V, 1735.00 mA
1736, 2.507690, 0.00 V, 1736.00 mA
1737, 2.507696, 0.00 V, 1737.00 mA
1738, 2.507703, 0.00 V, 1738.00 mA
1739, 2.507710, 0.00 V, 1739.00 mA
1740, 2.508267, 0.00 V, 1740.00 mA
1741, 2.509030, 0.00 V, 1741.00 mA
1742, 2.509948, 0.00 V, 1742.00 mA
1743, 2.510947, 0.00 V, 1743.00 mA
1744, 2.512067, 0.00 V, 1744.00 mA
1745, 2.512973, 0.00 V, 1745.00 mA
1746, 2.513943, 0.00 V, 1746.00 mA
1747, 2.516227, 0.00 V, 1747.00 mA
1748, 2.516279, 0.00 V, 1748.00 mA
1749, 2.517017, 0.00 V, 1749.00 mA
1750, 2.517944, 0.00 V, 1750.00 mA
1751, 2.518952, 0.00 V, 1751.00 mA
1752, 2.520017, 0.00 V, 1752.00 mA
1753, 2.520939, 0.00 V, 1753.00 mA
1754, 2.522184, 0.00 V, 1754.00 mA
1755, 2.522946, 0.00 V, 1755.00 mA
1756, 2.523933, 0.00 V, 1756.00 mA
1757, 2.524976, 0.00 V, 1757.00 mA
1758, 2.525922, 0.00 V, 1758.00 mA
1759, 2.526946, 0.00 V, 1759.00 mA
1760, 2.527952, 0.00 V, 1760.00 mA
1761, 2.529366, 0.00 V, 1761.00 mA
1762, 2.532158, 0.00 V, 1762.00 mA
1763, 2.533333, 0.00 V, 1763.00 mA
1764, 2.533362, 0.00 V, 1764.00 mA
1765, 2.533533, 0.00 V, 1765.00 mA
1766, 2.533974, 0.00 V, 1766.00 mA
1767, 2.534923, 0.00 V, 1767.00 mA
1768, 2.535962, 0.00 V, 1768.00 mA
1769, 2.536948, 0.00 V, 1769.00 mA
1770, 2.537951, 0.00 V, 1770.00 mA
1771, 2.539811, 0.00 V, 1771.00 mA
1772, 2.540188, 0.00 V, 1772.00 mA
1773, 2.543413, 0.00 V, 1773.00 mA
1774, 2.545811, 0.00 V, 1774.00 mA
1775, 2.545833, 0.00 V, 1775.00 mA
1776, 2.546210, 0.00 V, 1776.00 mA
1777, 2.546637, 0.00 V, 1777.00 mA
1778, 2.546659, 0.00 V, 1778.00 mA
1779, 2.547927, 0.00 V, 1779.00 mA
1780, 2.549926, 0.00 V, 1780.00 mA
1781, 2.550731, 0.00 V, 1781.00 mA
1782, 2.550756, 0.00 V, 1782.00 mA
1783, 2.551652, 0.00 V, 1783.00 mA
1784, 2.552000, 0.00 V, 1784.00 mA

For example
Between packet number 1685 and packet 1686, timestamp difference is:
2.453939 - 2.452991 = 0.000948 (0.9ms) which is almost correct.

However, if you look at some other packets such as 1704 and 1705:
2.507428 - 2.472141 = 0.035287 (35ms)

I have probed the RX/TX lines with logic analyzer to confirm whether irregularities are coming from the power analyzer side or not and I can confidently confirm that the analyzer works perfectly fine - it transmits packets at precisely every 1ms interavals.

If you want to check the whole log, you can download it via wetransfer link below:
https://we.tl/t-RCxUkiMwa6

At this point, I see only 2 possibilities:

logging_local->Write_to_file(logString); is causing my system to lag for a short amount of time. But that is not likely because logging happens in main thread whereas parsing the data happens in serialworker thread.
My processData method does not work reliably. Even though it succesfully parsed all 10000 packets as were expected, it did not manage to capture them "on time"

aha_1980

@lukutis222 How is the serial line connected to the computer? Via a hardware RS-232 or by an USB-Serial adapter?

Also, which operating system are you using?

lukutis222

@aha_1980
It is connected via USB->Serial connverter. But I am probing the RX TX lines and they seem perfectly fine on the Salae logic analyzer.

I am on Windows 11

JonB

@lukutis222
I am not an expert in hardware, please take this with a pinch of salt. But e.g. as per the https://forum.qt.io/topic/159883/timerevent-response-overtime we mentioned earlier, Windows is not a RTOS. How do you know when the OS might be off doing stuff? How do you know when your thread --- worker or main/UI --- gets scheduled? When the OS hands data to your program? Maybe 35ms is too much, I wouldn't know. But I don't think you can guarantee things to be consistently 1ms apart in real time. My guess is that if you rewrote the code from Qt to native Windows (using the same calls) you would see the same lumpiness? In which case it wouldn't be a Qt issue. But you may know more than I.

Christian Ehrlicher

As @JonB already wrote - if you want to process real time (for whatever reason) you have to use a proper os. There are so much buffers and schedulers involved that your numbers are in the exepcted range.

artwaw

I think I recall previous conversations in the forum, where it was hard to go below 15/16ms boundary with reliable results. You might want to dig a bit and find those posts.

Pl45m4

@JonB said in QSerialPort parsing binary packets at a fast speed:

But e.g. as per the https://forum.qt.io/topic/159883/timerevent-response-overtime we mentioned earlier

@artwaw Jon already "found" and referred to it :)

artwaw

Ah, great! Missed that entry...

Additionally, speaking of Windows configuration - default OS timer setting:

The default platform timer resolution is 15.6ms (15625000ns) and should be used whenever the system is idle. If the timer resolution is increased, processor power management technologies may not be effective. The timer resolution may be
increased due to multimedia playback or graphical animations.
Current Timer Resolution (100ns units) 156250

This probably can be tweaked but I'd rather not to.

kuzulis

No, it is impossible to rely on the packets reception time. You can't take a timestamps from that. As, I understand, do you need a timestamps, for the plotting voltage/current values, e.g. on the plot?

If, so, then you have a two options:

Use a proxy/route MCU which will receive a packages and set a timestamp for every received package, and then route a modified packages to your PC (e.g. with some other your protocol).
Or, as you know that a package received every one millisecond, then just increase a timestamp +1. So, your X-axis will contains some relative time, not absolute. You then can assign that time to everything.))

p. 2 is a more simple and preferable, IMHO. ))

lukutis222

Thanks again for all the responses.

@JonB and @Christian-Ehrlicher
So from what I understood, it is completely not doable to achieve what I want by using my current method.
Regards to using a proper OS, could you please point me in the right direction for some reading material about how to implement an OS within my QT application. I am familliar with FreeRTOS but that is for the embedded devices and not for QT desktop applications. Does the same principles apply here as well? Can I use FreeRTOS for this or this is completely different? When I look up on "google" about QT and OS, not much comes up apart from:
https://doc.qt.io/archives/QtForMCUs-2.2/qtul-using-with-freertos.html

In response to @kuzulis statement. Yes I need the timestamps to plot the current/voltage values. The workarround that you have mentioned (using relative time instead of an actual time when the packet was received) might work. I might have to use this method if I dont manage to find any other way to fix this "properly".