QT responds slow when reading in serial data on the Rpi.

bask185

I am testing my new MMI system based on a Rpi 3. My Qt app runs perfectly fine except on one point.

When I navigate through the menu, I can go to the Inputs page. This page refreshes @ 5Hz and the machine sends bytes to fill the fill the display. We are talking about 40 or less bytes, which are send at 115200bps. The machine however is limited in his hardware and cannot send a continous stream of bytes. There is a delay of ~2ms between bytes.

When the machine is sending soo many bytes, the Rpi reacts really really really slow. I think this problem is not really the fault of the Pi's hardware, but in the Qt's way of reading the data. This code is not something I came up with.

The code looks as follows

void MainWindow::serialReceived()
{
    QDataStream serialStream(serial);
    unsigned char b;
    for(;;){
       serialStream.startTransaction();
       serialStream >> b;
       if(serialStream.commitTransaction()) {

        ... code...

The delay between 2 following bytes is never less than 2ms. When the machine is sending data continuously, Qt does not escape the for-loop in this time. And I believe that the Raspberry is in the worst case 40 x 2ms x 5Hz = 400ms occupied with being stuck in a for-loop and not doing important stuff. IIRC most electronics hate infinite for loops.

I have verified this by adding: qDebug() << "Entering the for-loop of death"; right in front of the for-loop. And when I don't see this line of text between 40 received bytes, I know that the Rpi has spend atleast 80ms waiting on bytes inside that for-loop

Therefor I need a different way to read in serial bytes. Either the line "serialStream.startTransaction();" or "serialStream.commitTransaction()" waits for <2ms on a byte if there are none present in the buffer. The best solution would be to change the Qt parameter of the function in question so the code would not wait this incredibly long.

bask185

@mrjj said in QT responds slow when reading in serial data on the Rpi.:

@bask185
You can use the
the synchronously API
http://doc.qt.io/qt-5/qtserialport-creadersync-example.html
so you can do
 while (serialPort.waitForReadyRead(5000))
        readData.append(serialPort.readAll());
Which is like your board does. synchronously.

This will of course strangulate your GUI but that fact seems not so important to you... :)

This seems promising! I will give it a try if my program freezes again. The big freezes I had, were caused by the machine which would send a byte which my program would not handle properly, this only occured on one special page. It turned out that somebody not gave me all the pieces of the puzzle. Regardless my program still spends a whopping 100ms in a never ending for-loop when 50 bytes are coming. But now this not seems to have any significant impact... yet ;)

I now have sufficient information to solve another part of the great puzzle, so I'll add this on the solved stack ;)

next step: graphic widgets

jsulm

@bask185 I'm not sure what's the point to call startTransaction() inside the loop all the time.
Why not just call http://doc.qt.io/qt-5/qiodevice.html#readAll inside serialReceived()? I gues serialReceived() is connected to QSerialPort::readyRead() signal?

bask185

@jsulm said in QT responds slow when reading in serial data on the Rpi.:

@bask185 I'm not sure what's the point to call startTransaction() inside the loop all the time.
Why not just call http://doc.qt.io/qt-5/qiodevice.html#readAll inside serialReceived()? I gues serialReceived() is connected to QSerialPort::readyRead() signal?

@jsulm The point of all code I ever write or use is: to get something to work. I did not devise this code myself, but it came from this forum. I am quite dissappointed in Qt's way of reading in serial data because it is so extremely over-complicated. In arduino or processing you merely do

if (serial.available() > 0) {   // this does the exact same thing as what I posted in the OP, and is more efficient
   char c = serial.read();

And when I read bytes I could store them in alot of different variable types.

Strangly I have not yet managed to reproduce the bug, but my MMI system is hooked up at the simplest machine we have. So I will test more machines and I will have a look at the link you sent me.

EDIT
I am trying to change my code but it not really working.

void MainWindow::serialReceived()  // gets called every time when a byte is received
{
    unsigned char b;
    QByteArray array = serial->readAll();
    b = (unsigned char)array[0]; // casts signed char into an unsigned char
 
   QString bAsHex = QString("%1").arg(b, 0, 16);
   qDebug() << "<< " << bAsHex << ". " << array << sizeof(array);

.. rest of code...

But it does not work and I do not understand what is happening.
An output is as follows.

<<  "4" .  "\x04\x0E\x01\x01\x0E\x02\x01\x0E\x03\x01\x0E\x04\x01\x0E\x05\x01\r\x01\x19\r\x02\x19\r\x03\x19\r\x04\x19\r\x05\x19\x18\n\x10\x01\x0E\x01\x05\x1A\x01\t\x85wheel trueing robot version\x02\x1A\x0F\nDAD 1.04 pst \x0E\x01\x1A\x01\f\x85 \x02\x85 \x02\x0E\x01\x1A\x01\x0E\x85 \x02\x85 \x02\x0E\x01\x1A\x01\x0E\x85press green st" 4

variable b holds the correct value, but the entire array is full. Serial received gets called every time when a byte is received. It was my thought that with the first byte the array would hold only 1 value, then I would stuff that value in b, then do something with it and than return to whatever it is that Qt is doing in the mean time until ~2ms later a new byte is received and the cycle gets repeated

At first I was thinking that my entire application runs slower than 2ms, meaning that the machine is actually sending bytes faster than I can process them. the variable 'array' is a local variable so this is the only thing what I can think of that makes sense. But if I count the amount of array elements this cannot be the case and something else is causing problems.

What I am really not getting is that after Qdebug() << array gives me like 50+ elements back while at the same line qDebug() << sizeof(array) is still '4'. Because the variable array is local, this cannot continue to grow.

I understand nothing of this behaviour, can you tell me what is happening, what I am doing wrong and how I can fix this?

jsulm

@bask185 "extremely over-complicated" - I disagree. The thing is: Qt is event driven framework (as most other frameworks providing UI functionality are). That means you usually program asynchronously with Qt.
To read from serial you just connect a slot to readyRead() signal and call readAll() there - that's all. Why is it "extremely over-complicated"? How would you do it in a GUI application without blocking the UI? Alternative is to use an extra thread for reading serial and do polling there (what you do in your last example), which is possible too with Qt. But this solution isn't easier than the first one.

sierdzio

QSerialPort inherits QIODevice and thus supports asynchronous API. No need for an infinite loop at all. See readyRead() signal.

bask185

I am already using the ready read signal. I don't even know how else I could do it. I use this in the constructor (or setup idk what to call it)

connect(serial, &QSerialPort::readyRead, this, &MainWindow::serialReceived);

But the page says thats the signal get emitted every time, new data is received. "This signal is emitted once every time new data is available for reading from the device's current read channel."

So that means that my function "serialReceived" gets called with every received byte, right?

All I want is this: when I receive one byte, I want to put in a unsigned char variable 'b' and that is it. How do I do this?

Right now my code is as follows:

void MainWindow::serialReceived()
{
    unsigned char b;
   
    QByteArray array = serial->readAll();
    b = (unsigned char)array[0];
    
    QString bAsHex = QString("%1").arg(b, 0, 16);
    qDebug() << "byte b =" << bAsHex << ", array =" << array ;

Part of the output I am getting:

byte b = "31" , array = "1.1.3 ("
byte b = "53" , array = "S"      // << this is how everything should look like. Just one single byte.
byte b = "65" , array = "ep  6 2012 - 11:04:44)\r\n\r\nCPU   : Nios-II\r\nSYSID : 377370c0, Wed Sep "
byte b = "31" , array = "12 07:59:51 2012\r\nBOARD : Micro-key MCM2291 (for BIM)\r\n"
byte b = "48" , array = "Hit 'H' ke"
byte b = "79" , array = "y to stop autoboot:  5 "
byte b = "8" , array = "\b\b\b"
byte b = "20" , array = " 4 "

If serialReceived() is called as soon as there is a byte received, how can it be that 'array' holds more than just 1 byte?

sierdzio

@bask185 said in QT responds slow when reading in serial data on the Rpi.:

If serialReceived() is called as soon as there is a byte received, how can it be that 'array' holds more than just 1 byte?

Your assumption may be wrong. Subclass decides when to call readyRead(). For example, in networking (QNetworkAccessManager), readyRead() is called only after a full TCP packet is received.

In your case I suspect that this is what happens: a byte comes in, serial emits readyRead(). While your slot is being called and processed, more data comes in, and then you read it using readAll(). This is actually quite good, because it means your serialReceived() slot will be called less frequently and code will execute quicker.

If you want single bytes, you can always process the QByteArray byte by byte, right? just treat it as input buffer.

If you really want to read the serial port semi-synchornously, don't connect to readyRead() and use this (pseudo-)code instead:

forever {
  char temp = 0;
  const qint64 result = serial->read(&temp, 1); // using http://doc.qt.io/qt-5/qiodevice.html#read
  if (temp != -1) {
    b = temp; // There! Single byte is read
    // ... further processing of your single byte
  }
  QCoreApplication::processEvents(); // run the event loop so that the application GUI 
  // does not freeze. Or run the code above in a separate thread, like you should
}

I do not recommend this solution! It is better and more safe to use readyRead().

bask185

You know what is funny, in the very beginning I wanted to use that very same read function, but I did not understand what to do with that pointer in
data: qint64 QIODevice::read(char *data, qint64 maxSize)

I asked on this forum specifically how to use that function and I had one answer which said something like: "how about you don't and you use the serialstream instead", followed by the working code example I am using at the moment. People keep telling how wonderful each and every Qt function is, but when you ask, "how can I use this function" suddenly nobody knows and provides with some alternative. Pretty much like Jsulm is helping me but is not actually 'helping' (no offense).

Anyways back to the problem solving part. I understand how the code works. You continuously do 2 things, you run everything of the application using QCoreApplication::processEvents(); and you continuously execute the read command which does something when it has a byte and does not return -1. This is essentially the exact same thing as using if(serial.available() > 0) in processing or arduino.

But this raises a question, where exactly do I need to put the code?
Is it at the end of MainWindow::MainWindow(QWidget *parent) like this?

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    qDebug().noquote() << "hello raspberry pi";
    serial = new QSerialPort();
...
...
  forever {
    char temp = 0;
    const qint64 result = serial->read(&temp, 1); // using http://doc.qt.io/qt-5/qiodevice.html#read
    if (temp != -1) {
      b = temp; // There! Single byte is read
      // ... further processing of your single byte
    }
    QCoreApplication::processEvents(); // run the event loop so that the application GUI 
    // does not freeze. Or run the code above in a separate thread, like you should
  }
}

But I have an idea of which I would like to hear your opinion about. I want to make a time-based event which gets called every millisecond. Than in that event I can poll

const qint64 result = serial->read(&temp, 1);

Assuming my Rpi handle the rest of the function under 1ms, it could be more efficient than executing QCoreApplication::processEvents() as a round robin task??

@jsulm said in QT responds slow when reading in serial data on the Rpi.:

@bask185 "extremely over-complicated" - I disagree. The thing is: Qt is event driven framework (as most other frameworks providing UI functionality are). That means you usually program asynchronously with Qt.
To read from serial you just connect a slot to readyRead() signal and call readAll() there - that's all. Why is it "extremely over-complicated"? How would you do it in a GUI application without blocking the UI? Alternative is to use an extra thread for reading serial and do polling there (what you do in your last example), which is possible too with Qt. But this solution isn't easier than the first one.

My processing sketch was also very much event driven. Mouseclicks/releases would give an event when I would press a button. And in the mainloop I had 1 single task running and that was checking if there were bytes received. For this there also exists a special eventhandler but that can be incovenient because it can be called between every two lines of code which could screw up. I have written alot of arduino and processing programs with empty main loops.

The reason why I called this readyRead crap over-complicated is that it cannot preform that one puny little task I want it to do. It's respons time is too slow, it is nice and all that it works for reading in TCP packets, but you cannot even read 1 byte every 2ms efficiently. I need an event which gets called the split second a byte is received and readyRead does no such thing. The fact that I am already here on this forum to ask people how I can read in 1 single byte every 2ms because I cannot figure it out, a question for which you have not provided a concrete answer yet, that is what makes this over-complicated.

And also that what i said in the top of this post. I ask on this forum, how I can use that function... no answers -> nobody seems to know -> apparantly 'over-complicated'

mrjj

@bask185
You can use the
the synchronously API
http://doc.qt.io/qt-5/qtserialport-creadersync-example.html
so you can do

 while (serialPort.waitForReadyRead(5000))
        readData.append(serialPort.readAll());

Which is like your board does. synchronously.

This will of course strangulate your GUI but that fact seems not so important to you... :)

J.Hilk

@bask185 hi, Ill post a view snippets from one of my projects, it might be usefull for you.

serialThread = new QThread;
    sPort = new SerialPort;

    sPort->moveToThread(serialThread);

    connect(serialThread, &QThread::started, sPort, &SerialPort::initSerialPort);

....

    connect(serialThread, &QThread::finished, serialThread, &QThread::deleteLater);
    connect(serialThread, &QThread::finished, sPort, &SerialPort::deleteLater);

class SerialPort : public QObject
{
    Q_OBJECT
public:
    SerialPort();
    ~SerialPort();

public slots:
    void initSerialPort();


private slots:
    void readData();

private:
    QSerialPort *sPort;
}

void SerialPort::initSerialPort()
{
    sPort = new QSerialPort(this);
    connect(sPort, &QSerialPort::readyRead, this, &SerialPort::readData);
    
    sPort->setPortName(settings.name);
    ....
    sPort->setFlowControl(settings.flowControl);
    if(sPort->open(QIODevice::ReadWrite)){
       ....
    }

void SerialPort::readData()
{
    QByteArray data = sPort->readAll();

 
    for(int i = 0; i < data.size(); i++)
        qDebug()  << QString::number(static_cast<unsigned char>(data[i]),16).toUpper().rightJustified(2,'0')+ " ");

.....
}

Keep in mind even threaded, you might get more than 1 Byte per ReadyRead signal. The port is checked once every event-loop cycle and if theres new data available the signal is fired. That means while your event loop is running new data might still arive. That than will trigger a 2nd ReadyRead signal, next cycle.

bask185

@mrjj said in QT responds slow when reading in serial data on the Rpi.:

@bask185
You can use the
the synchronously API
http://doc.qt.io/qt-5/qtserialport-creadersync-example.html
so you can do
 while (serialPort.waitForReadyRead(5000))
        readData.append(serialPort.readAll());
Which is like your board does. synchronously.

This will of course strangulate your GUI but that fact seems not so important to you... :)

This seems promising! I will give it a try if my program freezes again. The big freezes I had, were caused by the machine which would send a byte which my program would not handle properly, this only occured on one special page. It turned out that somebody not gave me all the pieces of the puzzle. Regardless my program still spends a whopping 100ms in a never ending for-loop when 50 bytes are coming. But now this not seems to have any significant impact... yet ;)

I now have sufficient information to solve another part of the great puzzle, so I'll add this on the solved stack ;)

next step: graphic widgets