SerialCommunication parsing failing

[PaulMax]

Hi everyone!

I am working on an arduino uno board with an incremental encoder module which I use to detect movement direction and elapsed time between each pulse generated by the encoder which communicates via serial connection and divides two values by comma: a counter int which handles direction and a time int which handles the elapsed time between pulses

#include <SoftwareSerial.h>


unsigned long time;
unsigned long previousTime;
unsigned long elapsedTime;
int contador;
double aceleracion;


int ANTERIORclk;
int VALORclk;
int VALORdt;

//Locación de los pines
int clk=7;
int dt=4;



void setup() {
  pinMode(clk, INPUT);
  pinMode(dt, INPUT);
  ANTERIORclk=digitalRead(clk);
  previousTime= 0;
  contador = 0;

  Serial.begin(9600);
}

void loop() {
  
  time = millis();
  elapsedTime = time;
  
  VALORclk=digitalRead(clk);

  //Algoritmo de detección de movimiento
  if(VALORclk!=ANTERIORclk){
    
    //Si hay movimiento se realizan calculos
    VALORdt=digitalRead(dt);
    elapsedTime -= previousTime;
   
    if(VALORdt!=VALORclk){
    contador++; 

    }else {
      if (contador>0){
      contador--;
      }
    }
    
    //Imprimir resultados
    printResults();
    
    //Se actualizan las lecturas
    ANTERIORclk=VALORclk;  
    previousTime= time;
    delayMicroseconds(.01);
  }//Fin de if y de algoritmo de detección de movimiento
}

  void printResults(){

    Serial.print(contador);
    Serial.print(",\n");
    Serial.print(elapsedTime);
    Serial.print(",\n\n");
    
  }

Then in qt following one of the QSerialPort tutorials from QT I have a QtWidgets project here: https://github.com/PaulMaxAvalosAguilar/Encoder which as usual it has a MainWindow and for Serial Connection it relies on Connection/SerialConnectionn class which does a connection to readyRead signal in its constructor to the next function:

void SerialConnection::receiveDeviceInfo()
{
    QStringList bufferSplit;

    //serial is of type QSerialPort 
    QByteArray data  = serial->readAll();
    QString parsedDATA;
    double counter = 0;
    double time = 0;

   //serialbuffer is of type QString
    serialbuffer += QString::fromStdString(data.toStdString());
    bufferSplit = serialbuffer.split(",");

    if(!(bufferSplit.length()<3))
    {
        serialbuffer="";

        parsedDATA= bufferSplit.at(0);
        counter = parsedDATA.toDouble();
        parsedDATA= bufferSplit.at(1);
        time = parsedDATA.toDouble();
        qDebug()<<counter<<" -- "<<time<<" "<<bufferSplit.at(2);
    }
}

As you can see the function intends to separate the values by commas so that I can assign it to two variables which later can be displayed on GUI.... The problem, however, is that it all works fine for printing the values but often when I move the encoder a little faster some data is not displayed correctly e.g.:

QApplication: invalid style override passed, ignoring it.
New static SerialConnection Object created
connect() method called
Serial port Opened.... "ttyACM0"
1 -- 342 "\n\n"
2 -- 294 "\n\n"
3 -- 149 ""
4 -- 132 "\n\n"
5 -- 84 "\n\n"
6 -- 78 "\n\n"
7 -- 125 "\n\n"
6 -- 139 "\n\n"
7 -- 1945 "\n\n"
8 -- 89 "\n\n"
9 -- 43 ""
10 -- 32 "\n\n"
11 -- 25 "\n\n"
12 -- 17 "\n"
13 -- 15 "\n\n"
14 -- 990 ""
15 -- 6 "\n\n1"
6 -- 7 "\n\n1"
7 -- 5 "\n\n1"
8 -- 7 "\n\n1"
9 -- 4 "\n\n2"
0 -- 4 "\n\n2"
1 -- 3 "\n\n2"
2 -- 2 "\n\n"
23 -- 2 "\n\n"
24 -- 4 "\n\n"
25 -- 6 "\n\n"
26 -- 7 "\n\n"
27 -- 6 "\n\n"
28 -- 7 "\n\n"
29 -- 4 "\n\n"
30 -- 5 "\n"
31 -- 2 "\n"
30 -- 3 "\n"
31 -- 9 "\n"
30 -- 7 "\n"
31 -- 487 "\n\n"
32 -- 16 "\n\n"
33 -- 13 "\n\n"
34 -- 12 "\n\n3"
5 -- 5 "\n\n3"
4 -- 1 "\n\n3"
5 -- 0 "\n\n3"
6 -- 1 "\n\n"
35 -- 0 "\n\n"
36 -- 1 "\n\n"
37 -- 2 "\n\n"
38 -- 1 "\n\n"
37 -- 0 "\n\n"
38 -- 7 "\n\n"
39 -- 10 "\n"
40 -- 8 ""
41 -- 8 ""
42 -- 456 "\n"
43 -- 7 "\n"
44 -- 4 "\n"
45 -- 4 ""
46 -- 3 ""
47 -- 2 ""
48 -- 2 ""
49 -- 2 ""
50 -- 2 ""
51 -- 2 ""
52 -- 2 ""
53 -- 3 "\n\n5"
4 -- 3 "\n\n5"
5 -- 7 "\n\n5"
6 -- 9 "\n\n5"
7 -- 8 "\n\n5"
8 -- 9 "\n\n5"
9 -- 12 "\n\n"
60 -- 3330 ""
Serial Connection Deleted

You can see how whenever I move the encoder "too fast" some digits are passed to the [2] element in the list then so the next counter is display incorrectly because its last digits moved to the [2] element of the list are missing.

The curious thing is that this never happens when using QT QPlainTextEdit example which no matters how fast I move the encoder always displays everything fine.

Hope anyone can help me

[aha_1980]

Hi @PaulMax,

I guess your problem is, that you have some expectations on readyRead that are not met.

readyRead just tells you, that at least one new char has arrived, but not how many. before splitting you have to make sure your buffer is valid.

you can do this by collecting all incoming chars to a new buffer and evaluate them up to your last separation char. all following chars are kept for the next iteration.

that's also the reason why it works with QTextEdit.

[PaulMax]

@aha_1980 Where can I see QTextEdit or QPlainTextEdit implementation of this buffer I haven't found it anywhere, to give me an idea

[mrjj]

@PaulMax
Hi, its just its internal string buffer. ( and you append to it on each read)
You can easy have the same if you
QByteArray data = serial->readAll();
->
QByteArray data;// move as class memeber
void SerialConnection::receiveDeviceInfo() {
data += serial->readAll();
if (data.size() >= min_val ) {
// process data
}

[aha_1980]

@PaulMax said in SerialCommunication parsing failing:

@aha_1980 Where can I see QTextEdit or QPlainTextEdit implementation of this buffer I haven't found it anywhere, to give me an idea

The QTextEdit itself is the buffer.

Imagine the following example: A microcontroller sends the message: Hello World to Qt\n. On the other side, the message can:

• appear in one chunk: Hello World to Qt\n
• char by char: 'H', 'e', 'l', 'l', 'o', ... 'Q', 't', '\n'
• or in strings of mixed size: "Hel", "o W", "orld t", "o Q", "t\n"

You have to be aware of this (and there is no control where the chunks begin and end)! So before calling split on a chunk of received data in readyRead you have to verify what you received. It is also possible to peek into QSerialPorts buffer and only read (parts of) the data when there is enough in the receive buffer.

Now it's up to you and your serial protocol.