QSerialPort data missing while reading

[biomed12]

Hello,

I am tring to get data over serial port with my desktop program.

Data is corrupted many times like below (I except always 0 data)

"266"
"10"
"10"
"10"
"10"
"10"
"3"
"3"
"2"
"2"
"0"
"0"
"0"
"0"
"2"
"2"
"4"
"4"
"0"
"0"
"4"
"4"
"0"
"0"
"0"
"0"
"0"

Device side code is:

//uart send function
void UARTSend(uint32_t ui32UARTBase, const uint8_t *pui8Buffer, uint32_t ui32Count)
{
    while(ui32Count--)
    {
        UARTCharPut(ui32UARTBase, *pui8Buffer++);
    }
		
}

Qt side is:

union{
    unsigned char bytes[2];
    unsigned int integerPart;
    unsigned char *ptr;
}serData;

void MySoftware::serialReadData()
{
    //read all data
     serialInData = serial->readAll();

//assign it to union
    serData.bytes[0] = (unsigned char)serialInData[0];
    serData.bytes[1] = (unsigned char)serialInData[1];

//debug
    qDebug()<<QString::number(serData.integerPart);

}

note: I used qSerialExamples. I saw some topic which is related with my problem but, there was no certain solution.

[SGaist]

Hi and welcome to devnet,

How are you configuring your serial port ?

What do you mean by corrupted ?
How do you know that you have received a full frame of data ?

[biomed12]

This is my settings. I can read all data with labview. This is also my configuring code which get the definitions from gui.

    serial->setPortName(p.name);
    serial->setBaudRate(p.baudRate);
    serial->setDataBits(p.dataBits);
    serial->setFlowControl(p.flowControl);
    serial->setParity(p.parity);
    serial->setStopBits(p.stopBits);

and how can i know if i get full frame of data or not?

[karti gesar]

QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
QByteArray q = serial.readAll();
block.append(q);

[jsulm]

@biomed12 Well, it would help more if you would say what the values of baudRate, dataBits, ... actually are.
"and how can i know if i get full frame of data or not?" - this is something you need to specify (define a protocol).

[VRonin]

• serialInData = serial->readAll(); does not read ALL you sent, it just read all that is available at that moment
• use QDataStream

void MySoftware::serialReadData()
{
quint32 receivedNumber;
QDataStream readStream(serial);
for(;;){
readStream.startTransaction();
readStream >> receivedNumber;
if(readStream.commitTransaction())
qDebug() << receivedNumber;
else
break;
}
}

I'm not sure how UARTCharPut handles endianness, QDataStream assumes big-endian byte order

[biomed12]

@jsulm

ok, program side is:

    serial->setBaudRate(QSerialPort::Baud57600);
    serial->setStopBits(QSerialPort::TwoStop);
    serial->setPortName("COM8");
    serial->setDataBits(QSerialPort::Data8);
    serial->setFlowControl(QSerialPort::NoFlowControl);
    serial->setParity(QSerialPort::NoParity);

   serial->setReadBufferSize(4);

device side is:

ROM_UARTConfigSetExpClk(UART6_BASE,systemClock , 57600,
                            (UART_CONFIG_WLEN_8 | UART_CONFIG_STOP_TWO|
                             UART_CONFIG_PAR_NONE));

...

//uart send function
void UARTSend(uint32_t ui32UARTBase, const uint8_t *pui8Buffer, uint32_t ui32Count)
{
    while(ui32Count--)
    {
        UARTCharPut(ui32UARTBase, *pui8Buffer++);
				
				while(!UARTBusy(ui32UARTBase))
				{
					
				}
	}	
}
...

//the data will be sended
	sendData[0] = 'a';
	sendData[1] = adcValue.bytes[0];
	sendData[2] = (adcValue.bytes[1] & 15);
	sendData[3] = 'b';

//and send the data
UARTSend(UART6_BASE,sendData , 4);

And I created a protocol. Data frame start with 'a' char and stop with 'b' char. So I added a control loop like below, but i get a lot of break frame. So, I discard the value in this situation but, the data is valueable for me. There are much broken frame...

union{
    unsigned char bytes[2];
    unsigned int integerPart;
    unsigned char *ptr;
}serData;

void MedicalSoftware::serialReadData()
{
    QByteArray serialInData;
    //
    serialInData = serial->readAll();
    //
    if(serialInData.length()!= 4 || serialInData[0] != 'a' || serialInData[3] != 'b')
    {
        
        ErrorMessage("missing"); //this is my custom error debugger which shows message
        serial->clear();
        return;
    }

    serData.bytes[0] =(unsigned char)serialInData[1];
    serData.bytes[1] =(unsigned char)serialInData[2];


    if(plotDataButtonStatus == true)
    {
        plotMainGraph(serData.integerPart);
    }
    serData.integerPart = 0;
}

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@jsulm

ok, program side is:

    serial->setBaudRate(QSerialPort::Baud57600);
    serial->setStopBits(QSerialPort::TwoStop);
    serial->setPortName("COM8");
    serial->setDataBits(QSerialPort::TwoStop);
    serial->setFlowControl(QSerialPort::NoFlowControl);
    serial->setParity(QSerialPort::NoParity);

device side is:

ROM_UARTConfigSetExpClk(UART6_BASE,systemClock , baud,
                            (UART_CONFIG_WLEN_8 | UART_CONFIG_STOP_TWO|
                             UART_CONFIG_PAR_NONE));

Is your serialReadData() connected to readyRead signal ?

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

This is my settings. I can read all data with labview. This is also my configuring code which get the definitions from gui.

    serial->setPortName(p.name);
    serial->setBaudRate(p.baudRate);
    serial->setDataBits(p.dataBits);
    serial->setFlowControl(p.flowControl);
    serial->setParity(p.parity);
    serial->setStopBits(p.stopBits);

and how can i know if i get full frame of data or not?

You can add a frame check sequence for example

https://en.wikipedia.org/wiki/Frame_check_sequence

[biomed12]

@mostefa

I am saying that data frame is broken, not data is not coming. İf the situation you say was true, there would not be any data. But I mention, there is frame drifting.

[mrjj]

@biomed12
Hi
Why not check with other serial app and see what coming ?
http://doc.qt.io/qt-5/qtserialport-terminal-example.html
If that works we can conclude its in app code and if it dont
something up with driver or serial port.
Last i would check if anything to do with Qt. :)
Its not very likely unless you compiled it yourself or have a very interesting distro.

[mostefa]

    serial->setBaudRate(QSerialPort::Baud57600);
    serial->setStopBits(QSerialPort::TwoStop);
    serial->setPortName("COM8");
    serial->setDataBits(QSerialPort::TwoStop);
    serial->setFlowControl(QSerialPort::NoFlowControl);
    serial->setParity(QSerialPort::NoParity);

   serial->setReadBufferSize(4);

device side is:

ROM_UARTConfigSetExpClk(UART6_BASE,systemClock , 57600,
                            (UART_CONFIG_WLEN_8 | UART_CONFIG_STOP_TWO|
                             UART_CONFIG_PAR_NONE));

In your config ,are you sure about , serial->setDataBits(QSerialPort::TwoStop);?

setDataBits contains an enum and can be

QSerialPort::Data5
5
The number of data bits in each character is 5. It is used for Baudot code. It generally only makes sense with older equipment such as teleprinters.
QSerialPort::Data6
6
The number of data bits in each character is 6. It is rarely used.
QSerialPort::Data7
7
The number of data bits in each character is 7. It is used for true ASCII. It generally only makes sense with older equipment such as teleprinters.
QSerialPort::Data8
8
The number of data bits in each character is 8. It is used for most kinds of data, as this size matches the size of a byte. It is almost universally used in newer applications.

have a look at this enum from qt doc

http://doc.qt.io/qt-5/qserialport.html#DataBits-enum

So for me this should be :

serial->setDataBits(QSerialPort::Data8);

[biomed12]

@mostefa

sorry it was an error while editing the source code. it was like you mention

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@mostefa

sorry it was an error while editing the source code. it was like you mention

Just for information are you receiving a part of your data ?

Or you do not receive your data at all and receive another insignificant data?

[mrjj]

Hi
Adding to @mostefa , your logic is not optimal ( in serialReadData)
serialInData = serial->readAll(); // this might not read all and next signal will override data already received.
//
if(serialInData.length()!= 4 || serialInData[0] != 'a' || serialInData[3] != 'b')

a more common design is

void MedicalSoftware::serialReadData()
{
//// -> this goes as memeber of class (.h file) QByteArray serialInData;
//
serialInData += serial->readAll(); // we ADD

if ( serialInData.size() >= COUNT_EXPECTED ) { // process
// check format etc
if(serialInData.length()!= 4 || serialInData[0] != 'a' || serialInData[3] != '
serialInData.clear();// clear buffer for next full read. Note if you have excess data u need to handle it here.

[mostefa]

Adding to what @mrjj said

When readall is called you may received just part of your frame,

For example , we are going to send "HELLO WORLD" on the serial port

this can be sent in two times :

at time t0 "HEL" is received
at time t1 "O WORLD" is received

If you are doing your readall, at time t0 you can receive just part of data

To avoid this you can use readyRead signal, and QByteArray buffer

connect(nameOfyourSerialPort,SIGNAL(readyRead(),this,SLOT(onPartOfFrameAvailable());

QByteArray mWholeData;//var member this is our buffer

void onPartOfFrameAvailable()
{
     mWholeData.append(nameOfyourSerialPort->readAll());
     qDebug() << "for the moment wholeData contains" << mWholeData;
}

If you send "HELLO WORLD" the result will be for example:

First output : for the moment wholedata contains "HEL";
Second output : for the moment wholedata contains "HELLO WORL"
Thirs output : for the moment wholedata contains "HELLO WORLD";

[biomed12]

@mostefa
Yes, I have this trouble definitely. I will try and share the results.

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

[biomed12]

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

Getting larger than what?

I don't understand could you provide more details please?

[biomed12]

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

Getting larger than what?

I don't understand could you provide more details please?

like
axx
axxxaxa
axaxaxxxx

I am trying now this algorithm, what is your opinion?

QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    if(serialData.length()<4 || serialData[0] != 'a' || serialData[3] != 'b')
    {
        return;
    }

    serData.bytes[0] = serialData[1];
    serData.bytes[1] = serialData[2];
    qDebug()<<QString::number(serData.intPart);
    serData.intPart = 0;
    //qDebug()<<QString::number(data.length());
    console->append(QString::fromLatin1(serialData));
    serialData.clear();
}

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

Getting larger than what?

I don't understand could you provide more details please?

like
axx
axxxaxa
axaxaxxxx

I am trying now this algorithm, what is your opinion?

QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    if(serialData.length()<4 || serialData[0] != 'a' || serialData[3] != 'b')
    {
        return;
    }

    serData.bytes[0] = serialData[1];
    serData.bytes[1] = serialData[2];
    qDebug()<<QString::number(serData.intPart);
    serData.intPart = 0;
    //qDebug()<<QString::number(data.length());
    console->append(QString::fromLatin1(serialData));
    serialData.clear();
}

I am not able to understand all what you are doing

Why don't we start with simple things:

So

1. When you readData is CALLED ? , is it a SLOT ? , is it connected to readyRead signal?

2. What is the output of this code?

For example we keep only this part of the code :

// i am trying only to keepthe basic lines for reading serial port
QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    qDebug() << "receivedthat for the moment = " << serialData;
}

[biomed12]

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

Getting larger than what?

I don't understand could you provide more details please?

like
axx
axxxaxa
axaxaxxxx

I am trying now this algorithm, what is your opinion?

QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    if(serialData.length()<4 || serialData[0] != 'a' || serialData[3] != 'b')
    {
        return;
    }

    serData.bytes[0] = serialData[1];
    serData.bytes[1] = serialData[2];
    qDebug()<<QString::number(serData.intPart);
    serData.intPart = 0;
    //qDebug()<<QString::number(data.length());
    console->append(QString::fromLatin1(serialData));
    serialData.clear();
}

I am not able to understand all what you are doing

Why don't we start with simple things:

So

1. When you readData is CALLED ? , is it a SLOT ? , is it connected to readyRead signal?

2. What is the output of this code?

For example we keep only this part of the code :

// i am trying only to keepthe basic lines for reading serial port
QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    qDebug() << "receivedthat for the moment = " << serialData;
}

Guy,

1. Of course the signal slot mechanism such you mention.
2. The code is always append received data to buffer. But, I need just four bytes. My microcontroller send datas like that:

starts with 'a' char and, 2 data bytes which include what i want, and terminates with 'b'

for example; "affb", i need the ff data.

So, your codes output like that:

affb
affbaffb
affbaffbaffb

because you are appending new datas consistently the buffer.

[kuzulis]

Foo::onReadyRead()
{
    for (;;) {
        if (serial->bytesAvailable() <  4)
            return; 
       const QByteArray peeked = serial-> peek(4);
       QDataStream ds(peeked);
       qint8 a;
       qint8 b;
       qint16 data;
       ds >> a >> data >> b;   
       if (a == 'a' && b == 'b') {
           // all good, data variable contains ADC value
           serial->read(4); // dummy read whole frame
       } else {
           serial->read(1);  // try to synchronise
       }
    }  //end for
}

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

@biomed12 said in QSerialPort data missing while reading:

Yes, I have this trouble definitely. I will try and share the results.

Ok keep us informed ;)

ooppss.. Qbyte array is getting larger always.

Getting larger than what?

I don't understand could you provide more details please?

like
axx
axxxaxa
axaxaxxxx

I am trying now this algorithm, what is your opinion?

QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    if(serialData.length()<4 || serialData[0] != 'a' || serialData[3] != 'b')
    {
        return;
    }

    serData.bytes[0] = serialData[1];
    serData.bytes[1] = serialData[2];
    qDebug()<<QString::number(serData.intPart);
    serData.intPart = 0;
    //qDebug()<<QString::number(data.length());
    console->append(QString::fromLatin1(serialData));
    serialData.clear();
}

I am not able to understand all what you are doing

Why don't we start with simple things:

So

1. When you readData is CALLED ? , is it a SLOT ? , is it connected to readyRead signal?

2. What is the output of this code?

For example we keep only this part of the code :

// i am trying only to keepthe basic lines for reading serial port
QByteArray serialData;
union
{
    unsigned char bytes[2];
    unsigned int intPart;

}serData;
//! [7]
void MainWindow::readData()
{
    serialData.append(serial->readAll());
    qDebug() << "receivedthat for the moment = " << serialData;
}

Guy,

1. Of course the signal slot mechanism such you mention.
2. The code is always append received data to buffer. But, I need just four bytes. My microcontroller send datas like that:

starts with 'a' char and, 2 data bytes which include what i want, and terminates with 'b'

for example; "affb", i need the ff data.

So, your codes output like that:

affb
affbaffb
affbaffbaffb

because you are appending new datas consistently the buffer.

Ok , it's clear for now

QByteArray dataToTreat;
void MainWindow::readData()
{
    serialData.append(serial->readAll());

    if(serialData.startWith('a')
    {
	if(serialData.length() > 4)
	{
		 dataToTreat = serialData.left(4);
		serialData.remove(0,4);
	}
    }
    else
    {
           // Not a normal case
     }

// DO WHAT YOU WANT
    serData.bytes[0] = serialData[1];
    serData.bytes[1] = serialData[2];
    qDebug()<<QString::number(serData.intPart);
    serData.intPart = 0;
    //qDebug()<<QString::number(data.length());
    console->append(QString::fromLatin1(serialData));
    serialData.clear();
}

[biomed12]

@mostefa

I want to get the second and third byte of data to in another union type TO MAKE it unsigned int like 4095.

[mostefa]

even with the last code i proposed?
Or you can try to use the code of @kuzulis , this can be the best way to achieve what you want

[biomed12]

@mostefa said in QSerialPort data missing while reading:

even with the last code i proposed?
Or you can try to use the code of @kuzulis , this can be the best way to achieve what you want

baü
baü
baÿ
baÿ
baÿ
baÿ
baþ
baþ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ

Again. always framing error.

[biomed12]

@kuzulis said in QSerialPort data missing while reading:

//your code here
```Foo:onReadyRead()
{
    for (;;) {
        if (serial->bytesAvailable() <  4)
            return; 
       QByteArray peeked = serial-> peek(4);
       QDataStream ds(peeked);
       qint8 a;
       qint8 b;
       qint16 data;
       ds >> a >> data >> b;   
       if (a == 'a' && b == 'b') {
           // all good, data variable contains ADC value
           serial-> read(4); // dummy read whole frame
       } else {
           serial-> read(1);  // try 
       }
  }  //end for
}

how to leave from for(;;)?

[mostefa]

@biomed12 said in QSerialPort data missing while reading:

@mostefa said in QSerialPort data missing while reading:

even with the last code i proposed?
Or you can try to use the code of @kuzulis , this can be the best way to achieve what you want

baü
baü
baÿ
baÿ
baÿ
baÿ
baþ
baþ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ
baÿ

Again. always framing error.

baÿ

Looks like ba is correctly in hex , but ÿ is in ascii

The hex code equivalent to ÿ is ff

http://www.rapidtables.com/convert/number/ascii-to-hex.htm

this will lead actually to baffbaffbaff... and i think that this is what you need?

So there is a conversion problem in your code( or maybe just a display problem)

[kuzulis]

@biomed12 said in QSerialPort data missing while reading:

how to leave from for(;;)?

if (serial->bytesAvailable() <  4)
            return;

@biomed12 said in QSerialPort data missing while reading:

Again. always framing error.

Are you sure, your setting:

serial->setStopBits(QSerialPort::TwoStop);

were applied?

@biomed12 said in QSerialPort data missing while reading:

baÿ
baÿ
baÿ

Try to output the data variable instead...

PS1: QSerialPort is a buffered class and nothing missed.

PS2:

serial->setReadBufferSize(4);

need to remove it if you do not know what for..

[biomed12]

@mostefa

I solved framing errror problem with this algorithm:

Device send datas like:

sendData[0] = (char)'a';
sendData[1] = (char)'k';
sendData[2] = adcValue.bytes[0];
sendData[3] = (adcValue.bytes[1] & 15);

Read data algorithm:

union{
    unsigned char bytes[2];
    unsigned int intPart;
    unsigned char *ptr;
}serData;

void MedicalSoftware::serialReadData()
{

    if(serial->bytesAvailable()<4)
    {
        return;
    }

    QByteArray serialInData = serial->readAll();

    //my algorithm
    if(serialInData[0] == 'a' && serialInData[1] == 'k')
    {
        serData.bytes[0] = serialInData[2];
        serData.bytes[1] = serialInData[3];

    }else if(serialInData[2] == 'a' && serialInData[3] == 'k')
    {
        serData.bytes[0] = serialInData[0];
        serData.bytes[1] = serialInData[1];
    }
    else if(serialInData[1] == 'a' && serialInData[2] == 'k')
    {
        serial->read(1);
        return;
    }else if(serialInData[0] == 'k' && serialInData[3] == 'a')
    {
        serData.bytes[0] = serialInData[1];
        serData.bytes[1] = serialInData[2];

    }
  
    if(plotDataButtonStatus == true)
    {
        plotMainGraph(serData.intPart);
    }
    serData.intPart = 0;
}

I have another problem now:) but it is another topic. Thanks

my new problem if you wonder:

http://stackoverflow.com/questions/42742033/qcustomplot-huge-amount-of-data-plotting