Issues from Serialread to Qcustomplot display.

srikanth2006

Hi ,
I have a project which needs to read adc at 115200 baudrate on serial and display it on Qcustomplot on QT. Qcustomplot is on main thread and serialread is on another thread. I am having hard time to catch up the incoming data to display. Iam also using a hdlc wrapper in other thread which extracts the data from serial read.
The code for serial side is briefly below:
connect(this->mySerialObj,&QSerialPort::readyRead,this,&SerialCommunication::readFromSerial);
QByteArray data = mySerialObj->readAll();
emit dataReceived(data);
HDLC wraps and sends the extracted data to Qcustomplot on main thread. I use blocking connection. SO GUI is smooth and plots the graph, however it slow in catching the incoming serial data.
At 115200baud, should the customplot be displaying the data atleast 115200/10 = 11520 bytes persecond. even with 7 bytes wrapper overhead(CRC etc) it should have atleast 11520/7 = 1645 bytes per second. But at the moment the customplot displays max 200bytes a second and when i stop sendingthe data from microcontroller, qcustomplot still takes quite a while to display all data filled in Qbyte array buffer.
Any suggestion on how to overcome this? THanks inadvance.

mrjj

Hi
You could inspect its code since it rather small. And see if you can find the bottleneck.
Or try on the author's forum.
https://www.qcustomplot.com/index.php/support/forum

He should know if anything can be done to speed it up.

Without profiling your actual running code, its hard to give suggestions on what is
the root cause. It sounds like you did it right.

MikhailG

@srikanth2006 would you consider to display average chunks of data is it critical in your project?

srikanth2006

@mrjj THank you for reply. This is my first post on Qt and was waiting for first reply. SO thank you for that.
THank you for letting me know that i have done the correct way.
I have been trying to debug and wondering why QT is not able to grab a serial data fast enough and display on graph quick. I use corei7 8th generation @ 2.2Ghz(2) PC and it should handle the serial data superfast,isn't?
I will check the link you provided.
THis is the code for packet wrapper, may be you can help if this is stoping displaying on graph quick.

/**
 * All data is passed to HDLC receiver.
 * Here we check the packet consistency, and pass on the data only.
 * @brief HDLC_qt::charReceiver
 * @param dataArray Received ch         unck of data of some length.
 * @return emit hdlcValidFrameReceived(data, length) When packet arrived.
 */
void HDLC_qt::charReceiver(QByteArray dataArray)
{
  //  static qint64 hdlcValidFrameReceivedcall = 0;
  //  QMutexLocker locker(&(SerialCommunication::m_Instance->mutexSerial));
 //  QMutexLocker locker(&mutexcharreceive);
 //   qDebug() << "hdlccharReceiverentry  is ";
//    for(int i =10;i<20;i++)
//    {
//         qDebug() << i ;
//    }
   // qDebug() << this->thread();
    char data = 0;
    for(QByteArray::iterator it = dataArray.begin(); it != dataArray.end(); it++) {
        data = (*it);
        /* Start flag or end flag */
        if(data == FRAME_BOUNDARY_OCTET)
        {
            if(escape_character == true)
            {
                escape_character = false;
            }
            /* Do CRC check if frame is valid */
            else if(  (frame_position >= 2)
                      &&(this->hdlcFrameCRC_check(frame_position)) )
            {
                /* Call user defined function to handle HDLC frame */
//                 qDebug() << "hdlcValidFrameReceivedcall  is ";
//                qDebug() << hdlcValidFrameReceivedcall++;
                receive_frame_buffer.chop(2);//Remove CRC data which is 2 bytes
                emit hdlcValidFrameReceived(receive_frame_buffer, static_cast<quint16>(frame_position-2));
            }
            /* Reset all for next frame */
            frame_position = 0;
            frame_checksum = 0;
            receive_frame_buffer.clear();
            continue;
        }

        if(escape_character)
        {
            escape_character = false;
            data ^= INVERT_OCTET;
        }
        else if(data == CONTROL_ESCAPE_OCTET)
        {
            escape_character = true;
            continue;
        }
        receive_frame_buffer[frame_position] = data;

        /* In Qt we don't calculate CRC here iteratively.
         * In Arduino, we needed to save RAM, but here small
         * buffer is no problem, we have more RAM.
         */

        frame_position++;

        /* If we don't ever receive valid frame,
         * buffer will keep growing bigger and bigger.
         * Hard coded max size limit and then reset
         */
        if(frame_position >= 2048)
        {
            receive_frame_buffer.clear();
            frame_position = 0;
            frame_checksum = 0;
        }
    }
  //  qDebug() << "hdlccharReceiverexit  is ";
   // qDebug() << this->thread();
}

ANd the code for displaying on graph is below:

void AcquisitionWindow::plotDataToGraph_Slot(QByteArray buffer, quint16 bytes_received)
{
   // QElapsedTimer timer;
   // timer.restart();
    qv_x.clear();
    qv_y.clear();
//    static int plotgraph = 128;
    static quint16 xaxisdata  = 0;
 //   int size = buffer.size();//test
    quint16 bufferdata = buffer[1]&0xff;
//    quint16 bufferdata = (buffer[2]&0xFF);//|((buffer[2]<<8)&0xFF);
//    bufferdata<<=8;
//    bufferdata |= ((buffer[1])&0xFF);

//    qv_x.append(xaxisdata++);
//    qv_y.append(bufferdata);
//    if(xaxisdata == plotgraph)
//    {
//       ui->widget_customPlot->graph(0)->addData(qv_x,qv_y);
//       ui->widget_customPlot->replot();
//       plotgraph+=128;
//    }
    ui->widget_customPlot->graph(0)->addData(xaxisdata++,bufferdata);
    if(xaxisdata == 2048)//((int)buffer[1] == 0)
    {
//        plotgraph = 128;
//       ui->widget_sipm->graph(0)
        xaxisdata = 0;
        qv_x.clear();
        qv_y.clear();
         ui->widget_customPlot->graph(0)->setData(qv_x,qv_y);
        //ui->widget_sipm->graph(0)->
     //   ui->widget_customPlot->replot();
    }
    ui->widget_customPlot->replot();
    ui->plainTextEdit->appendPlainText(static_cast<QString>(buffer.at(1)));

//    qDebug()<<"Timeelapsed";
    //    qDebug()<<timer.nsecsElapsed();
}

srikanth2006

@MikhailG I could do average chunks but it needs to be fast enough to display as it receives from serial port. AT the moment I have generated a picoscope wave generator connected to ADC on STM and sending the data to QT using serialport on 115200 baud. I am expecting 115200/10/7 = 1645Bytes/sec and by nyquist rate i magine i should a smooth sine wave of 1645/5 = 329HZ from pico .But i only get 10Hz max. So chunks of data would not help i think. Any ideas is helpfull. Thanks

srikanth2006

@mrjj
I have removed Qcustomplot now and just displaying serial to Qplaintext and it still not catching up.
So all it does now is read from serial, and using wrapper to extract data and display data on plaintext window. All three in three different threads. Any advice on what could be going wrong? Thanks

srikanth2006

@MikhailG I have removed HDLC wrapper and only reading serial and displaying on texttwindow. Its fast and it just prints whole chunk of data on ttext window from serial.
So question is how do i extract each packet from the received serial data and display the data from each packet on textwindow without losing data being received from serial and be fast...

MikhailG

@srikanth2006 You could profile your project and look for bottlenecks. Maybe some unnecessary copying of data while transfering.

srikanth2006

@MikhailG Sorry what is a profile project mean?
Is there is any example on how to profile the project to find bottlenck issue?

mrjj

@srikanth2006
Hi
To profile the project means to use a tool that can show info about the runtime performance of
the app.
Like
http://www.codersnotes.com/sleepy/
There exits quite a few and often the paid for are far more advanced but depending on what platform you need it for, often
a free version exits.

https://stackoverflow.com/questions/67554/whats-the-best-free-c-profiler-for-windows

Often it takes a bit of time to get to know the tool. (as with any tool) but the payoff is huge sometimes.

srikanth2006

Hi,
I dont seem to workout anything using profiling the project. May be need more learning on profile tools i guess.
HOweever I am consider to not to use HDLC wrappe at all and just see for end of string character and parse the data if that is any quicker.
Iam attaching my complete hdlc code, if you think if that is causing any issue.```
#include "hdlc_qt.h"

HDLC_qt* HDLC_qt::m_Instance = nullptr;

/**

All data is passed to HDLC receiver.
Here we check the packet consistency, and pass on the data only.
@brief HDLC_qt::charReceiver
@param dataArray Received ch unck of data of some length.
@return emit hdlcValidFrameReceived(data, length) When packet arrived.
*/
void HDLC_qt::charReceiver(QByteArray dataArray)
{
// static qint64 hdlcValidFrameReceivedcall = 0;
// QMutexLocker locker(&(SerialCommunication::m_Instance->mutexSerial));
// QMutexLocker locker(&mutexcharreceive);
// qDebug() << "hdlccharReceiverentry is ";
// for(int i =10;i<20;i++)
// {
// qDebug() << i ;
// }
// qDebug() << this->thread();
char data = 0;
for(QByteArray::iterator it = dataArray.begin(); it != dataArray.end(); it++) {
data = (it);
/ Start flag or end flag /
if(data == FRAME_BOUNDARY_OCTET)
{
if(escape_character == true)
{
escape_character = false;
}
/ Do CRC check if frame is valid /
else if( (frame_position >= 2)
&&(this->hdlcFrameCRC_check(frame_position)) )
{
/ Call user defined function to handle HDLC frame /
// qDebug() << "hdlcValidFrameReceivedcall is ";
// qDebug() << hdlcValidFrameReceivedcall++;
receive_frame_buffer.chop(2);//Remove CRC data which is 2 bytes
emit hdlcValidFrameReceived(receive_frame_buffer, static_cast<quint16>(frame_position-2));
}
/ Reset all for next frame */
frame_position = 0;
frame_checksum = 0;
receive_frame_buffer.clear();
continue;
}
```
 if(escape_character)
 {
     escape_character = false;
     data ^= INVERT_OCTET;
 }
 else if(data == CONTROL_ESCAPE_OCTET)
 {
     escape_character = true;
     continue;
 }
 receive_frame_buffer[frame_position] = data;

 /* In Qt we don't calculate CRC here iteratively.
  * In Arduino, we needed to save RAM, but here small
  * buffer is no problem, we have more RAM.
  */

 frame_position++;

 /* If we don't ever receive valid frame,
  * buffer will keep growing bigger and bigger.
  * Hard coded max size limit and then reset
  */
 if(frame_position >= 2048)
 {
     receive_frame_buffer.clear();
     frame_position = 0;
     frame_checksum = 0;
 }
```
}
// qDebug() << "hdlccharReceiverexit is ";
// qDebug() << this->thread();
}

/**

Wrap the data in HDLC frame
@brief HDLC_qt::frameDecode
@param buffer
@param bytes_to_send
/
void HDLC_qt::frameDecode(QByteArray buffer, quint16 bytes_to_send)
{
// QMutexLocker locker(&(SerialCommunication::m_Instance->mutexSerial));
// qDebug() << "hdlcframeDecode is ";
// qDebug() << this->thread();
char data;
//quint8 data;
QByteArray packet;
/ The frame check sequence (FCS) is a 16-bit CRC-CCITT /
quint16 fcs = 0;
// Update checksum
fcs = qChecksum(static_cast<const char>(buffer.constData()), bytes_to_send);
/* Start flag */
packet.append((char)FRAME_BOUNDARY_OCTET);
int i = 0;
while (i < bytes_to_send)
{
data = buffer[i];
if( (data == static_cast<char>(CONTROL_ESCAPE_OCTET)) || (data == static_cast<char>(FRAME_BOUNDARY_OCTET)) )
{
packet.append(static_cast<char>(CONTROL_ESCAPE_OCTET));
data ^= static_cast<char>(INVERT_OCTET);
}
packet.append(static_cast<char>(data));
i++;
}

/* Invert bits in checksum
- For avrlibc crc_ccitt_update() compatibility
  */
  fcs ^= 0xFFFF;
/* Low byte of inverted FCS */
data = low(fcs);
if((data == static_cast<char>(CONTROL_ESCAPE_OCTET)) || (data == FRAME_BOUNDARY_OCTET))
{
packet.append(static_cast<char>(CONTROL_ESCAPE_OCTET));
data ^= static_cast<char>(INVERT_OCTET);
}
packet.append((char)data);

/* High byte of inverted FCS */
data = high(fcs);
if((data == static_cast<char>(CONTROL_ESCAPE_OCTET)) || (data == FRAME_BOUNDARY_OCTET))
{
packet.append(static_cast<char>(CONTROL_ESCAPE_OCTET));
data ^= static_cast<char>(INVERT_OCTET);
}
packet.append(static_cast<char>(data));

/* End flag */
packet.append(static_cast<char>(FRAME_BOUNDARY_OCTET));
emit hdlcTransmitByte(packet);
}

bool HDLC_qt::hdlcFrameCRC_check(int frame_index)
{
/* frame = ...[CRC-LO] [CRC-HI] /
quint16 crc_received = 0;
crc_received = static_cast<quint16>(receive_frame_buffer[frame_index-1]); // msb
crc_received = crc_received << 8;
crc_received |= (static_cast<quint16>(receive_frame_buffer[frame_index-2]))&0xFF; // lsb
quint16 crc_calculated = qChecksum(static_cast<const char>(receive_frame_buffer.constData()),static_cast<quint16>(frame_index-2));
crc_calculated = crc_calculated^0xFFFF;
if(crc_received == crc_calculated) {
return true;
} else {
return false;
}
}

srikanth2006

Hi,
I changed the plotting rate of qcustomplot to 100ms and that seemed to make things better.
Also changed blockedqueued connection to just queued connection between wrapper and GUI and it worked fine.
However the max sampling frequency i can get is 100Hz at 115200 serial baudrate. Any ideas on how to improve this would be helpful. For now it is fine. THanks for all your help.