QSerialPort Async read - How to ensure full packet read

Michal Poplawski

Hi!

This is my first post on this forum, so welcome everybody!

I'm developing an app to read some data from my car and output it to MPG display. The protocol is very simple, 9 bits (temperature, instant MPG, average MPG, Miles remaining and so on) over serial port at 2400 kbps, data packet is transmitted in approx 55ms with 46ms packet spacing. I've got hardware layer mostly worked up, as well as software layer (Qt desktop app in non-blocking approach).

My problem is that if I start listening for packets (with readyRead() signal) when the signal is already being transmitted, readyRead() is emitted, which results in corrupted data read (i.e. temperature is read as Miles remaining, instant MPG as temperature from next data packet and so on), as I append read bytes until full packet length has been received.

receive_data_pack () {
	const QByteArray data = serial_in->readAll();
	read_buffer.append(data);
	if (read_buffer.length() > 3 ) {
		params_in.temp = uchar(read_buffer.at(0));
		params_in.inst_cons = uchar(read_buffer.at(1));
		params_in.avg_cons = uchar(read_buffer.at(2));
		params_in.dist_remain = uchar(read_buffer.at(3));
		qDebug() << read_buffer.toHex();
		read_buffer.clear();
	} 
}

So my idea is to wait with readyRead() signal handling start until no data is received, however different combinations of waitForReadyRead(some msecs) proved to be unsuccessful (if I start reading packets at correct time there are no errors during the transmission, so the key is to start at correct time. First data packet is not essential, so I can afford skipping partial read.

My initial attempt was to do something like this:

	connect(this->serial_in, SIGNAL(readyRead()), this, SLOT(receive_data_pack()));
	if (serial_in->waitForReadyRead()) 
		serial_in->clear(QSerialPort::Input);

However this doesn't seem to work properly.

Does anyone have some idea how to implement this correctly?

Best regards,
Michal

SGaist

Hi and welcome to devnet,

Do you have any protocol for the serial port data ?
What you need is a mean to identify a full data frame. For example a frame starts with aaa sequence and ends with a zzz (whatever fits your purpose). Then on the receiver end you cumulate your serial port data in a buffer and each time your receive something, you put it in the buffer, check the buffer content for a full frame and if it's there, extract it, parse it and do whatever you want and if not, continue as usual.

Michal Poplawski

Hi!

Nice to be here! :)

Unfortunately there is no STX/ETX or such sequences to identify full data frame. Only packet intervals and that's the problem. I can work this out by phisically starting to listen for packets before they start being sent, but this doesn't seem to me like an elegant solution.

SGaist

The device is locked ?

Michal Poplawski

What do you mean by "locked"?

I open it with this code:

if (serial_in->open(QSerialPort::ReadOnly)) {
			serial_in->setBaudRate(QSerialPort::Baud2400);
			serial_in->setDataBits(QSerialPort::Data8);
			serial_in->setParity(QSerialPort::NoParity);
			serial_in->setStopBits(QSerialPort::OneStop);
			serial_in->setFlowControl(QSerialPort::NoFlowControl);
}

SGaist

Sorry, I meant the data source. So in your case, you wrote it's your car. Are you using a custom device to read the information ? IIRC, cars rather use industrial serial buses like CAN rather than RS-232.

eyllanesc

@Michal-Poplawski I mean you set the parameters of the serial device to open it? Isn't it illogical? Serial communication has rules and among this is the baudrate, for example, which is indicated by the communication speed so that both speak the same "language" and after establishing those rules, they can no longer be changed. So it changes to:

serial_in->setBaudRate(QSerialPort::Baud2400);
serial_in->setDataBits(QSerialPort::Data8);
serial_in->setParity(QSerialPort::NoParity);
serial_in->setStopBits(QSerialPort::OneStop);
serial_in->setFlowControl(QSerialPort::NoFlowControl);
if (serial_in->open(QSerialPort::ReadOnly)) {
    qDebgu() << "open";
}

Michal Poplawski

@SGaist
Hardware layer is simply a transoptor (PS2501) plus pull-up resistors on input and output lines (both 1kOhm), because input signal is inversed. I have no problem reading data on serial port behind that transoptor.

In my car (Subaru Outback 2008) most of internal communication indeed is done via CAN bus, however that particular display (clock plus MPG) runs on serial UART and receives data from combination meter.

What I'm basically trying to achieve is to read data, convert it to metric and put it to metric display unit (originally I have imperial one and the display has fixed fields, so it cannot be reprogrammed conviniently). Eventually the conversion will be done with Arduino, but for now I have PC software based on QT classes, because it's simpler to visualize incomming and outgoing data.

I have a working solution now.

First of all:

read_buffer.clear();

That's a REALLY STUPID WAY OF CLEARING BUFFER AFTER READ. Because buffer is async in nature so I have no way of knowing how much data is in it. Which means that I'm leaking frames/parts of them that have been added to the buffer from port. What I should have done is to:

read_buffer.remove(some_start_point_mostly_0, read_data_size);

As for a method of ensuring that entire frame has been read... Each frame has a checksum added at the end (least significant byte of sum of all parameters). Bruthal method is to calculate that checksum on incominng stream (if length is proper) and if the checksum is OK, then read parameters. If checksum is not OK then I remove first byte of data stream and start over.

I understand that it's not a perfect way od doing things (i.e. I can read garbage if checksum over stream is the same as one of the data fields), but it works for now. However if there is a more elegant way of solving this I'd be happy to hear about it.

Thanks for your interest!

Michal Poplawski

@eyllanesc
Unfortunately you cannot set baud rate per your needs on non-opened port:

baudRate : qint32

This property holds the data baud rate for the desired direction

If the setting is successful or set before opening the port, returns true; otherwise returns false and sets an error code which can be obtained by accessing the value of the QSerialPort::error property. To set the baud rate, use the enumeration QSerialPort::BaudRate or any positive qint32 value.

Note: If the setting is set before opening the port, the actual serial port setting is done automatically in the QSerialPort::open() method right after that the opening of the port succeeds.
https://doc.qt.io/qt-5/qserialport.html

The same rule applies to the rest of communication parameters, so you must set them according to your needs after the port has been opened.

SGaist

@Michal-Poplawski said in QSerialPort Async read - How to ensure full packet read:

I understand that it's not a perfect way od doing things (i.e. I can read garbage if checksum over stream is the same as one of the data fields), but it works for now. However if there is a more elegant way of solving this I'd be happy to hear about it.

Sounds fine, you have a way to ensure that you got your data and that it is in a consistant form so it's all good.

Since you mentioned Arduino, you may also develop the code on the Arduino and forward the data to your application so you have both up and running and available for tests.

Michal Poplawski

@SGaist
That's precisely the idea! :)

QtWS: Early Bird Tickets Available!

QSerialPort Async read - How to ensure full packet read