QSerialPort on Linux with high baudrate

Texture

Hello all! I have faced with the next issue. I have external device connecting as serial port via usb. I need to send and receive big package of data (about 8 kB at once). I could change its baudrate. Everything is okay up to 250000 baud. When I choose 500000 my Linux machine can not receive the whole package (only about 7.9 kB). I have checked the same program on Windows machine and it is okay.

I am connecting to readyRead() signal and read as follows:

QByteArray Serial::read() {
	QByteArray answer;
	if (_serialPort->bytesAvailable() == _bytesToRead) {
		answer = _serialPort->readAll();
		emit messageReceived(answer);
	}
	return answer;
}

Serial port is opened with following code:

if (_serialPort->open(QIODevice::ReadWrite)) {
	_serialPort->clear();
	_serialPort->setBaudRate(500000);
	_seialPort->setParity(QSerialPort::NoParity);
	_serialPort->setDataBits(QSerialPort::Data8);
	_serialPort->setStopBits(QSerialPort::OneStop);
	_serialPort->setFlowControl(QSerialPort::NoFlowControl);
	_serialPort->setReadBufferSize(0);
return true;
}

My setup is Xubuntu 18.04, Qt 5.13.0 (gcc_64) and GCC 7 compiler. I would be appreciate for any help.

aha_1980

Hi @Texture,

When I choose 500000 my Linux machine can not receive the whole package (only about 7.9 kB)

You can never(!) expect the whole data arrive at once. The readyRead() slot may be called several times and return your data in chunks. It's your task to reassemble these chunks to a complete block (that requires a protocol on top that allows this).

So please check first if readyRead() is called several times.

Regards

Texture

@aha_1980 Thank you for answer, I understand that whole data can be received partially. So in my code I am waiting till inner buffer has necessary size:

if (_serialPort->bytesAvailable() == _bytesToRead) {
   answer = _serialPort->readAll();
   ...
}

Signal readyRead() is called several times, collecting new 64 bytes at once. I expect to receive 7392 bytes, but only 7352 bytes comes.

aha_1980

@kuzulis, do you have an idea here?

May there be some internal buffer overrunning (as we are soon at the 8kB border)?

Thanks

Christian Ehrlicher

@aha_1980 said in QSerialPort on Linux with high baudrate:

May there be some internal buffer overrunning

To be sure I would read out the data into an own buffer

aha_1980

Hi @Texture,

I thought about this problem again. Can you create a dump of the received data on both systems and compare it?

So at least we know if the missing chars are at the start, middle or end.

Regards

Texture

@Christian-Ehrlicher I have tried this approach and have the same issue. Every time when readyRead() is catched, I save information to my external buffer and continue. Nothing changes.

Texture

@aha_1980 I have tried to look through data as you proposed and what I could say: the whole package comes (I have special CRC in the end and test increasing uint16_t numbers as data so I could identify the real end), but bytes partially lost somewhere in the middle. I have tested several times and first bytes disappearance occurs at byte #64.

aha_1980

Hi @Texture,

very strange behavior!

Well, next thing would be to find out the source of the problem. It may be anywhere from

Hardware
Linux Kernel (driver)
Qt Serial Port
Your program

What comes to my mind is, that most PC's traditionally use "odd" baud rates: 1200, 9600, 57600, 115200 and so on. Even modern USB-serial adapters often use these baudrates or multiples of that.

So may it be your device cannot be set to exactly 500000 baud? How is the USB implemented on the device side? With an external chip like FTDI FT232R or similar? Or as software stack within the controller?

To exclude possible bugs within QtSerialPort, you'd have to write a receive example in pure POSIX code yourself. I have not done that for ages, so no example at hand. But you can find many of them in the internet.

Regards

JonB

@Texture

You don't slot onto the QSerialPort::errorOccurred signal, you should check in case the missing bytes are being reported as such!
It's only a few(-ish) lines to test with C/OS level [_]open() & [_]read() lines to count up how many bytes you receive with the least possible code overhead. If that does not keep up with the baud rate, Qt won't!
You could also run your program under Linux strace. From that you could discover what bytes the port is actually sending/receiving, in case that differs from what your reader code sees.
I don't know about external devices, but is it possible that it's the serial device which cannot keep up with the baud rate? Or the Linux driver, or could it need hardware flow control?

Texture

@aha_1980 Thank you for your suggestions!
The same device perfectly works with the same code running on windows machine. I had some experiments today and force my device send fixed array. In this case my Linux program could catch every byte. Looks like external device has not so good timings, but why Windows case works without any issue?

External device is STM32 connected with PC via uart with FTDI converter (as well as I've tested with CH340).

I will try to check with pure POSIX code.

Texture

@JonB Thank you for the answer.

I have connected with errorOccurred() slot, but it has no messages/signals while receiving the bytes.

External device is STM32 connected with PC via uart with FTDI converter (as well as I've tested with CH340) and it perfectly works with this baudrate with the same code on Windows machine. I will give a try to check bytes with strace.

kuzulis

So,

do you use FTDI && CH340 as a serial converters in both OS's ?
had you checked it in a simple console application?

I'm ask, because earlier we too faced with a troubles with the serial port communications on Linux. But in our case we used iMX53 boad with in-bound serial port (not USB). A problem was in a driver of that port in Linux: there are was allocated a small FIFO size, which are overrunned on a high speed (we got an overrun errors). But, switching. e.g. to the USB serial converters solved a problems (in our case)...

So, it is strange.

I have tested several times and first bytes disappearance occurs at byte #64.

It looks like an overrun errors. You can try to look on an overrun errors, like this: https://askubuntu.com/questions/459392/read-serial-error-counters-from-shell (but, I'm not sure).

Maybe, you can try to put your worker class to the other thread and give a high priority to a thread.
Maybe, you can try to update a linux kernel... or try other Linux distro...

Texture

@kuzulis

Thank you for your reply. I have spent a time to get some experiment results before answering. Some answers and notices are below:

As external device for testing I have STM32F4 Discovery and its UART connected with PC via CH340 serial converter.
First simple case (500k baudrate) is perfectly working on Linux with CuteCom terminal emulator (unfortunately, it doesn't support 1kk baudrate).
I have created pure simple console application with two approaches (singleton and dynamic QSerialPort handler classes), everything is working well with 500k and 1kk baudrate.
I have tried to put my serial port handler class to other thread in my program, but nothing changed.
Very weird and not obvious effect for me: if I programmatically generate answer array in STM32 (e.g. fill in for-loop with for indexes), my program is working without any problem. If I use ADC in STM32 to fill my array (as it supposed for my embedded device) I get issues described above.
UPD: I have tested the following case: my array is generated by STM32 with using not only indexes, but function (sin(), cos() or rand()). In this case even CuteCom can not receive the whole package. As far I understand it is also based on Qt5 and QSerialPort.

I am so confused with so many factors that affect on serial port reading and don't know, what should I try to fix at first...