QSerialPort::readyRead triggered twice on raspberry pi 4B if data larger than 16bytes

[Hu Zhang]

Recently, I use QModbus on the Raspberry Pi, and a problem often occurs，if a long modbus rtu request sent to raspberry pi, error message show, and no response for this request.

qt.modbus: (RTU server) ADU does not match expected size, ignoring

After looking at the source code, I found if a serial port data longer than 16bytes, QSerialPort::readyRead will trigger twice to receive this data, that resulting in a interval which is longer than 3.5 char，so QModbusRtuSerialSlave consider it as two requests

I've test Qt 5.12.8 on raspberry pi 4b, and 5.12.4 on Desktop PC, Desktop is fine.

    static int cnt = 0;
    QObject::connect(&serialPort, &QSerialPort::readyRead, [this]() {

        const int size = serialPort.size();
        qDebug() << cnt++ << ":" << QTime::currentTime() << serialPort.read(size);
    });

    //serialPort.setPortName("/dev/serial0");  // raspberry pi
    serialPort.setPortName("COM3"); //Desktop PC
    serialPort.setParity(QSerialPort::NoParity);
    serialPort.setBaudRate(QSerialPort::Baud19200);
    serialPort.setDataBits(QSerialPort::Data8);
    serialPort.setStopBits(QSerialPort::OneStop);

    serialPort.open(QIODevice::ReadWrite);

send data from a desktop pc

02 10 00 80 00 06 0C 37 BD 35 86 CC CD 3D 4C 00 01 FF FF F8 C5 

on raspberry pi 4b, output is

0 : QTime("01:24:54.913") "\x02\x10\x00\x80\x00\x06\f7\xBD""5\x86\xCC\xCD=L\x00"
1 : QTime("01:24:54.917") "\x01\xFF\xFF\xF8\xC5"

on Desktop PC

0 : QTime("08:39:10.893") "\x02\x10\x00\x80\x00\x06\f7\xBD""5\x86\xCC\xCD=L\x00\x01\xFF\xFF\xF8\xC5"

[KroMignon]

@Hu-Zhang Serial port is a stream, like TCP socket. When you read data from it, you get what currently arrived, which does not mean it is a complete message. It is up to you to detect frame/message begin and end.
Depending on PC CPU charge, I am quite sure will get same behavior on your PC.

So this is not a bug in QSerialPort, this normal behavior.

[J.Hilk]

@KroMignon Usually, this holds true!

But @Hu-Zhang is saying he uses QModbus, which actually should take care of the data accumulation and the (Qt)user layer is a step higher.

That said, I'm not sure why the op is operating on Serialport level, maybe to investigate 🤷‍♂️

@Hu-Zhang I would consider upgrading! QSerialPort and especially QModbus were super buggy in 5.12! So buggy in fact that I dropped QModbus from our project and wrote my own version, because company wise we're stuck on 5.12

[KroMignon]

@J-Hilk said in QSerialPort::readyRead triggered twice on raspberry pi 4B if data larger than 16bytes:

But @Hu-Zhang is saying he uses QModbus, which actually should take care of the data accumulation and the (Qt)user layer is a step higher.

On his example, I can only see QSerialPort output, nothing about QModbus usage.

[J.Hilk]

@KroMignon A agree, in his example, no QModbus! the actual question:

Recently, I use QModbus on the Raspberry Pi, and a problem often occurs，if a long modbus rtu request sent to raspberry pi, error message show, and no response for this request.

qt.modbus: (RTU server) ADU does not match expected size, ignoring

[Hu Zhang]

Thank you for reply

I wrote the simple QSerialPort code for test, the behavior is same as QModbusRtuSerialSlave.

and here is the full qmodbus output

qt.modbus.lowlevel: (RTU server) Received ADU: "0210008000060c37bd3586cccd3d4c00"
qt.modbus: (RTU server) ADU does not match expected size, ignoring
qt.modbus.lowlevel: (RTU server) Dropping older ADU fragments due to larger than 3.5 char delay (expected: 3 , max: 4)
qt.modbus.lowlevel: (RTU server) Received ADU: "01fffff8c5"
qt.modbus: (RTU server) Discarding request with wrong CRC, received: 63685 , calculated CRC: 8624

below is qmodbus code, Under my test conditions, baudrate is 19200, The calculation result of m_interFrameDelayMilliseconds is 3 ms, and m_interFrameTimer.elapsed() is bigger than 3 ms (Normally 4, occasionally bigger), I guess it's because serial data is received in two parts, so the first part is dropped, and the second part CRC error.

File: qmodbusrtuserialslave_p.h

class QModbusRtuSerialSlavePrivate : public QModbusServerPrivate
{
    Q_DECLARE_PUBLIC(QModbusRtuSerialSlave)

public:
    void setupSerialPort()
    {
        Q_Q(QModbusRtuSerialSlave);

        m_serialPort = new QSerialPort(q);
        QObject::connect(m_serialPort, &QSerialPort::readyRead, [this]() {

            if (m_interFrameTimer.isValid()
                    && m_interFrameTimer.elapsed() > m_interFrameDelayMilliseconds   <<=============
                    && !m_requestBuffer.isEmpty()) {
                // This permits response buffer clearing if it contains garbage
                // but still permits cases where very slow baud rates can cause
                // chunked and delayed packets
                qCDebug(QT_MODBUS_LOW) << "(RTU server) Dropping older ADU fragments due to larger than 3.5 char delay (expected:"
                                       << m_interFrameDelayMilliseconds << ", max:"
                                       << m_interFrameTimer.elapsed() << ")";
                m_requestBuffer.clear();
            }

            m_interFrameTimer.start();

            const int size = m_serialPort->size();
            m_requestBuffer += m_serialPort->read(size);

            const QModbusSerialAdu adu(QModbusSerialAdu::Rtu, m_requestBuffer);
            qCDebug(QT_MODBUS_LOW) << "(RTU server) Received ADU:" << adu.rawData().toHex();

            // Index                         -> description
            // Server address                -> 1 byte
            // FunctionCode                  -> 1 byte
            // FunctionCode specific content -> 0-252 bytes
            // CRC                           -> 2 bytes
            Q_Q(QModbusRtuSerialSlave);
            QModbusCommEvent event = QModbusCommEvent::ReceiveEvent;
            if (q->value(QModbusServer::ListenOnlyMode).toBool())
                event |= QModbusCommEvent::ReceiveFlag::CurrentlyInListenOnlyMode;

            // We expect at least the server address, function code and CRC.
            if (adu.rawSize() < 4) { // TODO: LRC should be 3 bytes.
                qCWarning(QT_MODBUS) << "(RTU server) Incomplete ADU received, ignoring";

                // The quantity of CRC errors encountered by the remote device since its last
                // restart, clear counters operation, or power-up. In case of a message
                // length < 4 bytes, the receiving device is not able to calculate the CRC.
                incrementCounter(QModbusServerPrivate::Counter::BusCommunicationError);
                storeModbusCommEvent(event | QModbusCommEvent::ReceiveFlag::CommunicationError);
                return;
            }

            // Server address is set to 0, this is a broadcast.
            m_processesBroadcast = (adu.serverAddress() == 0);
            if (q->processesBroadcast())
                event |= QModbusCommEvent::ReceiveFlag::BroadcastReceived;

            const int pduSizeWithoutFcode = QModbusRequest::calculateDataSize(adu.pdu());

            // server address byte + function code byte + PDU size + 2 bytes CRC
            if ((pduSizeWithoutFcode < 0) || ((2 + pduSizeWithoutFcode + 2) != adu.rawSize())) {
                qCWarning(QT_MODBUS) << "(RTU server) ADU does not match expected size, ignoring";   <<=============
                // The quantity of messages addressed to the remote device that it could not
                // handle due to a character overrun condition, since its last restart, clear
                // counters operation, or power-up. A character overrun is caused by data
                // characters arriving at the port faster than they can be stored, or by the loss
                // of a character due to a hardware malfunction.
                incrementCounter(QModbusServerPrivate::Counter::BusCharacterOverrun);
                storeModbusCommEvent(event | QModbusCommEvent::ReceiveFlag::CharacterOverrun);
                return;
            }

            // We received the full message, including checksum. We do not expect more bytes to
            // arrive, so clear the buffer. All new bytes are considered part of the next message.
            m_requestBuffer.resize(0);

            if (!adu.matchingChecksum()) {
                qCWarning(QT_MODBUS) << "(RTU server) Discarding request with wrong CRC, received:"
                                     << adu.checksum<quint16>() << ", calculated CRC:"
                                     << QModbusSerialAdu::calculateCRC(adu.data(), adu.size());   <<=============
                // The quantity of CRC errors encountered by the remote device since its last
                // restart, clear counters operation, or power-up.
                incrementCounter(QModbusServerPrivate::Counter::BusCommunicationError);
                storeModbusCommEvent(event | QModbusCommEvent::ReceiveFlag::CommunicationError);
                return;
            }

[KroMignon]

@Hu-Zhang According to qmodbus output, there are more than 3.5 char delay:

• 19200 bits/sec
• 10 bits per byte (1 start, 8 data and 1 stop)

==> delay is about 1.8 ms

And your previous test with QSerialPort you have about 4ms (01:24:54.917 - 01:24:54.913) between each reception event.
I don't know if you are using RPi hardware serial port or an USB serial dongle, but you have to "tune" your serial driver to improve reception performances.
Perhaps changing reception buffer size?

[Hu Zhang]

@KroMignon

I'm using hardware serial port, now I change m_interFrameDelayMilliseconds to 5ms for temporary use, working much better.

not sure if it’s the problem of Raspberry Pi or Qt, need to go deeper.

[KroMignon]

@Hu-Zhang said in QSerialPort::readyRead triggered twice on raspberry pi 4B if data larger than 16bytes:

not sure if it’s the problem of Raspberry Pi or Qt, need to go deep

I think it is a driver issue, because your message is splitted in 16 byte blocks.
This looks to me as the serial port driver reception buffer size.

I don't have a RPi to check this, but 16 bytes is too "perfect" value ;)

[kuzulis]

Yes, also, you can't rely on m_interFrameDelayMilliseconds interval for the non-real-time OS. So, even 3, 3.5, 4, or 5 msecs will be interpreted as same value. I'm not sure, that QTimer e.g. with the 3 ms interval will trigger exacly every 3 ms for all use-cases (for all CPU loads and so on)...

To be honest, I'm don't know why the QtModbus module calculates and relies on this m_interFrameDelayMilliseconds parameter. As I remember, I suggested to introduce an additional "timeout" parameter for a whole request/response frame, but I'm don't know is it implemented now or not.

As a workaround, you can try to play with the QSerialPort handle, and to set the VTIME && VMIN values into the termios structure (currently there are zeros).