Terminal Example - read problem

bzgec

@hskoglund Awsome, this works!

void MainWindow::readData()
{
    for (;;)
    {
        const QByteArray data = m_serial->read(1);
        if (data.isEmpty())
            return;

        m_console->putData(data);
    }
}

If i understand correctly this is still an asynchronous way which just checks if more data is available after reading one character instead of reading every characters it received in one go. And this helps because the QSerialPort::readyRead() doesn't fire fast enough (or corresponding slot doesn't execute fast enough) and by the time readAll() is called internal buffer is already overwritten with new data?

kuzulis

@bzgec ,

is already overwritten with new data?

No, that's not possible, the QSP never overwrites the previous data, it just appends.

Could you please do the following test (please add the qdebug output to see what happpens):

void MainWindow::readData()
{
    qDebug() << "Bytes available:" << m_serial->bytesAvailable();
    const QByteArray data = m_serial->readAll();
    qDebug() << "Data:" << data.toHex();
    ...
}

hskoglund

Hi, read the documentation for the readyRead() signal and it says: "....readyRead() is not emitted recursively..."
Doesn't that mean there's a theoretical "blind spot" in the readData() function:

oid MainWindow::readData()
{
    const QByteArray data = m_serial->readAll();
// begin "blind spot"
    m_console->putData(data);
// end of "blind spot"
}

I.e. if new characters arrive exactly at the same time as the ->putData(data) function runs, no readyRead() signal is fired (because we haven't got the hell out of Dodge/left readData()?
But the characters should not be lost anyway, just queued somewhere in a buffer, as @kuzulis says above *... never overwrites... , just appends." Maybe the Terminal example works better with another OS than your Linux, if you have a Windows PC around, you could try with that.

JonB

@hskoglund
Do you know what you have said about "blind spot" to be true/the case? It would be almost a show-stopper if it did! I don't think there will be any "blind spot", so far as Qt is concerned I don't think it will report any data as "arriving" while you are inside readData(), any new data which arrives after readAll() should cause a new readyRead signal to be raised after you exit this slot? Note that readyRead is not emitted a second time until after readAll() has been performed, so user code must read all already-arrived data before he will get a new notification for newly-arriving data.

kuzulis

An idea is that the QSP does work in a buffered mode (let's say how it does work, to be clear):

When the FIFO receives at leas one byte, the serial port file descriptor gets trigegred with the 'RX' event.
This event calls the 'data read' callback which reads all available data from the FIFO to the internal QSP buffer.
If a bytes read more than zero, then emits the readyRead() signal.
The QSP::read() or QSP::readAll() methods read the data from the internal buffer of QSP (but not from the device's FIFO).
The QSP::bytesAvailable() method returns a size of the internal QSP buffer (not the data count from the device's FIFO).
If a new data gets arrived into the device's FIFO after the QSP already did read from the FIFO, then the 'RX' event will be triggered in next time when the application runs the next Qt-event loop. The 'RX' event will be triggered in each Qt-event loop until the device's FIFO contains at least one byte.

So, please provide the debug output for the 'bytesAvailable()' and the 'readAll()' after each readyRead() signal to see what bytes are received.

bzgec

@kuzulis I tried your suggestion:
terminal_debug

hskoglund

Well well well @kuzulis your idea with the hex dump really did the trick!
It seems that the problem isn't at all related to any serial communication but rather what happens when you stuff a QString with null bytes :-)

I.e. try tossing out the zero bytes at reception, something like thisL

void MainWindow::readData()
{
    QByteArray dataWithZeroes = m_serial->readAll();
    QByteArray data;
    for (auto b : dataWithZeroes)
         if (b)
            data += b;
             
    m_console->putData(data);
}

JonB

@hskoglund

 QByteArray data =  m_serial->readAll().replace('\0', "");

? Totally untested, but a lot shorter ;-)

bzgec

@kuzulis your idea to display hex data was quite good :)
Examples from @hskoglund and @JonB both work.
So the problem is that when "0\n" or "1\n" string is send and the new string is sent right behind them. Then readAll() reads whole buffer including string termination sign and other data behind it, so conversion to string (or when displaying to console) drops other characters behind terminating sign.

But i what about readLine(). If i try @hskoglund and @JonB example it works, but with original terminal example it completely drops everything right behind "0\n" or "1\n".
terminal_debug_readLine

JonB

@bzgec said in Terminal Example - read problem:

so conversion to string (or when displaying to console) drops other characters behind terminating sign.

Then don't do so! Converting bytes containing \0s to strings is "problematic", as is displaying them. Do all your work on QByteArrays, and use the returned count not nul-string termination.

I haven't looked, but readLine() is liable to want to find \ns, and not treat \0s specially, or well in a string. If you have to do that, split your input on \0s as necessary. or remove them as we did.

bzgec

@JonB Yes i tried readLine() (and posted picture in previous answer) and the problem is the same as with readAll().

Ok so the problem was that the device (Nucleo board) was sending null-string termination in a message. And conversion to string (or when displaying to console) dropped everything behind it and that is why this @hskoglund's example worked (because it displayed one character at the time):

void MainWindow::readData()
{
    for (;;)
    {
        const QByteArray data = m_serial->read(1);
        if (data.isEmpty())
            return;

        m_console->putData(data);
    }
}

So my guess is that Putty uses one of the examples above. Which one would you recommend to use so that mistake like this doesn't show in console?

Thank you all for helping!

kuzulis

The QSP terminal example assumes that a received data contains in the ASCII table, that it is a visible symbols. The terminal example it is just a "simple example", and it does not handles the non-visual symbols yet. :)

bzgec

Ok so in my console application i am going to use code where you can select to filter \0 and \r. This option is checked in settings dialog (next to Local echo checkbox). Do you guys think that this code is OK or should i use some other option for filtering \0 and \r?

void mainWindow::readData() {
    QByteArray data;

    data = serialPort->readAll();
    if(settings->settings().filter_r == true) {
        data.replace('\r', "");
    }
    if(settings->settings().filter_0 == true) {
        data.replace('\0', "");
    }
    ui->console->putData(data);
}