When does QSerialPort send data from the buffer to the serial interface?

? Offline

Hi to all,

I'm using the QSerialPort class for sending data via a serial interface. For this, I'm reading an ASCI text file line by line and write it also line by line to the serial interface. My code is:

void MainWindow::on_pushButton_clicked(){
    serial = new QSerialPort(this);
    openSerialPort();
    QFile file("/user_data/big2.txt");
    if (!file.open(QIODevice::ReadOnly)) return;
    while (!file.atEnd()){
        QByteArray testdata = file.readLine();
        writeData(testdata);
    }
    closeSerialPort();
}

void MainWindow::openSerialPort()
{
    serial->setPortName("/dev/ttyGS0");
    serial->setBaudRate(9600);
    serial->setDataBits(QSerialPort::Data8);
    serial->setParity(QSerialPort::NoParity);
    serial->setStopBits(QSerialPort::OneStop);
    serial->setFlowControl(QSerialPort::NoFlowControl);
    serial->open(QIODevice::ReadWrite));
}

void MainWindow::closeSerialPort()
{
    serial->close();
}

void MainWindow::writeData(const QByteArray &data)
{
    serial->write(data);
    //serial->flush();
    serial->waitForBytesWritten(1000);
}

So now to my questions:

Why doesn't QSerialPort send data without using waitForBytesWritten()? According to http://doc.qt.io/qt-5/qserialport.html#flush transmission should in most cases start automatically. In my case data is obviously written to the buffer, but not actually sent.
Why do I get data loss up to 90% when using 'serial->flush();' to flush the buffer "manually"? Is the buffer eventually written and emptied at the same time?
Is there somewhere documentation available on how/when QSerialPort writes data to the buffer and then sends it via the interface?

In case it is relevant, I'm using Linux on an embedded system and the g_serial gadget driver for sending data to a desktop PC via USB.

Can anybody help me with my questions?

Kind regards,

Markus

PS: How can I turn code highlighting on? Putting the @ sign before and after the code didn't work and the automatic code highlighting vanished after adding the line
void MainWindow::on_pushButton_clicked(){
to the top...

mrdebug

I assure QSerialPort works perfectly.
I suggest to do not use signals but qthread to manage read and write.
Are you sure is not a problem related to the hardware interface? Is a full duplex interface or half duplex (485)?

SGaist

Hi,

Did you check that you currently have the correct permission to write to the device ?

For the code formatting the @ as been replaced by ``` (three back ticks)

Rondog

I have experieced part of what you described.

Using the older ExtSerialPort (external to Qt) the data sent out the serial port was not buffered (except on Windows for some reason ?).

Using QSerialPort built into Qt5 the data is buffered. You need to be careful about certain things if there is any protocol or handshaking you are trying to follow. I have QSerialPort in a separate thread and using waitForBytesWritten(-1) makes sure the data is sent and everything is in sync.

I have not used the 'flush' command. Likely it will simply block until the buffer is emptied anyway (I hope). I wouldn't expect you would loose data though. If you close the serial port before the buffer is emptied you will loose whatever has not been sent (calling 'flush' then 'close' might delete data if 'flush' does not wait until all the data is actually sent).

You can change the buffer size by the way. I am aware that you can do this but I didn't go this route when I ran into buffer problems. It might be the best option if you set the buffer size really small or zero in your case.

? Offline

Thanks for your answers!

@mrdebug Using it as in my code snippet, everything works perfect, indeed. Problems only arise when using the flush method or no waitForBytesWritten(). Also, http://doc.qt.io/qt-5/qserialport.html#flush states that in the absence of an event loop one should use waitForBytesWritten(). I'm not a Qt expert, but I think that there should be an event loop in my case (at least for reacting on GUI button press events). So I was asking myself if there would be more documentation on this and didn't find any so far.
I don't think that it is a problem with the hardware, since transmission obviously works. Yet it could be a problem with the g_serial driver I use. This is a gadget driver (Linux) using USB as a virtual serial interface.
Thanks for the hint with the qthread - will have a look at threads in qt also.

? Offline

@SGaist I think it is not a problem with permissions, since using the above code works. Also flushing using the flush method obviously works, but causing problems on the data.
Thanks for the hint with ``` - I've already updated the code in my post! Is there also some help page on features like this of the forum?

@Rondog Maybe writing to the buffer is blocked when it is being flushed and the write function can sometimes write only part of the data to it?
How can I change the write buffer size? On http://doc.qt.io/qt-5/qserialport.html I've only found a method for changing the read buffer size...

SGaist

Sorry, I misread your problem. One thing you can try before starting with thread is to use the bytesWritten signal to trigger the following write

Rondog

@MarkusBraitner You are right about having access only to the read buffer; I forgot about that one. Putting it in a thread with a blocking waitForBytesWritten() works well, just don't forget to call the event loop on occasion.

I noticed similarities when using other methods like QTcpSocket. With QTcpSocket the data is sent very fast so you generally don't notice any buffering but I have had problems with data being truncated intermittently which was a real head scratcher (it was due to closing the connection before the write buffer was emptied; it only happened once in a while which made it hard to track down).

? Offline

@SGaist No problem. Thanks for the hint with the bytesWritten signal!

@Rondog Thanks for sharing your experience on qTcpSocket with me! Yes, I've already read for QSerialPort that one has to be careful not to close the interface before the buffer has been flushed.

? Offline

In case somebody else finds it useful, these are the solutions that I've tried out now, and which work:

Using waitForBytesWritten() after each call to write() as in the code above (blocking)
Constructing a QByteArray of the whole file to send using the append() method and writing it to the buffer as a whole. The interface closing is triggered by the bytesWritten signal when there are no more bytes in the write buffer left. My code is as follows:

void MainWindow::on_pushButton_clicked()
{

    if(!(serial == 0)){
        qDebug() << "interface currently busy...";
    }
    else{
        serial = new QSerialPort(this);
        connect(serial, &QIODevice::bytesWritten, this, &MainWindow::closeSerialPort);
        openSerialPort();
        QFile file("/user_data/big2.txt");
        if (!file.open(QIODevice::ReadOnly)) return;
        writeData(testdata);
        QByteArray testdata;
        while (!file.atEnd()){
            QByteArray testdata_a = file.readLine();
            testdata.append(testdata_a);
        }
        serial->write(testdata_a);
      }
    }
}

void MainWindow::openSerialPort()
{
    serial->setPortName("/dev/ttyGS0");
    serial->setBaudRate(9600);
    serial->setDataBits(QSerialPort::Data8);
    serial->setParity(QSerialPort::NoParity);
    serial->setStopBits(QSerialPort::OneStop);
    serial->setFlowControl(QSerialPort::NoFlowControl);
    if(serial->open(QIODevice::ReadWrite)) {
    } else {
//        qDebug() << "Error opening serial port...";
    }
}

void MainWindow::closeSerialPort()
{
    if(!serial->bytesToWrite()){
        serial->close();
        // Set interface object pointer to zero
        serial = 0;
    }
}
}

Using the linux command "cat" to directly copy the file via shell command. This is blocking and i guess could also go to a thread.

QProcess process;
    QString command = "/bin/sh";
    QStringList args;
    args << "-c" << "cat /user_data/big2.txt > /dev/ttyGS0";
    process.start(command, args, QIODevice::ReadOnly);
    process.waitForFinished();
    QString StdOut = process.readAllStandardOutput();
    QString StdErr = process.readAllStandardError();
    qDebug() << StdOut;
    qDebug() << StdErr;

same as last item, but using the system() function (also blocking):

system("cat /user_data/big2.txt > /dev/ttyGS0");

Obviously, writing to the write buffer while it is being flushed, causes problems.

Also, I have found two docu pages, which seem helpful - don't know, why I didn't find them before...

http://doc.qt.io/qt-5/qtserialport-terminal-example.html (non-blocking approach)
http://doc.qt.io/qt-5/qtserialport-blockingslave-example.html (blocking approach)

Kind regards,

Markus

SGaist

Since you will dump the content of the file anyway, why not use: serial->write(file.readAll()); directly ?

Your QProcess solution is blocking because you call waitForFinished. If you want it to run without blocking just drop that call, allocate QProcess on the heap and handle the rest through signals and slots.

? Offline

@SGaist Good hint with the direct call :-) Sometimes one is just immersed into coding and overlooks the obvious... :-)
Yes, I was aware of the fact that waitForFinished makes it blocking. But thanks for your hint to allocate QProcess on the heap! Will eventually also try it.

In the meanwhile I've also found the forum guide at https://forum.qt.io/topic/52200/forum-general-guide, explaining me also the use of the voting buttons (since there are no tooltips available for them, I could only guess that they could be voting buttons) ;-) So I could give you some votes to say thank you for your help now :-)

SGaist

I didn't saw that there was no tooltip for them… At least there is for the flag and quote :)

Thanks :)