Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Terminal Example - read problem
Forum Updated to NodeBB v4.3 + New Features

Terminal Example - read problem

Scheduled Pinned Locked Moved Solved General and Desktop
22 Posts 5 Posters 2.4k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • B Offline
    B Offline
    bzgec
    wrote on last edited by
    #1

    Hello.

    My application is that when character is written to serial device, the device responds with received character and new LED state.
    With Putty terminal i have no problem, here is an example of how it should work:
    putty_example
    Here is also the Putty configuration:
    putty_config

    The problem is with Qt terminal example. I downloaded the example code from GitHub repo, branch 5.15.1. Here is the output that i am getting:
    terminal_example
    The send character is always received but LED state string is corrupted most of the time.
    Otherwise if Qt terminal is just receiving data there is no problem (connected device sends data automatically every 500ms).
    terminal_rcv

    What could be done to save this problem, also i didn't change any code from the terminal example?

    Thank you for any answers.

    1 Reply Last reply
    0
    • hskoglundH Offline
      hskoglundH Offline
      hskoglund
      wrote on last edited by
      #2

      Hi, interesting problem! Do you have Local echo enabled? if so, try turning it off. It might be that your keypresses and what's received collide somehow in the QPlainTextEdit window.

      1 Reply Last reply
      0
      • B Offline
        B Offline
        bzgec
        wrote on last edited by
        #3

        Local echo is disabled, i also tried to output data with qDebug() but result is the same.

        1 Reply Last reply
        0
        • hskoglundH Offline
          hskoglundH Offline
          hskoglund
          wrote on last edited by
          #4

          Hmm maybe the scrolling disturbs the reception of characters, what happens if you remove the linefeeds, say by changing the Console::putData() function (on line 66 in console.cpp) to this:

          void Console::putData(const QByteArray &data)
          {
              QString s = data;
              s = s.simplified();
              insertPlainText(s);
          }
          
          B 1 Reply Last reply
          0
          • mrdebugM Offline
            mrdebugM Offline
            mrdebug
            wrote on last edited by
            #5

            Maybe the problem is that simple terminal appends a new line each time On ReadyRead event is fired and sometime the event is fired two time on one line in, the first time for the first part of the row and the second time on the second part of the row.

            Please try this
            https://github.com/denisgottardello/QtComPort
            in hexadecimal mode and using new line after 200 ms

            Need programmers to hire?
            www.labcsp.com
            www.denisgottardello.it
            GMT+1
            Skype: mrdebug

            B 1 Reply Last reply
            0
            • hskoglundH hskoglund

              Hmm maybe the scrolling disturbs the reception of characters, what happens if you remove the linefeeds, say by changing the Console::putData() function (on line 66 in console.cpp) to this:

              void Console::putData(const QByteArray &data)
              {
                  QString s = data;
                  s = s.simplified();
                  insertPlainText(s);
              }
              
              B Offline
              B Offline
              bzgec
              wrote on last edited by
              #6

              @hskoglund same problem but everything in one line.

              1 Reply Last reply
              0
              • mrdebugM mrdebug

                Maybe the problem is that simple terminal appends a new line each time On ReadyRead event is fired and sometime the event is fired two time on one line in, the first time for the first part of the row and the second time on the second part of the row.

                Please try this
                https://github.com/denisgottardello/QtComPort
                in hexadecimal mode and using new line after 200 ms

                B Offline
                B Offline
                bzgec
                wrote on last edited by
                #7

                @mrdebug When i try to import this project in Qt creator it throws error:

                No valid settings file could be found.
                
                All settings files found in directory "x/QtComPort" where either too new or too old to be read
                

                If i proceed and try to build and run the project it throws error:

                This application failed to start because it could not find or load the Qt platform plugin "xcb"
                in "".
                
                1 Reply Last reply
                0
                • B Offline
                  B Offline
                  bzgec
                  wrote on last edited by bzgec
                  #8

                  Ok so i put delay before sending LED status from device (Nucleo board) and it works fine (if delay is bigger than 2ms). So the problem is that Nucleo board is sending second message right after the first one with no delay between them.
                  But that is not a solution to my problem (it works in Putty why not in Terminal example).

                  EDIT:
                  I also tried to send messages from Nucleo board without '\n' but it makes no difference.

                  1 Reply Last reply
                  0
                  • hskoglundH Offline
                    hskoglundH Offline
                    hskoglund
                    wrote on last edited by
                    #9

                    So the Terminal example program only works if the device pauses for > 2 ms (but Putty works anyway). It looks like when QSerialPort::readyRead() fires and the readAll() is issued, the program is oblivious of new characters for 2 ms :-(

                    One way is to simplify the technology, instead of relying on the readyRead() signal, you could try polling with a QTimer. As a start (just guessing!) if you change the MainWindow::readData() function to this:

                    void MainWindow::readData()
                    {
                        for (;;)
                        {
                            const QByteArray data = m_serial->read(1);
                            if (data.isEmpty())
                                return;
                            
                            m_console->putData(data);
                        }
                    

                    does it change the behavior?

                    B 1 Reply Last reply
                    0
                    • hskoglundH hskoglund

                      So the Terminal example program only works if the device pauses for > 2 ms (but Putty works anyway). It looks like when QSerialPort::readyRead() fires and the readAll() is issued, the program is oblivious of new characters for 2 ms :-(

                      One way is to simplify the technology, instead of relying on the readyRead() signal, you could try polling with a QTimer. As a start (just guessing!) if you change the MainWindow::readData() function to this:

                      void MainWindow::readData()
                      {
                          for (;;)
                          {
                              const QByteArray data = m_serial->read(1);
                              if (data.isEmpty())
                                  return;
                              
                              m_console->putData(data);
                          }
                      

                      does it change the behavior?

                      B Offline
                      B Offline
                      bzgec
                      wrote on last edited by bzgec
                      #10

                      @hskoglund Awsome, this works!

                      void MainWindow::readData()
                      {
                          for (;;)
                          {
                              const QByteArray data = m_serial->read(1);
                              if (data.isEmpty())
                                  return;
                      
                              m_console->putData(data);
                          }
                      }
                      

                      If i understand correctly this is still an asynchronous way which just checks if more data is available after reading one character instead of reading every characters it received in one go. And this helps because the QSerialPort::readyRead() doesn't fire fast enough (or corresponding slot doesn't execute fast enough) and by the time readAll() is called internal buffer is already overwritten with new data?

                      K 1 Reply Last reply
                      0
                      • B bzgec

                        @hskoglund Awsome, this works!

                        void MainWindow::readData()
                        {
                            for (;;)
                            {
                                const QByteArray data = m_serial->read(1);
                                if (data.isEmpty())
                                    return;
                        
                                m_console->putData(data);
                            }
                        }
                        

                        If i understand correctly this is still an asynchronous way which just checks if more data is available after reading one character instead of reading every characters it received in one go. And this helps because the QSerialPort::readyRead() doesn't fire fast enough (or corresponding slot doesn't execute fast enough) and by the time readAll() is called internal buffer is already overwritten with new data?

                        K Offline
                        K Offline
                        kuzulis
                        Qt Champions 2020
                        wrote on last edited by kuzulis
                        #11

                        @bzgec ,

                        is already overwritten with new data?

                        No, that's not possible, the QSP never overwrites the previous data, it just appends.

                        Could you please do the following test (please add the qdebug output to see what happpens):

                        void MainWindow::readData()
                        {
                            qDebug() << "Bytes available:" << m_serial->bytesAvailable();
                            const QByteArray data = m_serial->readAll();
                            qDebug() << "Data:" << data.toHex();
                            ...
                        }
                        
                        B 1 Reply Last reply
                        0
                        • hskoglundH Offline
                          hskoglundH Offline
                          hskoglund
                          wrote on last edited by
                          #12

                          Hi, read the documentation for the readyRead() signal and it says: "....readyRead() is not emitted recursively..."
                          Doesn't that mean there's a theoretical "blind spot" in the readData() function:

                          oid MainWindow::readData()
                          {
                              const QByteArray data = m_serial->readAll();
                          // begin "blind spot"
                              m_console->putData(data);
                          // end of "blind spot"
                          }
                          

                          I.e. if new characters arrive exactly at the same time as the ->putData(data) function runs, no readyRead() signal is fired (because we haven't got the hell out of Dodge/left readData()?
                          But the characters should not be lost anyway, just queued somewhere in a buffer, as @kuzulis says above *... never overwrites... , just appends." Maybe the Terminal example works better with another OS than your Linux, if you have a Windows PC around, you could try with that.

                          JonBJ 1 Reply Last reply
                          0
                          • hskoglundH hskoglund

                            Hi, read the documentation for the readyRead() signal and it says: "....readyRead() is not emitted recursively..."
                            Doesn't that mean there's a theoretical "blind spot" in the readData() function:

                            oid MainWindow::readData()
                            {
                                const QByteArray data = m_serial->readAll();
                            // begin "blind spot"
                                m_console->putData(data);
                            // end of "blind spot"
                            }
                            

                            I.e. if new characters arrive exactly at the same time as the ->putData(data) function runs, no readyRead() signal is fired (because we haven't got the hell out of Dodge/left readData()?
                            But the characters should not be lost anyway, just queued somewhere in a buffer, as @kuzulis says above *... never overwrites... , just appends." Maybe the Terminal example works better with another OS than your Linux, if you have a Windows PC around, you could try with that.

                            JonBJ Offline
                            JonBJ Offline
                            JonB
                            wrote on last edited by JonB
                            #13

                            @hskoglund
                            Do you know what you have said about "blind spot" to be true/the case? It would be almost a show-stopper if it did! I don't think there will be any "blind spot", so far as Qt is concerned I don't think it will report any data as "arriving" while you are inside readData(), any new data which arrives after readAll() should cause a new readyRead signal to be raised after you exit this slot? Note that readyRead is not emitted a second time until after readAll() has been performed, so user code must read all already-arrived data before he will get a new notification for newly-arriving data.

                            1 Reply Last reply
                            0
                            • K Offline
                              K Offline
                              kuzulis
                              Qt Champions 2020
                              wrote on last edited by kuzulis
                              #14

                              An idea is that the QSP does work in a buffered mode (let's say how it does work, to be clear):

                              1. When the FIFO receives at leas one byte, the serial port file descriptor gets trigegred with the 'RX' event.

                              2. This event calls the 'data read' callback which reads all available data from the FIFO to the internal QSP buffer.

                              3. If a bytes read more than zero, then emits the readyRead() signal.

                              4. The QSP::read() or QSP::readAll() methods read the data from the internal buffer of QSP (but not from the device's FIFO).

                              5. The QSP::bytesAvailable() method returns a size of the internal QSP buffer (not the data count from the device's FIFO).

                              6. If a new data gets arrived into the device's FIFO after the QSP already did read from the FIFO, then the 'RX' event will be triggered in next time when the application runs the next Qt-event loop. The 'RX' event will be triggered in each Qt-event loop until the device's FIFO contains at least one byte.

                              So, please provide the debug output for the 'bytesAvailable()' and the 'readAll()' after each readyRead() signal to see what bytes are received.

                              1 Reply Last reply
                              0
                              • K kuzulis

                                @bzgec ,

                                is already overwritten with new data?

                                No, that's not possible, the QSP never overwrites the previous data, it just appends.

                                Could you please do the following test (please add the qdebug output to see what happpens):

                                void MainWindow::readData()
                                {
                                    qDebug() << "Bytes available:" << m_serial->bytesAvailable();
                                    const QByteArray data = m_serial->readAll();
                                    qDebug() << "Data:" << data.toHex();
                                    ...
                                }
                                
                                B Offline
                                B Offline
                                bzgec
                                wrote on last edited by
                                #15

                                @kuzulis I tried your suggestion:
                                terminal_debug

                                1 Reply Last reply
                                0
                                • hskoglundH Offline
                                  hskoglundH Offline
                                  hskoglund
                                  wrote on last edited by
                                  #16

                                  Well well well @kuzulis your idea with the hex dump really did the trick!
                                  It seems that the problem isn't at all related to any serial communication but rather what happens when you stuff a QString with null bytes :-)

                                  I.e. try tossing out the zero bytes at reception, something like thisL

                                  void MainWindow::readData()
                                  {
                                      QByteArray dataWithZeroes = m_serial->readAll();
                                      QByteArray data;
                                      for (auto b : dataWithZeroes)
                                           if (b)
                                              data += b;
                                               
                                      m_console->putData(data);
                                  }
                                  
                                  JonBJ 1 Reply Last reply
                                  0
                                  • hskoglundH hskoglund

                                    Well well well @kuzulis your idea with the hex dump really did the trick!
                                    It seems that the problem isn't at all related to any serial communication but rather what happens when you stuff a QString with null bytes :-)

                                    I.e. try tossing out the zero bytes at reception, something like thisL

                                    void MainWindow::readData()
                                    {
                                        QByteArray dataWithZeroes = m_serial->readAll();
                                        QByteArray data;
                                        for (auto b : dataWithZeroes)
                                             if (b)
                                                data += b;
                                                 
                                        m_console->putData(data);
                                    }
                                    
                                    JonBJ Offline
                                    JonBJ Offline
                                    JonB
                                    wrote on last edited by
                                    #17

                                    @hskoglund

                                     QByteArray data =  m_serial->readAll().replace('\0', "");
                                    

                                    ? Totally untested, but a lot shorter ;-)

                                    1 Reply Last reply
                                    0
                                    • B Offline
                                      B Offline
                                      bzgec
                                      wrote on last edited by bzgec
                                      #18

                                      @kuzulis your idea to display hex data was quite good :)
                                      Examples from @hskoglund and @JonB both work.
                                      So the problem is that when "0\n" or "1\n" string is send and the new string is sent right behind them. Then readAll() reads whole buffer including string termination sign and other data behind it, so conversion to string (or when displaying to console) drops other characters behind terminating sign.

                                      But i what about readLine(). If i try @hskoglund and @JonB example it works, but with original terminal example it completely drops everything right behind "0\n" or "1\n".
                                      terminal_debug_readLine

                                      JonBJ 1 Reply Last reply
                                      0
                                      • B bzgec

                                        @kuzulis your idea to display hex data was quite good :)
                                        Examples from @hskoglund and @JonB both work.
                                        So the problem is that when "0\n" or "1\n" string is send and the new string is sent right behind them. Then readAll() reads whole buffer including string termination sign and other data behind it, so conversion to string (or when displaying to console) drops other characters behind terminating sign.

                                        But i what about readLine(). If i try @hskoglund and @JonB example it works, but with original terminal example it completely drops everything right behind "0\n" or "1\n".
                                        terminal_debug_readLine

                                        JonBJ Offline
                                        JonBJ Offline
                                        JonB
                                        wrote on last edited by JonB
                                        #19

                                        @bzgec said in Terminal Example - read problem:

                                        so conversion to string (or when displaying to console) drops other characters behind terminating sign.

                                        Then don't do so! Converting bytes containing \0s to strings is "problematic", as is displaying them. Do all your work on QByteArrays, and use the returned count not nul-string termination.

                                        I haven't looked, but readLine() is liable to want to find \ns, and not treat \0s specially, or well in a string. If you have to do that, split your input on \0s as necessary. or remove them as we did.

                                        B 1 Reply Last reply
                                        0
                                        • JonBJ JonB

                                          @bzgec said in Terminal Example - read problem:

                                          so conversion to string (or when displaying to console) drops other characters behind terminating sign.

                                          Then don't do so! Converting bytes containing \0s to strings is "problematic", as is displaying them. Do all your work on QByteArrays, and use the returned count not nul-string termination.

                                          I haven't looked, but readLine() is liable to want to find \ns, and not treat \0s specially, or well in a string. If you have to do that, split your input on \0s as necessary. or remove them as we did.

                                          B Offline
                                          B Offline
                                          bzgec
                                          wrote on last edited by
                                          #20

                                          @JonB Yes i tried readLine() (and posted picture in previous answer) and the problem is the same as with readAll().

                                          Ok so the problem was that the device (Nucleo board) was sending null-string termination in a message. And conversion to string (or when displaying to console) dropped everything behind it and that is why this @hskoglund's example worked (because it displayed one character at the time):

                                          void MainWindow::readData()
                                          {
                                              for (;;)
                                              {
                                                  const QByteArray data = m_serial->read(1);
                                                  if (data.isEmpty())
                                                      return;
                                          
                                                  m_console->putData(data);
                                              }
                                          }
                                          

                                          So my guess is that Putty uses one of the examples above. Which one would you recommend to use so that mistake like this doesn't show in console?

                                          Thank you all for helping!

                                          1 Reply Last reply
                                          0

                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved