Tail-f function in qt



  • hi
    I have a file from which I will extract data and display. The file will keep on getting new entries from other program. Now My gui should be able to extract new entries from the file. Something like "tail -f" in linux.
    Is there any way to implement such thing in qt.
    Thank you



  • what you want is a pipe, not an ever growing file.

    Either use RPC pipes, or, preferably, on unix use mknod to use file-system based streams. The latter you can just use QFile on.



  • Thanks Thomas...
    If possible can u please provide some sample or a code snippet or guide to some docks...i m trying but not able to get properly...





  • Thanks thomas...
    but the problem is the file from which i have to get data is in csv format and i dont have control over it to change...and it has to be a csv file itself...so is there anyway of getting data from that csv file on the fly



  • Ok, I'm probably going too fast and assuming knowledge of unix filesystem details.

    In unix you can create a stream between two applications by just creating a special file on the filesystem. One application can then write to it and another application can read from it.
    The beauty of this approach is that your application does not have to do anything different in parsing content from the pipe or from a normal file.
    They are both accessible using the QFile class, with little-to-no alterations.

    Read more in your basic unix tech manual about mknod and S_FIFO.

    As such, the content of the file is irrelevant when you use a fifo-node as a file, you just make one application write to it and the other reads it at the same time. Using normal file APIs.

    Reading a csv file is just like regular, which means using QFile and things like QString::split or something like that.
    I'm sure there are some simple csv parsers written in qt findable using google if thats what your biggest problem is :)



  • [quote author="Thomas Zander" date="1361953152"]
    Reading a csv file is just like regular, which means using QFile and things like QString::split or something like that.
    I'm sure there are some simple csv parsers written in qt findable using google if thats what your biggest problem is :)[/quote]

    Using QString::split is ONLY convenient if the file has no embedded "commas". QRegExp is another way but trust me there are no Universal Regular Expression for CSV files.

    I have a different approach I wrote a weeks ago. @devfeel, if you have time to read, you may visit my blog at "QtSimplify":http://qtsimplify.blogspot.com/2013/02/dealing-with-csv-files-easy-way.html



  • Thanks thomas...
    I am able to extract the contents from the csv file...no issue in it...my only aim is that as and when the csv file is updated, my gui should also get updated...

    And I am unable to write the contents of the csv file to fifo file...



  • bq. And I am unable to write the contents of the csv file to fifo file…

    why?



  • This is the code...dont know what i am doing wrong
    @#include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <fcntl.h>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <unistd.h>
    #define MAX 512

    int main()
    {
    int fd;
    char * myfifo = "myfifo";

        /* create the FIFO (named pipe) */
        mkfifo(myfifo, 0666);
    
        /* write  to the FIFO */
        open(myfifo, O_WRONLY);
    

    FILE *file, *file2;
    char line[MAX];

    file = fopen("alert.csv", "r");
    //file2 = fopen(myfifo, "w");
    while (fgets(line,sizeof(line),file) != NULL)
    {
    /*Write the line */
    fputs(line, myfifo);
    printf(line);

    }
    fclose (file);
    // fclose (file2);
    unlink(myfifo);
    close(fd);
    return 0;
    }
    @


  • Moderators

    If you just run a process as a QProcess and need to get its output you could also just have it print to stdout and use QProcess' methods to access the output.



  • After you create the fifo, you open it like its a file, then it works.

    Your app modified;
    @
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <fcntl.h>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <unistd.h>
    #define MAX 512

    int main()
    {
        int fd;
            char * myfifo = "myfifo";
    
            /* create the FIFO (named pipe) */
            mkfifo(myfifo, 0666);
    
            /* write  to the FIFO */
            open(myfifo, O_WRONLY);
    
      FILE *file, *file2;
      char line[MAX];
    
      file = fopen&#40;"alert.csv", "r"&#41;;
      file2 = fopen&#40;myfifo, "w"&#41;;
      while (fgets(line,sizeof(line),file) != NULL)
      {
        /*Write the line */
        fputs(line, file2);
        printf(line);
    
      }
      fclose (file);
     // fclose (file2);
       unlink&#40;myfifo&#41;;
       close(fd);
      return 0;
    }
    

    @

    Naturally, its a pipe, a first-in-first-out pipe.
    If you put stuff in but nobody is on the other side to read it, it will soon stop being able to take more in.

    So start a second shell and type
    tail -f myfifo
    directly after your app started.



  • Thanks for having so much patience with me...
    Now its working..but the fifo is not updating as and how the alert.csv is updating...



  • Dear Thomas
    My gui app is unable to get the contents from the fifo file...even though starting simultaneously...
    The below code is to write to fifo file
    @ #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <fcntl.h>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <unistd.h>
    #define MAX 512

    int main()
    {
        int fd;
            char * myfifo = "myfifo";
    
            /* create the FIFO (named pipe) */
            mkfifo(myfifo, 0666);
    
            /* write  to the FIFO */
            open(myfifo, O_WRONLY);
    
      FILE *file, *file2;
      char line[MAX];
    
      file = fopen&#40;"alert.csv", "r"&#41;;
      file2 = fopen&#40;myfifo, "w"&#41;;
      while (fgets(line,sizeof(line),file) != NULL)
      {
        /*Write the line */
        fputs(line, file2);
        printf(line);
    
      }
      fclose (file);
     // fclose (file2);
       unlink&#40;myfifo&#41;;
       close(fd);
      return 0;
    }@
    

    The below code is to display the contents from the fifo file
    @#include "mainwindow.h"
    #include "ui_mainwindow.h"
    #include "QtGui"
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    #include <fcntl.h>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <unistd.h>
    QTableWidgetItem *item;
    int row=0,column=0;
    MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
    {
    ui->setupUi(this);
    ui->alertshow->setColumnWidth(0,200);
    ui->alertshow->setColumnWidth(1,250);
    }

    MainWindow::~MainWindow()
    {
    delete ui;
    }

    void MainWindow::on_update_clicked()
    {
    QString line;
    int fd;
    char * myfifo = "/root/qt/untitled4/myfifo";

        /* create the FIFO (named pipe) */
        //mkfifo(myfifo, 0666);
    

    fd=mkfifo(myfifo, S_IFIFO | 0666);
    //qDebug()<<fd;
    /* write to the FIFO */
    open(myfifo, O_RDONLY);
    //qDebug()<<fd;

     QFile file2(myfifo);
    
    file2.open(QIODevice::ReadOnly | QIODevice::Text);
    QTextStream in(&file2);
    while (!in.atEnd())
    {
        qDebug()<<"inside while";
     line = in.readLine();
     qDebug()<<line;
     QString delimiterPattern(",");
     QStringList fonts = line.split(delimiterPattern);
    
             qDebug() << fonts;
    

    display(fonts);
    // row++;
    file2.close();

    }
    qDebug()<<"outside";
    

    }

    void MainWindow::display(QStringList list)
    {
    for(int i=0;i<list.count();i++)

    {
    //i++;
    // qDebug()<<i;
    QString li=list[i];

    item=new QTableWidgetItem(li);
    ui->alertshow->setItem(row,column,item);
    column++;
    }
    }@



  • Your beginning idea was that you want to update whenever something new is written by the first program.
    A fifo can do that, but indeed its a bit tricky.
    The basic concept behind the fifo is that its blocking. So your code that reads from the fifo will never return. Not untill you delete the fifo, at least.

    As such you likely want to run the read in a different thread and make it notify the main gui thread whenever a new line was successfully read.
    It also means you never close the file.



  • Hi Thomas...
    fifo was giving some problems..so used pipe...in terminal i open my gui executable with tail command.
    i.e tail -f alert.csv | ./mygui

    i am getting all the values in console using qdebug...but mygui is freezing and displaying nothing.... any idea?
    @QTableWidgetItem *item;
    int row=0,column=0;
    MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
    {
    ui->setupUi(this);
    ui->alertshow->setColumnWidth(0,200);
    ui->alertshow->setColumnWidth(1,250);
    }

    MainWindow::~MainWindow()
    {
    delete ui;
    }

    void MainWindow::on_update_clicked()
    {
    QString line;

    QTextStream in(stdin);
    do
    {
        qDebug()<<"inside while";
     line = in.readLine();
     qDebug()<<line;
     QString delimiterPattern(",");
     QStringList fonts = line.split(delimiterPattern);
    
     qDebug() <<"fonts"<< fonts;
     display(fonts);
      
    }while(!line.isNull());
    
    qDebug()<<"outside";
    

    }

    void MainWindow::display(QStringList list)
    {
    for(int i=0;i<list.count();i++)
    {

    QString li=list[i];
    qDebug()<<"csv"<<li;
    item=new QTableWidgetItem(li);
    ui->alertshow->setItem(row,column,item);
    column++;
    }
    }
    @


  • Lifetime Qt Champion

    Aren't you running the do while loop forever ?

    Try with line.isEmpty(), IIRC "" is empty but not null.

    Hope it helps



  • You switched to reading a pipe, but the actual problem is not with the fifo its that your method on_update_clicked() will never return.

    What you may not realize is that this means that the painting system will never do anything anymore, since its waiting for the on_update_clicked method to finish.

    So, I think the fifo was a great idea, I suggest using that again.

    I realize that threads are not easy, so let me suggest a different approach that will work too.
    You can modify your on_update_clicked to exit when there is no data to read. See QIODevice::canReadLine()

    If you get tired of pressing the button you might want to instead connect that slot up to the QIODevice::readyRead() signal from your input file.

    GOod luck!



  • Thanks SGaist...it didnt worked...

    And Thomas i tried with fifo also...same issue...I have attached the code yesterday...if possible check...I am working on readyread as u said...



  • The below code is giving error...can you please help
    @QTextStream in(stdin);
    QObject::connect(in, SIGNAL(readyRead()), this, SLOT(readcsv()));@

    error: no matching function for call to 'MainWindow::connect(QTextStream&, const char [13], MainWindow* const, const char [11])'


  • Lifetime Qt Champion

    QTextStream does not have a readyRead signal, QFile does (from QIODevice).



  • Evn this is not working...its not entering readcsv()
    @ QFile file("alert.csv");

    file.open(QIODevice::ReadOnly | QIODevice::Text);

    QObject::connect(&file,SIGNAL(readyRead()),  SLOT(readcsv()));@


  • "qfile wont emit signals like qio":http://qt-project.org/doc/qt-4.8/qfile.html#signals


  • Lifetime Qt Champion

    Right ! I forgot about that.
    Did you have a look at QSocketNotifier ? It might be what you need to monitor a pipe on unix.



  • thanks Sgaist
    @ notifier = new QSocketNotifier(STDIN_FILENO, QSocketNotifier::Read);
    connect(notifier, SIGNAL(activated(int)), this, SLOT(readcsv()));@

    and used repaint() function in readcsv function...now gui is showing the contents ..but gui is still freezed...
    any possible solution


  • Lifetime Qt Champion

    Do you still have an infinite loop somewhere ?



  • hi sgaist...This is the code...only using while(!line.isEmpty());
    have tried with isnull() also..but same
    @void MainWindow::readcsv()
    {
    QString line;
    //ui->alertshow->repaint();
    qDebug()<<"in readcsv";
    QTextStream in(stdin,QIODevice::ReadOnly);

    do
    {
      // ui->alertshow->repaint();
        qDebug()<<"inside while";
     line = in.readLine();
     qDebug()<<line;
     QString delimiterPattern(",");
     QStringList fonts = line.split(delimiterPattern);
    
     qDebug() <<"fonts"<< fonts;
    
     display(fonts);
    
    }while(!line.isEmpty());
    
    qDebug()<<"outside";
    

    }

    void MainWindow::display(QStringList list)
    {
    ui->alertshow->repaint();
    for(int i=0;i<list.count();i++)
    {

    QString li=list[i];
    qDebug()<<"csv"<<li;
    item=new QTableWidgetItem(li);
    ui->alertshow->setItem(row,column,item);
    column++;
    }
    // ui->alertshow->repaint();
    }
    @


  • Lifetime Qt Champion

    Wild idea (i don't have *nix system right now at hand)

    Why don't you do a readAll() and then parse the lines from that ?



  • Not sure if you would be interested, but I have abit of code that starts a tail -f as a QProcess and just redirects the output of that process to a QLineEdit

    It's not hard to do, let me know and ill lookup the snippet when I get back at work on Monday.

    Cheers



  • Thanks clogwog...It would be very helpful if u provide the snippet.
    [quote author="clogwog" date="1362136280"]Not sure if you would be interested, but I have abit of code that starts a tail -f as a QProcess and just redirects the output of that process to a QLineEdit

    It's not hard to do, let me know and ill lookup the snippet when I get back at work on Monday.

    Cheers[/quote]



  • header:

    @myclass
    {
    startFollowingTail();

    private:

    QProcess procTail;

    private slot:
    void readFromStdoutForTail();
    }@

    in constructor:

    @myclass::myclass() : ..... , procTail(this)
    {
    connect( &procTail, SIGNAL(readyReadStandardOutput()),this, SLOT(readFromStdoutForTail()) );
    }

    myclass::startFollowingTail()
    {
    if( procTail.Running)
    procTail.kill();

            QStringList alist;
            alist << "-f" << "/tmp/software_update.log";
            procTail.start("/usr/bin/tail", alist, QIODevice::ReadWrite);
    

    }

    void myclass::readFromStdoutForTail()
    {
    while ( procTail.canReadLine())
    {
    QByteArray a = procTail.readLine();
    QString data = a.data();
    ui->plainTextEdit->insertPlainText(data);
    // scroll to end
    ui->plainTextEdit->verticalScrollBar()->setValue(ui->plainTextEdit->verticalScrollBar()->maximum());
    }
    }@



  • I've only tested this on Linux (Fedora Core 16) and Qt 4.8.4, but this worked like "tail -f" for me:

    @volatile bool shutdown = false;
    QTextStream textStream;

    void tailFollow()
    {
    while (!shutdown)
    {
    if (textStream.atEnd())
    QThread::sleep (1);
    else
    {
    QString line = textStream.readLine();
    // Do something with line here...
    }
    }
    }@

    In the actual code, "shutdown" and "textStream" were member variables of a QThread subclass, and tailFollow() was a method on that QThread subclass. The client would set "shutdown" to true when the loop should exit.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.