Logging System



  • So i'm trying to create a small logging system to log some data my application is spitting out.

    I have set it up like this.

    //header
    Log(enmu flag, Qstring msg);
    
    // cpp file
    Log( msg_info, "hello my sexy application");
    // the log function just calls  arguments of enumeration flag with Qstring message
    // this function is built off a case system which when flag hits logs the file to a text file
    

    in my GUI application i built tabs each tab reads a text file and appends it to a text browser.

    /// press shoe tab
    // read shoes.txt
    //append(msg); // to textbrowser
    

    so my question is this some of my text files can get as big as 1gb of text data. I feel like this is gonna consume so much memory and cpu that this type of loggin isn't the greatest idea.

    I am looking for idea or suggestions of a better system or if this idea will work.. Other issues are the text file can get updated ever few seconds so there will be alot of writing and reading from a text file to always update the text browser.

    I could use a signal on a timer and create a function that updates the read and write base on if the file has been written too and if the tab is selected.. or should i just thread this to its own class..



  • Hi,
    Have you tried QMessageLogger class? Or - for more tailored solution - custom message handler?



  • QmessageLogger is based on qDebug(), qInfo(), qWarning(), qCritical, or qFatal() functions

    Mines more built base on tabs and the same system just custom message and handler. The custom method you linked mine is built like that only difference i wrote a read and write to file function that i call.

    Im just looking for ideas or if this way will work or cost me to much cpu/memory


  • Qt Champions 2016

    @Sunfluxgames
    Hi
    We log to a sqllite database.
    When showing we only select the last x. to keep memory use down.

    Do you need to have the whole 1 GB text load all at once?

    Normally you would use a sliding view as it can get really heavy with many log lines.



  • @Sunfluxgames The word "log" made me misunderstood what you are looking for. In that case I would go with @mrjj solution - simple sqlite db with simple model to handle the data.



  • Hi all,

    This log system is just for showing whats going on inside the server.

    Each tab is just a window to display different parts of the server. I think i might just limit it base on dates.

    So if todays date matches todays text then display the text. The sqllite database is intresting but i use mysql.

    .. how the program works

    4 tabs each displays 4 different log files.. if you click on tab 1 it displays that log and updates the log each time someting is written to the log file. Then appends the data to the text browser. So say if tab 2 is later clicked on and the file is huge most likely will eat memory.


  • Moderators

    @Sunfluxgames If you're writing in files then memory consumptions will not be high, at least not if you're doing it properly (I don't know how you're writing). You just open the file in append mode and append messages - the content of the file will not be loaded into memory in this case.
    Regarding performance: you should open the file once and keep it open.
    There are already logging frameworks available, so you actually do not have to implement your own.



  • My method is pretty simple and works out of the box.

    void Log::Logged(enum flags, const QString msg)
    {
        switch (flags) 
        {
        case Server1:
            Write( LOG_FOLDER_SERVER, msg );
            break;
        case Server2:
            Write( LOG_FOLDER_SERVER2, msg );
            break;
        case Server3:
            Write( LOG_FOLDER_SERVER3, msg );
            break;
        case Server4:
            Write( LOG_FOLDER_SERVER4, msg );
            break;
        }
    }
    
    void Log::Write(filename, const Qstring msg)
    {
        QFile file(QApplication::applicationDirPath() + filename);
        if(!file.open(QFile::WriteOnly | QFile::Append |
                      QFile::Text))
        {
            return;
        }
    
        QTextStream out(&file;);
        out << msg << endl;
    
        // appen signal to textbrowser
        AppendSingal( msg );
    
        file.flush();
        file.close();
    }
    
    void Log::Read(QString filename)
    {
        QFile file(filename);
        if(!file.open(QFile::ReadOnly |
                      QFile::Text))
        {
            return;
        }
    
        QTextStream in(&file;);
        QString myText = in.readAll();
    
        // appen signal to textbrowser
        AppendSingal( myext );
    
    
        file.close();
    }
    

    I can create a read/write mutex class with semaphores which is pretty easy. I built mine off of QtMessageHandler system and just made it read and write. I still haven't added the filewatcher system so that everytime the file gets written that it updates the text browser.



  • @Sunfluxgames
    That is actually a quite heavy implementation. I would suggest that you at least make QFile file a class Member.



  • Moving the Qfile to the log class member wouldn't affect performance as the compiler is just gonna QFile it into a pointer anyways. Having 300 clients connecting to the server and spitting out logs onto a textbrowser with QT's QFileSystemWatcher seems not the best idea i could come up with.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.