Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Fast text file downloading. Fast searching in the text file.



  • The problem with download speeds of a text file and the problem with the speed of search in a text file.

    I have a program generates a text file - "Generator"
    This file must be read simultaneously and displayed on the screen by
    another program - "Reader".

    I ran into a situation where reading a file by "Reader" program ends
    some time after the end of "Generator". After the long time.

    File size is ~10Kb - 100MB
    Delay is of 3 seconds to 10 minutes
    Letter 'e' searching in the file size of 2 MB takes about 80 seconds.

    I use the following code to download the file and search for it :
    @QTextEdit *textEdit ;

    QTextStream in( file );
    textEdit -> append( in.readAll() );
    .........................

    QTextDocument *tmpDoc = textEdit -> document( ) ->clone( this ) ;

    QTextCursor cursor ( tmpDoc ) ;
    cursor = tmpDoc -> find ( regexp , cursor );@

    So I want to speed up text loading and searching in it.

    Could you please advise something. ?
    Thanks.



  • [quote author="zerg2011" date="1307106068"]
    QTextDocument *tmpDoc = textEdit -> document( ) ->clone( this ) ;
    [/quote]
    You seem to generate a copy of your original file. If you can avoid the clone, it should be faster.



  • Depending what it is you are searching for (just simple words or really complex RegExp), you could just read a line at a time and search that line for the text and append it to the TextDocment á la:

    @
    QTextDocument *document = textEdit->document();
    QTextCursor cursor(document);
    cursor.beginEditBlock();
    while(!in.atEnd())
    {
    QString data = in.readLine();
    int index = data.indexOf(regexp);
    //do something with that
    cursor.insertText(data);
    }
    cursor.endEditBlock();
    @

    But as I said this will only work if you're looking for simple expressions that don't span multiple lines (if that's what you want)



  • [quote author="koahnig" date="1307107463"]

    ... If you can avoid the clone, it should be faster.
    [/quote]
    Frankly I did not notice much difference with clone() and without.

    [quote author="loladiro" date="1307123109"]Depending what it is you are searching for (just simple words or really complex RegExp), you could just read a line at a time and search that line for the text and append it to the TextDocment ....)[/quote]

    I do both - looking for simple words and complex RegExp

    I experimented with reading line by line like this :

    QtextStream in ;
    QString line = in . readLine() ;
    .........
    processing( line ) ;
    .........
    textEdit -> append( line ) ;

    I believe that the append function works very slowly.

    textEdit -> append( in.readAll() );
    works faster than
    textEdit -> append( line ) ;
    but still too slow.

    I would like to achieve the speed of reading and finding comparable with gvim.
    There is also insertPlainText(). But after insertPlainText() needed to re-format all text

    A search for likely have to rewrite by myself :)



  • Maybe you shouldn't always append to the textEdit, just try out "this":http://doc.qt.nokia.com/4.6/qstring.html#more-efficient-string-construction method, and in the and this way you have to call the setText() function only once.


Log in to reply