Fast text file downloading. Fast searching in the text file.



  • The problem with download speeds of a text file and the problem with the speed of search in a text file.

    I have a program generates a text file - "Generator"
    This file must be read simultaneously and displayed on the screen by
    another program - "Reader".

    I ran into a situation where reading a file by "Reader" program ends
    some time after the end of "Generator". After the long time.

    File size is ~10Kb - 100MB
    Delay is of 3 seconds to 10 minutes
    Letter 'e' searching in the file size of 2 MB takes about 80 seconds.

    I use the following code to download the file and search for it :
    @QTextEdit *textEdit ;

    QTextStream in( file );
    textEdit -> append( in.readAll() );
    .........................

    QTextDocument *tmpDoc = textEdit -> document( ) ->clone( this ) ;

    QTextCursor cursor ( tmpDoc ) ;
    cursor = tmpDoc -> find ( regexp , cursor );@

    So I want to speed up text loading and searching in it.

    Could you please advise something. ?
    Thanks.


  • Moderators

    [quote author="zerg2011" date="1307106068"]
    QTextDocument *tmpDoc = textEdit -> document( ) ->clone( this ) ;
    [/quote]
    You seem to generate a copy of your original file. If you can avoid the clone, it should be faster.



  • Depending what it is you are searching for (just simple words or really complex RegExp), you could just read a line at a time and search that line for the text and append it to the TextDocment á la:

    @
    QTextDocument *document = textEdit->document();
    QTextCursor cursor(document);
    cursor.beginEditBlock();
    while(!in.atEnd())
    {
    QString data = in.readLine();
    int index = data.indexOf(regexp);
    //do something with that
    cursor.insertText(data);
    }
    cursor.endEditBlock();
    @

    But as I said this will only work if you're looking for simple expressions that don't span multiple lines (if that's what you want)



  • [quote author="koahnig" date="1307107463"]

    ... If you can avoid the clone, it should be faster.
    [/quote]
    Frankly I did not notice much difference with clone() and without.

    [quote author="loladiro" date="1307123109"]Depending what it is you are searching for (just simple words or really complex RegExp), you could just read a line at a time and search that line for the text and append it to the TextDocment ....)[/quote]

    I do both - looking for simple words and complex RegExp

    I experimented with reading line by line like this :

    QtextStream in ;
    QString line = in . readLine() ;
    .........
    processing( line ) ;
    .........
    textEdit -> append( line ) ;

    I believe that the append function works very slowly.

    textEdit -> append( in.readAll() );
    works faster than
    textEdit -> append( line ) ;
    but still too slow.

    I would like to achieve the speed of reading and finding comparable with gvim.
    There is also insertPlainText(). But after insertPlainText() needed to re-format all text

    A search for likely have to rewrite by myself :)



  • Maybe you shouldn't always append to the textEdit, just try out "this":http://doc.qt.nokia.com/4.6/qstring.html#more-efficient-string-construction method, and in the and this way you have to call the setText() function only once.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.