Is there a direct way to know the lines of a text file ?
-
@JonB said in Is there a direct way to know the lines of a text file ?:
Or, possibly, implement the scroll drag slot to incrementally read the file as required.
That is what @artwaw is suggesting. It is the neatest, most efficient way to read in only what it is needed, but it does require some work. Also, if for the sake of argument the user scrolls in one go to the end of file, you would have to read the whole file, which is what you do not like on the main thread as it will cause UI freeze.
-
Hi, try this:
How to load large data from txt file in Qt
https://stackoverflow.com/questions/46753753/how-to-load-large-data-from-txt-file-in-qt -
Hi, try this:
How to load large data from txt file in Qt
https://stackoverflow.com/questions/46753753/how-to-load-large-data-from-txt-file-in-qt@RonaldViscarraL
The trouble is, the approach there which supposedly is the quickest is simplydata = readFile.readAll();. If the OP here says he has triedreadAll()and it's too slow for his size file, it doesn't really solve.@cyberpunker
You are usingstream.readLine( ). You could use somestream.read( )and do your own line counting, there might be an improvement there.Just how big/how many lines is your log file going to grow to? Have you actually tried any realistic testing of the read delay?
-
@cyberpunker
You cannot get to a particular line number in a text file without somehow reading all the lines (up to the desired one at least) and counting, period.If you're finding it freezes the UI, you'll have to push the reading into a thread. Or, put your own slot on the scrolling and read incrementally only as necessary.
-
The api : QString QTextStream::readLine(qint64 maxlen = 0) ?
I also try to use a thread to deal with the text file , then send the readlines to the QTextEdit by SIGNAL~SLOT , and it seems very clumsy.
@cyberpunker
I am suggesting you might try not usingQTextStream::readLine(), and insteadQTextStream::read()orQTextStream::readAll(), plus then do your own in-memory searching for\ns to count lines. The theory is that may be faster thanQTextStream::readLine()s. Frankly I'm dubious how much difference it might make, but you could give it a go.I did ask you just how big this file is, in bytes & lines? Have you actually tried some code with timings?
Your original question was, is it possible to know where a line is in a text file without reading each line, and the answer is, no it is not.
So, to prevent GUI freeze you are left with either doing something in a separate thread, or doing something where the user scrolling causes the file to be read in incrementally. at that time.
-
@cyberpunker
I am suggesting you might try not usingQTextStream::readLine(), and insteadQTextStream::read()orQTextStream::readAll(), plus then do your own in-memory searching for\ns to count lines. The theory is that may be faster thanQTextStream::readLine()s. Frankly I'm dubious how much difference it might make, but you could give it a go.I did ask you just how big this file is, in bytes & lines? Have you actually tried some code with timings?
Your original question was, is it possible to know where a line is in a text file without reading each line, and the answer is, no it is not.
So, to prevent GUI freeze you are left with either doing something in a separate thread, or doing something where the user scrolling causes the file to be read in incrementally. at that time.
hi,
@JonB said in Is there a direct way to know the lines of a text file ?:
original question was, is it possible to know where a line is in a text file without reading each line
on windows you can use FINDSTR command with qprocess
FINDSTR /N "the word i search" file.txt -
hi,
@JonB said in Is there a direct way to know the lines of a text file ?:
original question was, is it possible to know where a line is in a text file without reading each line
on windows you can use FINDSTR command with qprocess
FINDSTR /N "the word i search" file.txt -
@LeLev
That just reads each line in the file, so why should that be any more efficient than, say,QTextStream::readAll(), which the OP is saying is/might be too slow? -
In fact , it is the glog file , the file size is variable , I want to parse the log file and show its content according to a time period. The time range may may spans several log files , and the start time or stop time can be any position of the log file . So I need a mechanics which is efficecy , resource save, and human kind .
-
In fact , it is the glog file , the file size is variable , I want to parse the log file and show its content according to a time period. The time range may may spans several log files , and the start time or stop time can be any position of the log file . So I need a mechanics which is efficecy , resource save, and human kind .
@cyberpunker
So I have given you the options. You will have to read all of the log file(s) in order to accomplish this, there is no shortcut. Have you looked at/answered where I keep asking you " Have you actually tried some code with timings?"? -
Is there a direct way to know the lines of a text file ?
Instead of:
QFile file( "datum.txt" );
if ( file.open( IO_ReadWrite ) )
{
QTextStream stream( &file );while ( !stream.atEnd() )
{
count += 1;
line = stream.readLine( );
if(lineNo == count)
printf( "%3d: %s\n", count, line.latin1() );}
file.close();I am going to show show app's running log in the QTextEdit . The size of the log is unpredictable and changeable . So if I load a file into the QTextEdit , the time and efficency are variable, and if the file is too big to show , the GUI maybe dead.
I also try to use a thread to deal with the text file , then send the readlines to the QTextEdit by SIGNAL~SLOT , and it seem very clumsy.
So I going to get the line number of the text first , and the I can read the specific line out and show according to the scrollbar's draging . The api is QString QTextStream::readLine(qint64 maxlen = 0)
Or may use "memory map file "?
@cyberpunker To reinterpret your question, you want to know when another application appends another line to datum.txt? Perhaps QIODevice::waitForReadyRead() may be one way to determine when datum.txt changes size?
[https://doc.qt.io/qt-5/qiodevice.html#waitForReadyRead](link url) -
@cyberpunker To reinterpret your question, you want to know when another application appends another line to datum.txt? Perhaps QIODevice::waitForReadyRead() may be one way to determine when datum.txt changes size?
[https://doc.qt.io/qt-5/qiodevice.html#waitForReadyRead](link url)@Psnarf
I have not tried it out, but do you have any evidence thatQIODevice::waitForReadyRead()will be triggered if another process writes to a file which this process has open? My suspicion is that it will not, but I could be wrong. I would expect to have to use https://doc.qt.io/qt-5/qfilesystemwatcher.html#fileChanged if I wanted to be notified of a file content change from an external program.
