Viewing huge files with a Qt app
-
Hi,
Do you mean something like QTextDocument ?
-
@deisik it's the underpinning of the QTextEdit and QPlainTextEdit classes.
Interested in AI ? www.idiap.ch
Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct -
So it all comes down to dynamic loading and unloading data. I'm looking for something as simple and easy as adding/removing a row in a table
In that case, one thing you can consider is using the model view paradigm to show your file so you can use a QListView to show each line of your file.
You can either load your whole file as a QStringListModel but it will likely be too slow for your needs. Or create your own model that will have a certain amount of lines in memory to show and scroll and will load/unload what needed when the user scrolls.Interested in AI ? www.idiap.ch
Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct -
In that case, one thing you can consider is using the model view paradigm to show your file so you can use a QListView to show each line of your file.
You can either load your whole file as a QStringListModel but it will likely be too slow for your needs. Or create your own model that will have a certain amount of lines in memory to show and scroll and will load/unload what needed when the user scrolls. -
The problem with QListView (or QTableView, for that matter) is that you won't be able to select text across multiple lines (rows) in a regular way like with QTextEdit
-
It didn't change much, if anything at all. Tried to open a 90Mb file and had to euthanize it
@deisik
Then as previously suggested you probably need to read only a portion of the file into memory for display and handle scrolling programmatically. Which is the same principle if you used a model.Old thread on this forum https://forum.qt.io/topic/4519/how-do-i-open-large-file-on-qtextedit seems to indicate this kind of approach.
BTW: for your 90MB, was it taking a long time on reading the data from file (should not be) or on displaying it in the
QPlainTextEditon screen? -
@deisik
Then as previously suggested you probably need to read only a portion of the file into memory for display and handle scrolling programmatically. Which is the same principle if you used a model.Old thread on this forum https://forum.qt.io/topic/4519/how-do-i-open-large-file-on-qtextedit seems to indicate this kind of approach.
BTW: for your 90MB, was it taking a long time on reading the data from file (should not be) or on displaying it in the
QPlainTextEditon screen?It freezes reading, not displaying. Actually, it doesn't freeze, but will likely take a couple hours to process the file contents
It is not the part about reading a portion of the file which is cumbersome, it is the part where you have to select the text programmatically to erase it which just doesn't feel right
-
It freezes reading, not displaying. Actually, it doesn't freeze, but will likely take a couple hours to process the file contents
It is not the part about reading a portion of the file which is cumbersome, it is the part where you have to select the text programmatically to erase it which just doesn't feel right
@deisik
You are big on "things don't feel right" when they seem fine to me :)BTW if you have some inbuilt aversion to "using the cursor" to delete/insert, you don't have to. Just re-use
setPlainText()to set the whole of the document to the newly calculated range. I don't know, but in practice this may be just as fast as using the cursor. Also BTW you could try just putting the text in aQLabel(probably wrapped in a scroll area), that does not use a cursor. You have to set the complete text, there is no insert/remove.I suggest there is something wrong if your file reading code "takes a couple of hours" to read in 90MB, don't you think...? It should take like, I don't know, a few seconds.... This is fundamental no matter which approach you take to display it.
-
@deisik
You are big on "things don't feel right" when they seem fine to me :)BTW if you have some inbuilt aversion to "using the cursor" to delete/insert, you don't have to. Just re-use
setPlainText()to set the whole of the document to the newly calculated range. I don't know, but in practice this may be just as fast as using the cursor. Also BTW you could try just putting the text in aQLabel(probably wrapped in a scroll area), that does not use a cursor. You have to set the complete text, there is no insert/remove.I suggest there is something wrong if your file reading code "takes a couple of hours" to read in 90MB, don't you think...? It should take like, I don't know, a few seconds.... This is fundamental no matter which approach you take to display it.
@JonB said in Viewing huge files with a Qt app:
@deisik
You are big on "things don't feel right" when they seem fine to me :)BTW if you have some inbuilt aversion to "using the cursor" to delete/insert, you don't have to. Just re-use
setPlainText()to set the whole of the document to the newly calculated range. I don't know, but in practice this may be just as fast as using the cursor. Also BTW you could try just putting the text in aQLabel(probably wrapped in a scroll area), that does not use a cursor. You have to set the complete text, there is no insert/remove.I'm thinking about it. QLabel is probably the way to go. How can I make it look like a QTextEdit widget?
I suggest there is something wrong if your file reading code "takes a couple of hours" to read in 90MB, don't you think...? It should take like, I don't know, a few seconds.... This is fundamental no matter which approach you take to display it.
I have to parse the file (at least, would have the part to display)
And while we are at it, just switching to a tab (window) containing a big enough QTextEdit object takes a few minutes
-
@JonB said in Viewing huge files with a Qt app:
@deisik
You are big on "things don't feel right" when they seem fine to me :)BTW if you have some inbuilt aversion to "using the cursor" to delete/insert, you don't have to. Just re-use
setPlainText()to set the whole of the document to the newly calculated range. I don't know, but in practice this may be just as fast as using the cursor. Also BTW you could try just putting the text in aQLabel(probably wrapped in a scroll area), that does not use a cursor. You have to set the complete text, there is no insert/remove.I'm thinking about it. QLabel is probably the way to go. How can I make it look like a QTextEdit widget?
I suggest there is something wrong if your file reading code "takes a couple of hours" to read in 90MB, don't you think...? It should take like, I don't know, a few seconds.... This is fundamental no matter which approach you take to display it.
I have to parse the file (at least, would have the part to display)
And while we are at it, just switching to a tab (window) containing a big enough QTextEdit object takes a few minutes
@deisik said in Viewing huge files with a Qt app:
I have to parse the file (at least, would have the part to display)
Sorry, but if whatever parsing you do on a 90MB file takes "a couple of hours" as you state, or even if that's on 1GB, something is not right.
just switching to a tab (window) containing a big enough QTextEdit object takes a few minutes
Which is why you should use a
QPlainTextEditnot aQTextEdit. Your timing may be affected by options, for example word wrap mode. -
@deisik said in Viewing huge files with a Qt app:
I have to parse the file (at least, would have the part to display)
Sorry, but if whatever parsing you do on a 90MB file takes "a couple of hours" as you state, or even if that's on 1GB, something is not right.
just switching to a tab (window) containing a big enough QTextEdit object takes a few minutes
Which is why you should use a
QPlainTextEditnot aQTextEdit. Your timing may be affected by options, for example word wrap mode.@JonB said in Viewing huge files with a Qt app:
@deisik said in Viewing huge files with a Qt app:
I have to parse the file (at least, would have the part to display)
Sorry, but if whatever parsing you do on a 90MB file takes "a couple of hours" as you state, or even if that's on 1GB, something is not right
I assume that it would take a couple of hours or so. I need real time
Anyway, why don't you try it out and check for yourself?
-
@JonB said in Viewing huge files with a Qt app:
@deisik said in Viewing huge files with a Qt app:
I have to parse the file (at least, would have the part to display)
Sorry, but if whatever parsing you do on a 90MB file takes "a couple of hours" as you state, or even if that's on 1GB, something is not right
I assume that it would take a couple of hours or so. I need real time
Anyway, why don't you try it out and check for yourself?
@deisik
You think it would take a couple of hours to do your parsing on a 90MB/1GB file?!Anyway, why don't you try it out and check for yourself?
Sorry, you would like me to write and test what you want? Not to mention, without even knowing what your "parsing" entails? That's not how it works: YOU can "try it out and check for yourself?" since it's your issue, not me.
-
@deisik
You think it would take a couple of hours to do your parsing on a 90MB/1GB file?!Anyway, why don't you try it out and check for yourself?
Sorry, you would like me to write and test what you want? Not to mention, without even knowing what your "parsing" entails? That's not how it works: YOU can "try it out and check for yourself?" since it's your issue, not me.
-
Sorry, you would like me to write and test what you want?
No, just try opening a 90Mb file with QTextEdit (QPlainTextEdit), then feel free to make statements that "something is not right"
@deisik said in Viewing huge files with a Qt app:
then feel free to make statements that "something is not right"
If you think parsing such a file should take hours then I suggested something is not right. Feel free to ignore my suggestions, I was trying to help you but won't bother if that is your reaction.
-
Sorry, you would like me to write and test what you want?
No, just try opening a 90Mb file with QTextEdit (QPlainTextEdit), then feel free to make statements that "something is not right"
@deisik said in Viewing huge files with a Qt app:
No, just try opening a 90Mb file with QTextEdit (QPlainTextEdit), then feel free to make statements that "something is not right"
Just tried an 86MB file read into
QPlainTextEditand displaying it. About 8 seconds. Not exactly "hours". Now can I make my statement suggesting that "something is not right" in your estimation/finding? And btw this is considerably quicker that loading it into a text editor.That file (typical text?) came out as 2.5 million lines of text. As an end user I'm not sure what the point of displaying that many lines to me is anyway. So you still have the option of only reading/displaying portions as user scrolls to reduce it considerably if you wish.
