Solved Model for a list of large data in QML
-
@sierdzio
This sounds reasonable to me.
The only thing is: given the OP's "crazy" ( ;-) ) requirement to have "at least 100k" items in his lists, it sounds like the approach will require yet another copy of all the data rows! :( -
@JonB said in Model for a list of large data in QML:
@sierdzio
This sounds reasonable to me.
The only thing is: given the OP's "crazy" ( ;-) ) requirement to have "at least 100k" items in his lists, it sounds like the approach will require yet another copy of all the data rows! :(That's why it will be crucial to implement @dheerendra's idea of windowed view. Or at the very least use lazy initialization (
canFetchMore()
andfetchMore()
). -
@sierdzio
Yes, but independent of whether lazy reading is involved, my comment was that OP will have one ("large") list of rows in hisQSqlQueryModel
and another ("large", potentially very similar) list in his newQAbstractListModel
. It may be inevitable, but it was just an observation. -
Oh, right, that's true.
-
@JonB
It might be a minor problem but I am still trying to solve it. I have to signals: beginDBChange and endDBChange. I use the first one to store the index of the selected item and then the second one to set it as a current item index. It works but I still have some annoying scrollings to set the item at the beginning or the end of the table. I would like this to be completely transparent to the user. -
Sounds crazy :P but I think I'll give it a try.
Does someone know if the QSqlQuery use any kind of lazy initialization? or when you execute a query the data is being loaded regardless nobody ask for it??
-
@XDePedro
Not an area I know about; can only make possible suggestion:If you really cannot solve that by (somehow) pre-scrolling before new list is shown --- because list is being completely repopulated from new data --- ISTM you will have to come up with whatever solution to only append new rows to visual list without complete reset/redraw, whatever that might involve.
-
@XDePedro said in Model for a list of large data in QML:
Does someone know if the QSqlQuery use any kind of lazy initialization?
Like all Q*Model classes, there is canFetchMore() and fetchMore().
Whether any further laziness is implemented in the actual loading - I don't know.
-
@sierdzio
I have been wondering about just this forQSqlQueryModel
etc. for a while now!Do we know how the "fetchMore" is actually implemented? I presume this is a driver implementation (
QMYSQL
)? For example, does it actually use a SQLCURSOR
on a still on-going query (I hope not! but maybe it does), does it simply mean it fetches another packet from some complete SQL resultset, or what??This makes a huge difference to how I might implement efficiency in my SQL code....
-
Feel free to analyze the code here.
-
@sierdzio
Thanks for that!Unfortunately for me, I do all my work in PyQt. That means I can't step through the code in debugger to follow everything, as I would if I used C++ and had the sources.
Although the page's hyperlinks are good, they're not nearly nearly the same as following through stepping.
A brief look doesn't tell me much. Though:
QSqlQueryModel::fetchMore()
Fetches more rows from a database.
This only affects databases that don't report back the size of a query
(see QSqlDriver::hasFeature()).At a guess MySQL will return "size of a query" (perhaps SQLite does not), so it may be all irrelevant in my situation?
In any case, as you'll see I don't think the implementation will be in
QSql...
. That will rely on the abstraction level of the driver it's using,QMYSQL
, and I'm thinking that is where what I would like to know is buried...But thanks anyway.