Listmodel and large amount of data



  • Hi

    I have a book which has very many pages. The book has many chapters and subchapters. If I put books chapter 10 to the model and scroll the book forward, the view calls automatically canFetchMore and fetchMore functions (QAbstractListModel). These functions fetch more chapters to the model and it works well. The problem is how to implement a situation where the book shoulb be scrolled backwards from chapter 10. I can't load the whole book to the model because of performance reasons. So I kind of need a "window" which moves forwards or backwards when scrolling.

    I hope you understood my situation



  • From what I understood the docs, there is only a "forward" fetchMore(), but not a backward version.

    You could try to implement some "lazy loading" in your model. That would mean, that you populate the nodes with some dummy data only (say the chapters' titles or so) and load the actual data (the text?) only if it is needed in the data() method. If that works out depends on the sort and structure of the data of course. It could well be that I'm on the wrong track with that suggestion :-)



  • Are there any other solution suggestions?



  • [quote author="huot" date="1331724245"]Are there any other solution suggestions?[/quote]
    Perhaps you can first specify why Volker's suggestion isn't suitable for your situation, and tell us more about your underlying data structure?



  • My data structure consists of more than 30 000 items. To me loading 30 000 items feels like wasting memory. My view shows only 10 items. I would like to keep the memory consumption as small as possible.



  • It's hard to give better advice with that much information. We do not know the storage of your data, nor what it effectively consist of - that can be a few words up to many pages, and so on. You can "fake" the items in the model and create them as they are needed, provided you return appropriate data for the views to do their calculations (eg. the size hints for the items), etc. Not every node needs to incarnate "physically" in the model.



  • @huot: those 30 items:

    • Are they structured or indexed or something like that, or is it one huge list?
    • Is your data random accessible or not?
    • Are these items big (kilobytes) or small (a few bytes each)?
    • Are all your items going to be same size, or do they have different sizes?
    • ...

    You are simply not providing enough information.



  • Maybe I didn't make myself clear enough.

    I am developing a bible reader application for Nokia N9. As you may know the bible consists of books, chapters and verses. Item of the model is similar to bible verse. The data added to the model one chapter at a time (consist of many verses). If the user picks a verse he should be able to scroll both backwards and forwards.



  • Volker and Andre, any suggestions how I should implement this?



  • Please, practice a bit of patience. Kicking your issue this fast is not considdered very polite, we both have day jobs and families that demand our attention as well.

    It seems to me your data is very easy to structure. Do you want to have all books, chapters and verses in one big list? Or do you think structuring the data is acceptable?



  • I'm sorry if I seemed too impatient. I didn't mean to be rude. I was thinking that all books, chapters and verses would be in one big list so that it is easy for the user to scroll the book back and forth. I don't know if this is a good idea but this is how I have thought it.



  • huot,

    I've started to port symbianbible (http://compactbyte.com/symbianbible/) to qt (https://groups.google.com/forum/?fromgroups#!topic/symbianbible/lbSRh8e0r_o).

    The PalmVersion of Bible Reader (Bible++) had same problem: performance.

    Yohanes (the author) keep the original reader code, bacause is optimzed (low memory, low speed).

    I'm starting to port (not recompiliing) the PDB reader and mantain the same structures, because there's a lot PDB bible files in the net. But I'm thinking to migrate the data to SqlLite.

    I'd have pushed the initial source code (initial focus to PDB Reader ) to bitbucket, but I'll migrate to github.

    Can you check the sources?



  • I'd forgot the link to the project at BitBucket:

    https://bitbucket.org/raphaelquati/qtbible



  • TioRay:
    I use Sword project (www.crosswire.org/sword) as backend.

    Volker:
    I tried to implement your suggestion of "lazy loading" but I didn't get it working. The problem is that in the data() method the data of the model can't be modified because data() method is CONST.

    Any suggestions how to solve this problem?



  • Just mark the storage member of your model subclass as mutable. This way you can modify it even in a const method.

    Doing so is perfectly ok in your case, as the const of data() promises to not change the logical state of your object. In case of lazy loading, you do not change the logical state, but only the physical representation (i.e. the bits and bytes).



  • [quote author="Volker" date="1332158256"]Just mark the storage member of your model subclass as mutable. This way you can modify it even in a const method.

    Doing so is perfectly ok in your case, as the const of data() promises to not change the logical state of your object. In case of lazy loading, you do not change the logical state, but only the physical representation (i.e. the bits and bytes).[/quote]

    Volker:

    I tried your suggestion. The problem is that I can't call beginInsertRows() function within data() function because beginInsertRows() function isn't CONST.

    It seems to be very difficult to implement this with Qt. Any new suggestions are welcome!


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.