display a huge horizontal dataset in a QScrollingView



  • Hi,
    I'am trying to create a Human genom viewer. Something like this :
    alt text

    Briefly, You display a huge dna sequence by scrolling left and right. The dna sequence is like a string of ACGT caracter and you can select a region by its coordinate ( start, end ).
    You can draw annotation above the sequence, like "gene position" or "reads" which are short sequence alignement comming from a sequencer.

    I m looking for the most efficient way to build this viewer. I would like scrolling and zoom action. I plan to use a QAbstractScrollArea.
    My first idea is to create a QPixmap render(int start, int end, int viewportWidth) and create them if necessary using a QContiguousCache. But I have some difficult to see how it can work with scrolling and resize event.

    alt text

    If you have suggestion , you are welcome...
    I already have a POC avaible on github which display bacteria genom. You can clone it and run it easily . ( Be sure to copy Data into /tmp/ before ) https://github.com/labsquare/bamviewer-poc


  • Lifetime Qt Champion

    Hi,

    I'd suggest to rather use the model view approach with a custom QAbstractItemView. That way you don't have to create one huge pixmap for all the data. With a custom model you can render the nodes that are currently visible to the user rather than having everything loaded at once.


  • Moderators

    @dridk2
    Also you could use a QGraphicsView to display such an amount of data. It can handle thousands of items easily and also supports zooming. But don't use QPixmaps for just displaying colored rects!

    If you can't get around using QPixmaps you must make sure only to load/create the pixmaps on the fly for the visible area only. The target machine's memory will thank you.


  • Moderators

    @raven-worx said in display a huge horizontal dataset in a QScrollingView:

    If you can't get around using QPixmaps you must make sure only to load/create the pixmaps on the fly for the visible area only. The target machine's memory will thank you.

    But CPU won't ;-) For views with large amount of paths and shapes I've found it's often better to use cached pixmaps instead of painting everything repeatedly.

    However, that is not something OP should worry about now. It's better to implement something first and worry about optimization later, if it will really be a problem.


  • Moderators

    @sierdzio said in display a huge horizontal dataset in a QScrollingView:

    For views with large amount of paths and shapes I've found it's often better to use cached pixmaps instead of painting everything repeatedly.

    Correct me if i am wrong, but a genom can consist of millions (even billions?) of data. So by caching so much data in a pixmap the limits on the machine are reached very fast.

    However, that is not something OP should worry about now. It's better to implement something first and worry about optimization later, if it will really be a problem.

    Thats a very dangerous advice IMHO. In the (very likely) worst case everything has to be rewritten.
    As a developer it can never be wrong to think about such things before starting implementing, so i am a little bit confused about such an advise?!

    But this is just my opinion.


  • Moderators

    @raven-worx said in display a huge horizontal dataset in a QScrollingView:

    Correct me if i am wrong, but a genom can consist of millions (even billions?) of data. So by caching so much data in a pixmap the limits on the machine are reached very fast.

    But there is only a small number of pixmaps (~10 if I count right on the GIF from first post) that are repeated everywhere. So the cache won't be big. And obviously it is wasteful to paint anything that is not visible on the screen - that data won't contribute to RAM or CPU usage, if the model is written properly.



  • @sierdzio said in display a huge horizontal dataset in a QScrollingView:

    @raven-worx said in display a huge horizontal dataset in a QScrollingView:

    Correct me if i am wrong, but a genom can consist of millions (even billions?) of data. So by caching so much data in a pixmap the limits on the machine are reached very fast.

    But there is only a small number of pixmaps (~10 if I count right on the GIF from first post) that are repeated everywhere.

    Where do you get "~10" from? There can be any combinations of "ACGT"s. If he's creating a pixmap for every possible 16 contiguous letters (that's all I can think of you might mean, though I wouldn't do it that way), I make that 4 ^ 16 for the permutations? Which is big :)


  • Moderators

    @JonB said in display a huge horizontal dataset in a QScrollingView:

    @sierdzio said in display a huge horizontal dataset in a QScrollingView:

    @raven-worx said in display a huge horizontal dataset in a QScrollingView:

    Correct me if i am wrong, but a genom can consist of millions (even billions?) of data. So by caching so much data in a pixmap the limits on the machine are reached very fast.

    But there is only a small number of pixmaps (~10 if I count right on the GIF from first post) that are repeated everywhere.

    Where do you get "~10" from? There can be any combinations of "ACGT"s. If he's creating a pixmap for every possible 16 contiguous letters (that's all I can think of you might mean, though I wouldn't do it that way), I make that 4 ^ 16 for the permutations? Which is big :)

    4 letters, 2 colours = 8 images. Plus additional H, P and T on top -> 11 images. OK there are a few more colours there as sometimes the letter is red and sometimes black. There is no reason to store pixmaps for combinations of them.

    Heck, there is no concrete reason to store pixmaps at all, it was just an idea. OP wanted some ideas, and we've all come up with some hints. He'll need to experiment and choose from them.



  • @sierdzio
    OK, sorry, I completely misunderstood which bitmaps we were talking about...!


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.