Circular buffer model
-
Hi,
I'm pretty new to Qt and have been really enjoying getting into it, but I'm seeking the opinion of more experienced heads on a problem I'm working on.
My project is a data logging utility that will capture rapidly occurring events (10s of ms) and display various information about the events in a QTableView. The intention is to continuously capture data, but review recent events if something interesting happens. Now, there are probably lots of ways to skin that cat, but I was thinking about using a fixed size circular queue/fifo like structure to store the events. What I'm not quite sure about is how to build a model around that data.
I've tried subclassing QAbstractTableModel, which seems to work fine when the buffer is first filling up, by just calling
@beginInsertRows(, <number of items in buffer>, <number of items in buffer>);
... insert item into buffer ...
endInsertRows();@But when the buffer is full and I'm removing items as I insert new items, I'm not as sure. Right now, I'm first removing the earliest item
@beginRemoveRows(, 0, 0);
... remove item from buffer ...
endRemoveRows();@then doing an insert as above. This kind of works, but I'm not sure if it's conceptually correct, and sometimes, events which I know have occurred don't show up (although this could always be another problem).
So what I'm looking for are opinions about the right way to implement this kind of model. I'm also interested in ideas as to how the last item can be kept visible (if it was visible) across insertions and if the last item isn't visible, how the current top item can remain at the top across insertions.
Any ideas are much appreciated.
Dan
-
Hmmmm,
I'm more an embedded guy, so bear with me....
Do the insertRows until your data is full. Then when new data comes along, place it in a new spot. (keep the pointer in you model as member).
In your data() function of the model use that "pointer" to start as position 0 etc.
When new data is in you can place it in the row just beyond the pointer and emit the signal dataChanged (index = pointer + 1).
Increase the pointer and done.
One big downsize of this methode is that you will have to update the entire view every data update. So this is about the same as your way of doing it.
Is't it a better way to use a TextEdit box?? In here the widget can be used to add strings to the end and keep the end focussed. When the data exceeds the gui limit, you get scrollbars automatic. It's probably much faster then the View/Model way of doing it.
Maybe to test the data use the console option and use QDebug to show what events are captured in your model.Let me know what the solution is. Never to smart to learn.
-
Thanks for the reply Jeroen, I'm actually a low level embedded programmer most of the time too, hence my inclination towards fixed size fifos :)
I'm not sure if I completely follow your suggestion though. Currently I have an underlying fifo data structure which holds the data, and I map the model indexes onto my underlying data, with the earliest item always being model index 0. So once the fifo is full, with every insertion the model index for each item has effectively changed. I hoped that this would be accommodated for by using the beginRemoveData/endRemoveData calls although I don't know exactly the effect they have on all other items. I feel like your talking about letting the model handle the data too, but I may just be misunderstanding you.
As for other ways to solve this, I probably could do something like just having a TextEdit and adding lines to it, but I guess I wanted to have the flexibility to run this thing for days at a time and not worry about it dealing with an unwieldy amount of data (millions of lines). That may not be an actual problem, but there are also other more familiar ways I'd solve this, like just writing a simple console app if I wanted to go down that path.
Really, I'm interested in expanding my knowledge of how the model/view stuff works with this question (as well as solving the problem at hand), I'm really curious to know whether this sort of scenario can be handled elegantly and efficiently.
Thanks
-
Hi there,
Think we have the same ideas here. Your fifo data structure should do the trick and model index 0 will be the first inserted in the fifo. That's what I suggested, so sorry I couldn't get it clear in writing.The big disadvantage of that way is that the entire view needs to be updated every time you add a row (because old row 2 will become row 1 and so on). So the emit(dataChanged(index)) will contain the entire view causing all rows to be redrawn.
There are some pretty great examples of model/view programming
"Model/View programming":http://doc.qt.digia.com/4.7-snapshot/modelview.htmlThe beginRemoveData/endRemoveData stuff is a way to communicate between the model and the view class. The model and the view are both running independently sort of like threads. (I'm not really sure that they run in separate threads). The beginRemove/endRemove is only so the View understands that the model isn't ready to give any actions if the view asks for it. The view will not update until the model gives the endRemove. Otherwise you might get memory problems when the data function is called. e.g. your in the middle of updating your model in the setData function. The view is resized in the GUI causing the view to ask the model for data to display. It will first ask for a rowCount value. If you where about to remove a row (but your not done it just yet) the model will return too many rows to the view. The view then will use the data() function to read out all information needed to display. The model continues also, removing a row. The view gets to the last row (the one the model just deleted) and the program might crash or give strange data on screen.
Please read the model/view tutorial first, it gives a good idea what it does.You might want to use the QTextEdit to display a number of lines. If the number of lines get to big set the overwrite function and set the insertion position back to 0?
It might also be a good idea to have the fifo in a separate thread so the GUI doesn't influence the data stream.
Greetz. Hmm, you got a nice little problem here. Think I'll try to check it out myself this weekend. Will let you know.
-
Ok that clears things up a bit.
I did figure that the beingRemove/endRemove stuff was a protection mechanism, but I didn't realise it would always (at least in my case) cause a dataChanged to be emitted for the entire model. I had sort of thought that the redrawing would be smarter when using the more precise insert/remove operations rather than the more blunt beginResetModel/endResetModel, although I can see in this case that when you remove the first item(s) all other item's indexes are changed. Is it possible to have a model which doesn't have it's indexes starting at 0, so after the buffer is filled, on the next insertion, a new item would be added with an index one larger than the current last item, and all of the other indexes would remain the same, but there would no longer be an index 0?
I didn't quite follow this comment
bq. You might want to use the QTextEdit to display a number of lines. If the number of lines get to big set the overwrite function and set the insertion position back to 0?
are you talking about a different way of implementing this, because at the moment, I have a fixed size fifo, so the number of lines will increase to the size of the fifo then remain constant.
At any rate, I've fixed some bugs in my code and it does work correctly inserting rows until full, then doing a remove followed by an insert once full. Once the buffer is full, on a removal I take note of the top visible item, and adjust it for the number of lines being removed/inserted, and then use QTableView::scrollTo to ensure it is still the top item after the insertions are complete. This is a bit messy, but achieves the effect I desired. I guess this is the scenario which could be optimised though, where all the data on screen is unchanged, but will get redrawn on an insertion.
I think it's an interesting problem too, which is why I'm keen to discuss it. Thanks again for your thoughts on the topic.
