Qtablewidget max data?

App is x64 and compiled on msvc.
What is maximal ammount of sortable data that Qtablwidget can hold?
Is there any way to make sorting multithreaded?

What is maximal ammount of sortable data that Qtablwidget can hold?
The maximum value you can store in an
int
theoretically (about 2 bilion items for a 32bit integer), but it'll give away way before that.Is there any way to make sorting multithreaded?
I haven't tried it, but possibly if you provide your own model implementation. However,
QTableWidget
is not for that case, you should then useQTableView
instead.Kind regards.

Tested number sorting with float data data of ~20 million rows and 5 columns, it took ~5 sec on 4.3 ghz.
What would be best way to implement fast sorting of very long struct vector of 5 floating numbers?

@Q139 said:
Tested number sorting with float data data of ~20 million rows and 5 columns, it took ~5 sec on 4.3 ghz.
With that widget and the standard model, this is not a number sorting, but a string sorting. The model will sort based on the
Qt::DisplayRole
data which is interpreted as a string and then sorted accordingly.What would be best way to implement fast sorting of very long struct vector of 5 floating numbers?
My first instict would be to use a custom model, which treats the data as numbers and doesn't go through the internal
QVariant
s. 20M structs is a relatively small dataset and should be sorted quite fast if you do it internally for the model. Displaying these 20 million rows is another matter altogether.

There's no way you'll fit 20M of numbers on a screen in any reasonable way, so there's no point sorting them all.
I'd look into creating a model that fetches subsets of the data (see canFetchMore()) and do only partial sorting of the exposed subset, for example compose std::nth_element and std::partial_sort together to get a sorted subrange. That should make a big difference to start with. There are also experimental parallel versions of these algorithms. You could look into that for another boost.

@ChrisKawa
Well, I personally would consider BSPlike (i.e. a kd tree) organization of the data, so one can easily fetch small subsets of data from the proper place, but as you said there certainly isn't a good way to display that data in a table ... :)

Thanks for all info will try dif methods of sorting and displaying data.

@kshegunov
Might be a good idea, but this route requires special care. OP didn't mention what kind of data we are talking about. It's worth mentioning that BSPs are not well suited for a dynamic dataset, as constant balancing of the structure may proove quite an overhead. For a static set of data the implementation becomes a delicate problem. Any kind of node based approach is out of question with data this size and a naive implementation with a vector is gonna suffer from cache misses a lot. It can get hairy fast.I'd stick to a simple, nonstructured vector and sort incrementally i.e. first X of elements, then next X as the user scrolls through the view and
fetchMore()
is called. This way that 5s needed to sort all the data are distributed in time and the user won't notice it.

@ChrisKawa
Using fetchMore() method does it cause overall max/min values to be fully sorted at their place only after full 20M list is scrolled troug?

@Q139 Yes, that's the idea. The model returns a small number (lets say 100) as
rowCount
and only that number of element need to be at their sorted position (hence std::partial_sort comes to mind). The model also returns true fromcanFetchMore
, which causes the view to callfetchMore()
when user scrolls to the bottom. Inside fetch more you sort elements 100200 and from now on reportrowCount
as 200 . This goes on until you run out of elements at which point you return false fromcanFetchMore()
. This way fully sorted is only the range the user actually scrolls through.