[Solved]QStringlist - strings missing in long list
-
Hi,
Am fetching large data ( more than 30K records( only 1 column)) from database and keeping in QStringlist container for further processing and i found that some records went missing. The total items in the list are not matching with the records i got from the database.
Facing the same problem with QHash and QMap because of collision and it seems QStringlist is also using QHash in its internal implementation.
so i want to know.
- Is this is a bug or limitation of QStringlist
- Is there any other way to store large data without loss.
-
Data is stored with signed int indexes (definitely for QList and QVector, I'm not sure about the implementation of QHash and QMap), so when you go beyond that, you are on your own. You can try STL containers, or any other solution that is prepared for big amounts of data. As this is a special use-case, not required in most applications, Qt Project has no interest in making containers use an unsigned int or quint64.
-
A signed 32-bit int can still go up to 2,147,483,647 so I doubt 30K is an issue.
- How are you fetching the data?
- How can you tell that collisions happened?
-
Might this have to do with thread workings? Do you use multithreading? QStringList is NOT threadsafe!
It shouldn't be a problem to store 30k of data in a stringlist. I would advise to use a QList of QString * instead. The StringList will otherwise consume a large chunck of your stack. Using 'new' and the pointers to it will use the heap, which is as large as what your OS gives you. -
[quote author="Jeroentje@home" date="1383127266"]Might this have to do with thread workings? Do you use multithreading? QStringList is NOT threadsafe![/quote]Good thinking. Yes, that's a very possible cause.
[quote]I would advise to use a QList of QString * instead. The StringList will otherwise consume a large chunck of your stack. Using 'new' and the pointers to it will use the heap, which is as large as what your OS gives you. [/quote]Incorrect. Qt containers (including QList AND QString) all store their internal data on the heap.
On a stack:
- A QString, no matter how long, occupies the same amount of space as a pointer.
- A QList, no matter how large, occupies the same amount of space as a pointer.
That's how Qt containers can be "implicitly shared":http://qt-project.org/doc/qt-5.1/qtcore/implicit-sharing.html. There is never any need to call new QString().
-
Thanks you all for your suggestions.
Yes i was using multithread and even increased thread counts and was updating a single list. now i used mutex at time of updating list and its working fine.
-
Qt containers are great, but still I prefer to do it manually, even though Qt can do it better ;-) but I stand corrected, the qstrings are placed on the heap.
If you have found the answer, please place [SOLVED] in front of your first post!
