Solved What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?
-
Do you mean your alternative is to convert the QMap "on-the-fly" just in order to serialize it, or do you mean you would maintain the corresponding list/string already.
I mean both. What is the performance impact in converting the
QMap
on the the fly and then serializing the result as opposed to serializing theQMap
as is. And Also I mean what is the cost of serializing theQMap
as is Versus serializing an already converted QString.In other words, is serialization of the
QMap
as is, is significantly more costly in terms of size and speed as opposed to converting theQMap
to a string, and also as opposed to serializing a pre-converted QString.Another separate point is whether you need/already have to have a QMap rather than, say, a QHash?
I don't have a QMap already, because I have not yet decided on what data stricture to use. I would assume that serialization of QHash is more costly than QMap.
-
@Mike-Fidel
I am not a Qt expert, unlike some.My observation about "on-the-fly" was that, intuitively to me, I would be surprised if first converting to another data representation just in order to serialize would end up saving time or space --- I would have thought the opposite.
As for
QHash
vsQMap
. I assume you must have a large number of elements to care about serialization performance. Have a read of https://woboq.com/blog/qmap_qhash_benchmark.html. It is not to do with serialization, but might well affect your usage. And also note the conclusion:The typical rule is: Use QMap only if you need the items to be sorted or if you know that you always have a very small amount of items in your map.
-
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
I would like to know if there is a significant performance\size cost to serializing a QMap with QDataStream as opposed to converting the QMap into a QList or a regular QString
How would you convert a key-value container to a sequential one (i.e. QList)? And also
QList
is a bad choice if the data is larger than the size ofvoid *
.What is the performance impact in converting the QMap on the the fly and then serializing the result as opposed to serializing the QMap as is.
The conversion cost for the key and value into the temporary + the serialization cost of the intermediate result. With great probability the conversion cost to an intermediate data structure will overgrow the serialization cost of the key-value pairs significantly for all possible cases.
And Also I mean what is the cost of serializing the QMap as is Versus serializing an already converted QString.
I don't see how serializing to text as intermediary lifts from from the weight at all. Moreover deserializing from the intermediate results will really sink your ship ...
In other words, is serialization of the QMap as is, is significantly more costly in terms of size and speed as opposed to converting the QMap to a string
Almost certainly not.
and also as opposed to serializing a pre-converted QString
If you need to construct the pre-converted string which I imagine you do, same answer as above.
I would assume that serialization of QHash is more costly than QMap.
Why would you assume that?
-
@kshegunov said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
How would you convert a key-value container to a sequential one (i.e. QList)?
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
I would assume that serialization of QHash is more costly than QMap.
Why would you assume that?
Because
QHash
needs to also store the 'hash' value for each key. -
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
Which is already serializing the data to a text stream. You just would do it two times for no obviously identifiable reason.
Because QHash needs to also store the 'hash' value for each key.
No it doesn't. The hash table stores the key (to handle hash collisions) and the value in each bucket. There's no reason to store the hash, as it's an implementation detail that is not required to be persistent. If you access by key, the key is hashed on the fly to find the bucket. If you use iterators, then the hash is not at all needed.
-
Thanks! This is very informative.
If you need to construct the pre-converted string which I imagine you do, same answer as above.
What if I don't need to pre-convert the string. If I already have the string for 'free'...
Let's say I have a
QMap
and aQString
with the same amount of data. What would be the penalty for serializing QMap vs QString?I guess what I'm asking is whether or not serializing
QMap
(OrQHash
) incur significant overhead when compared to serializing the same amount of data in more basic types likeQString
. -
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
Let's say I have a QMap and a QString with the same amount of data. What would be the penalty for serializing QMap vs QString?
This is really hard to answer, I'd run a series of tests if I really cared about it.
I guess what I'm asking is whether or not serializing QMap (Or QHash) incur significant overhead when compared to serializing the same amount of data in more basic types like QString.
As the containers are templates it will vary between the used key and value types, but if we assume they are
QString
s, I would put an educated guess and say, no not significant. -
@Mike-Fidel
One thing: do you have, say, tens of thousands of keys in yourQMap
? -
@JonB No... No more than about10 key value pairs.
-
Then what's the relevance of how efficient serialization is ...?
-
@kshegunov Hence my question :)
-
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
No more than about10 key value pairs.
With such a small amount of data, the "performance, space and overhead" really doesn't matter (unless you're serializing and deserializing hundreds of thousands of times a second...?)
Choose the option that gives you the most readable code.
Related reading: https://stackify.com/premature-optimization-evil/
I would like to know if there is a significant performance\size cost to...
The best way to find out is to benchmark them.
-
@JKSH said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
With such a small amount of data, the "performance, space and overhead" really doesn't matter (unless you're serializing and deserializing hundreds of thousands of times a second...?)
And if you were serializing the same
QMap
(s) hundreds of thousands times per second, you should rather look a caching the result(s) of the serialization/deserialization than at the serialization code itself, of course! -
Thank you everyone for your answers!