What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?
-
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
I would like to know if there is a significant performance\size cost to serializing a QMap with QDataStream as opposed to converting the QMap into a QList or a regular QString
How would you convert a key-value container to a sequential one (i.e. QList)? And also
QListis a bad choice if the data is larger than the size ofvoid *.What is the performance impact in converting the QMap on the the fly and then serializing the result as opposed to serializing the QMap as is.
The conversion cost for the key and value into the temporary + the serialization cost of the intermediate result. With great probability the conversion cost to an intermediate data structure will overgrow the serialization cost of the key-value pairs significantly for all possible cases.
And Also I mean what is the cost of serializing the QMap as is Versus serializing an already converted QString.
I don't see how serializing to text as intermediary lifts from from the weight at all. Moreover deserializing from the intermediate results will really sink your ship ...
In other words, is serialization of the QMap as is, is significantly more costly in terms of size and speed as opposed to converting the QMap to a string
Almost certainly not.
and also as opposed to serializing a pre-converted QString
If you need to construct the pre-converted string which I imagine you do, same answer as above.
I would assume that serialization of QHash is more costly than QMap.
Why would you assume that?
@kshegunov said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
How would you convert a key-value container to a sequential one (i.e. QList)?
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
I would assume that serialization of QHash is more costly than QMap.
Why would you assume that?
Because
QHashneeds to also store the 'hash' value for each key. -
@kshegunov said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
How would you convert a key-value container to a sequential one (i.e. QList)?
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
I would assume that serialization of QHash is more costly than QMap.
Why would you assume that?
Because
QHashneeds to also store the 'hash' value for each key.@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
Which is already serializing the data to a text stream. You just would do it two times for no obviously identifiable reason.
Because QHash needs to also store the 'hash' value for each key.
No it doesn't. The hash table stores the key (to handle hash collisions) and the value in each bucket. There's no reason to store the hash, as it's an implementation detail that is not required to be persistent. If you access by key, the key is hashed on the fly to find the bucket. If you use iterators, then the hash is not at all needed.
-
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
For example I could put them in a string ''{Key: Value}'' if the QMap is <QString,QString>.
Which is already serializing the data to a text stream. You just would do it two times for no obviously identifiable reason.
Because QHash needs to also store the 'hash' value for each key.
No it doesn't. The hash table stores the key (to handle hash collisions) and the value in each bucket. There's no reason to store the hash, as it's an implementation detail that is not required to be persistent. If you access by key, the key is hashed on the fly to find the bucket. If you use iterators, then the hash is not at all needed.
Thanks! This is very informative.
If you need to construct the pre-converted string which I imagine you do, same answer as above.
What if I don't need to pre-convert the string. If I already have the string for 'free'...
Let's say I have a
QMapand aQStringwith the same amount of data. What would be the penalty for serializing QMap vs QString?I guess what I'm asking is whether or not serializing
QMap(OrQHash) incur significant overhead when compared to serializing the same amount of data in more basic types likeQString. -
Thanks! This is very informative.
If you need to construct the pre-converted string which I imagine you do, same answer as above.
What if I don't need to pre-convert the string. If I already have the string for 'free'...
Let's say I have a
QMapand aQStringwith the same amount of data. What would be the penalty for serializing QMap vs QString?I guess what I'm asking is whether or not serializing
QMap(OrQHash) incur significant overhead when compared to serializing the same amount of data in more basic types likeQString.@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
Let's say I have a QMap and a QString with the same amount of data. What would be the penalty for serializing QMap vs QString?
This is really hard to answer, I'd run a series of tests if I really cared about it.
I guess what I'm asking is whether or not serializing QMap (Or QHash) incur significant overhead when compared to serializing the same amount of data in more basic types like QString.
As the containers are templates it will vary between the used key and value types, but if we assume they are
QStrings, I would put an educated guess and say, no not significant. -
Thanks! This is very informative.
If you need to construct the pre-converted string which I imagine you do, same answer as above.
What if I don't need to pre-convert the string. If I already have the string for 'free'...
Let's say I have a
QMapand aQStringwith the same amount of data. What would be the penalty for serializing QMap vs QString?I guess what I'm asking is whether or not serializing
QMap(OrQHash) incur significant overhead when compared to serializing the same amount of data in more basic types likeQString.@Mike-Fidel
One thing: do you have, say, tens of thousands of keys in yourQMap? -
@Mike-Fidel
One thing: do you have, say, tens of thousands of keys in yourQMap?@JonB No... No more than about10 key value pairs.
-
@JonB No... No more than about10 key value pairs.
Then what's the relevance of how efficient serialization is ...?
Read and abide by the Qt Code of Conduct
-
@kshegunov Hence my question :)
-
@JonB No... No more than about10 key value pairs.
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
No more than about10 key value pairs.
With such a small amount of data, the "performance, space and overhead" really doesn't matter (unless you're serializing and deserializing hundreds of thousands of times a second...?)
Choose the option that gives you the most readable code.
Related reading: https://stackify.com/premature-optimization-evil/
I would like to know if there is a significant performance\size cost to...
The best way to find out is to benchmark them.
Qt Doc Search for browsers: forum.qt.io/topic/35616/web-browser-extension-for-improved-doc-searches
-
@Mike-Fidel said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
No more than about10 key value pairs.
With such a small amount of data, the "performance, space and overhead" really doesn't matter (unless you're serializing and deserializing hundreds of thousands of times a second...?)
Choose the option that gives you the most readable code.
Related reading: https://stackify.com/premature-optimization-evil/
I would like to know if there is a significant performance\size cost to...
The best way to find out is to benchmark them.
@JKSH said in What is the serialization cost of QMap VS QList or QString in terms of performance, space and overhead?:
With such a small amount of data, the "performance, space and overhead" really doesn't matter (unless you're serializing and deserializing hundreds of thousands of times a second...?)
And if you were serializing the same
QMap(s) hundreds of thousands times per second, you should rather look a caching the result(s) of the serialization/deserialization than at the serialization code itself, of course! -
Thank you everyone for your answers!
