Memory Layout of Qt QVarLengthArray of QString Internals

markanth

What is the internal data structure of a QVarLengthArray?

For example, if I where to have:

QVarLengthArray<QString> anArray;

QString string1 = "whatever";

QString string2 = "something else";

anArray [0] = string1;

anArray [1] = string2;

Is it easy to pre-calculate &anArray [1] given &anArray ?

I have been traipsing through the QVarLengthArray source code trying to understand how QVarLengthArray stores an array of QStrings in memory. As much as I like Qt, one thing that is particularly painful to me is its opaque pointer basis. (The helper functions in the debugger help in some cases, but when really trying to dig into the internals, the opaque pointers obscure a great deal of information that would otherwise be available in the debugger.)

I found a couple "Qt Internals" articles on codeproject.com and elsewhere, but none helped.

In general, it would be great to have a way to peer into the real data structures behind the opaque pointers, but for the immediate need it would be great to understand if there is a good way to predict the start address of each element in the QVarLengthArray of MyClass which contains pointers, QStrings, and integers. I do understand how it is moved from the stack to the heap based on a threshold, etc.

(Having this information will help simplify a custom serialization. I do understand the risks to reusability and am willing to accept those risks for this experiment.)

Rolias

Since this has been up for a while with no responses I thought I would hazard a reply.

From the docs it looks like each item is stored contiguously in memory based on the size of the pointer. Specifically, under the docs for the data() function is this example:
@QVarLengthArray<int> array(10);
int *data = array.data();
for (int i = 0; i < 10; ++i)
data[i] = 2 * i;@

So your QString example will return a pointer to a QString. You could write something like
@ QVarLengthArray<QString> anArray(2);
QString string1 = "whatever";
QString string2 ="something else";
anArray[0] = string1;
anArray[1] = string2;
QString* data_ptr = anArray.data();
qDebug() << data_ptr[0] << data_ptr[1];
qDebug() << *data_ptr;
qDebug() << *++data_ptr;@

And it would produce

whatever something else
whatever
something else

Of course you can't modify the contents of anArray once you start using the pointer from data().

markanth

Thank you very much for your reply, Rolias. Although, I have not tried the code you included, I do believe it would work exactly as you have described. What I intended to ask with the original question; however, was whether there was a way to pre-determine (mathematically--not with code), the exact address of each QString element. I understand I could just use the & operator to get it. Specifically, I was looking to understand how QStrings are allocated in memory and how they were stored in the containers.

I suppose I could use the & operator on some examples to help me infer the internals, but I have not done that yet. For that matter, I could traipse through the source code and eventually see exactly what is what. However, that would really take some effort and time. Any additional suggestions will also be very much appreciated.

As part of trying to understand the information about the internals requested above, I have been using gdb/Creator, but the opaque pointers and other internals seem to obscure everything despite the enabling of the pre-built python helpers. Any pointers on how to better view (for example, QVarLengthArray<MyNode>, where MyNode includes a couple integers and a QString) could also be very helpful to anyone trying to debug Qt applications. (I would think so, anyway.)

Maybe something like this is the best answer:

http://plohrmann.blogspot.com/2013/10/writing-debug-visualizers-for-gdb.html

But man (oh man), what a pain, and probably quite a few hours of diversion for the uninitiated like myself.

(Of course, it will likely continue to pay dividends as time goes on, but I do not have the time to invest in the near future. By the way, if anyone is interested in writing the helpers for me, I would be happy to pay them for their time. Please let me know if that offer violates any of the forum's protocol or is otherwise frowned-upon.)

JKSH

Have you seen the answer to your StackOverflow post? http://stackoverflow.com/questions/25902864/qt-qvarlengtharray-of-qstring-internals/26007582#26007582

markanth

I have not. Thanks. I'll check it out in the morning. Cheers.

andrep

[Edit] Visualization of QVarLengthArray should be supported out of the box in Creator, but isn't. Expect this to work in 3.3. Until then, put something like
@def qdump__QVarLengthArray(d, value):
data = d.extractPointer(value["ptr"])
size = int(value["s"])
d.putItemCount(size)
d.putPlotData(data, size, d.templateArgument(value.type, 0))@
at the end of your share/qtcreator/debugger/qttypes.py

markanth

Thanks for the suggestion. I just now tried it and it did not work for me, although I could easily have made a mistake having never tried to modify the qttypes.py before.

I cut and pasted your code above into the .py file, but I now get <not accessible> in the debugger for all QVarLengthArray data.

For example:
@
QVarLengthArray<int> intTest;

intTest.append(1);
intTest.append(2);
intTest.append(3);
intTest.append(4);@

shows up in Locals and Expressions as:

intTest <not accessible> QVarLengthArray<int, 256>

I added your code at the top of the file just above what was previously the first def instruction.

Here is the top of the file (excluding comments):

import platform
from dumper import *

@def qdump__QVarLengthArray(d, value):
data = d.extractPointer(value["ptr"])
size = int(value["s"])
d.putItemCount(size)
d.putPlotData(data, size, d.templateArgument(value.type, 0))

def qdump__QAtomicInt(d, value):
d.putValue(int(value["_q_value"]))
d.putNumChild(0)
@

markanth

One painful "workaround," that looks like it might somewhat work is to insert an new expression evaluator and put something like this in it:

@(*(((int *) &intTest)))@10@

And then after skipping the first couple, the memory of which are apparently used for something internal to the QVarLengthArray, it appears that the nodes can be seen.

Of course, having it understand the container would be much better.

andrep

What version of Qt Creator are you using?

markanth

That was it. (Version was the key.) I thought I was using 3.2.2, but when I checked, help->about it showed 3.1.2. This was after upgrading last week.

(Strange. The installer appeared to upgrade successfully.)

Anyway, by completely uninstalling everything, deleting the directories which were not completely removed by the uninstall process, and then reinstalling, I am now definitely running 3.2.2, and the definition you recommended works like a champ.