How to dynamically store address of a QVector in another QVector?

[CJha]

Hi, I am working with large amounts of data, something like 100,000 double values, being gathered every 100 milliseconds or so. I have to store 25 or even more of these generated data in my software at any given time. Both, the data length on each acquisition as well as the number of acquisitions varies depending on the current situation. I do not want to store my data as QVector<QVector<double>> because QVector stores data in adjacent memory locations and each addition of QVector<double> to QVector<QVector<double>> results in the entire thing being copied to a new location given, resulting in huge lags (especially if there is already 20 QVector<double> present and I am adding the 21st one to it).

My solution is to store the given data as QVector<QVector<double>*> i.e. each data acquisition I am just storing the address to the QVector for the current acquisition. However, I want it to be stored in QVector<QVector<double>*> independently i.e. as long as I don't clear the pointer I am able to access the data. I am unable to figure out how to do this. I have tried multiple methods. Let's say I declare QVector<QVector<double>*> myVec; in my .h file and I use a function to new add data to it on each acquisition like this (below function is just for testing, that's why I am creating data inside the function):

void MainWindow::addData()
{
    myVec << &QVector<double>(0) // Creating a new empty vector and storing it's location
    *myVec[0] << 2.7 << 3.4 << 4.5;
}

This doesn't work, most of the time it throws an exception, and a few times it just shows some garbage value.

void MainWindow::addData()
{
    QVector<double> tempVec; // Create a temprory vector
    tempVec << 2.7 << 3.4 << 4.5;
    myVec << &tempVec;
}

This also doesn't work, of course, because tempVec is destroyed as soon as addData() is exited.

void MainWindow::addData()
{
    QVector<double> tempVec; // Create a temprory vector
    tempVec << 2.7 << 3.4 << 4.5;
    myVec << &QVector(tempVec);
}

I thought this would work as I am not storing the address of tempVec but rather copying tempVec in a new QVector and assigning its address to myVec, but it is also throwing an exception.

Is there any way I can store QVector purely as pointers inside another QVector ?

[Christian Ehrlicher]

Create the vector on the heap. But don't forget to delete them later on.

btw: why do you need such a crude construct?

[CJha]

@Christian-Ehrlicher Thanks! I will try now to create it on heap and delete it when no longer required.
I need to do this so that my application remains responsive when I add new data even if there is lots of data already added. Because except for adding data there is a huge computational overhead of calculating with data and there is nothing I can do about reducing time there.

[Christian Ehrlicher]

@CJha How should this influence the responsiveness?

[CJha]

@Christian-Ehrlicher Due to memory reallocation? Please correct me if I am wrong (I am new to C++).
As far as I understand when the size of a QVector increases beyond the current capacity the entire vector will be moved to a different memory location where enough free memory is available. So, if I already have 25 QVectors each with 500,000 doubles stored in a QVector<QVector<double>> and I add a new (26th) QVector with 500,000 doubles to it, then in the situation that the current QVector<QVector<doubles>> does not have enough capacity for the new 26th QVector then the entire QVector<QVector<doubles>> will be copied from the current memory location to a new memory location where enough memory is available to add the 26th QVector. So, even though I am adding only 500,000 doubles the entire operation will take much longer due to the movement of the previous 25 QVector, which are all 500,000 doubles each, to accommodate the new 26th QVector.

[Christian Ehrlicher]

Since it's a QVector/std::vector and not a plain C array your assumption is wrong. QVector stores it's data on the heap so it does not matter if you store a pointer (= another indirection) or as object.

[CJha]

@Christian-Ehrlicher Ok, I understand storing a pointer or an object directly will result in approximately the same time taken.

According to Qt docs

If the new size is larger than the old size, QVector might need to reallocate the whole vector.

From this what I understand is that when a QVector<QVector<T>> is out of memory the entire thing will be reallocated. How will the reallocation happen when adding more QVector<T> to a QVector<QVector<T>> which is already at its capacity?

[Christian Ehrlicher]

Like you explained, but since the vector holds it's data on the heap, only the pointers need to be copied.

[CJha]

@Christian-Ehrlicher Thanks! That saves a lot of trouble for me :)