Simple big-looped program break down

[SGaist]

If it's an array of objects, yes, everything is destroyed. If it's an array of pointer, it's your duty to first cleanup.

As for passing stuff around, it will depend on how you pass them around (by value, reference, const reference, pointer).

If passing by value, it's indeed a copy that is done.

[Please_Help_me_D]

@SGaist thank you! I just made some experiments
Step by step I see the things you explain :)

[JonB]

@Please_Help_me_D
For arrays: in the C family of languages, and most "general" programming languages, arrays are passed by reference, not by value.

This if you declare a simple int fred and pass to a function func(int param), the value in fred is copied into param (and changing it inside the function has no effect on the caller's fred).

OTOH, if you declare an array int fred[3] and pass to a function func(int *param) the array is passed by reference/pointer, i.e. C passes the address of the start of the array (and changing values at *param or param[2] does change what is in the fred array back in the caller). The obvious reason is that arrays can be large, so you don't want to have to copy them unless you have to.

So in your example, when you pass your array (int[3]) to another function it does not "take twice the space in RAM". The only thing that takes more space is the pointer to the array in the receiving function, which is a fixed 4 or 8 bytes (32-/64-bit) regardless of whether the array itself occupies 400MB of RAM.

Meanwhile, going back to your timings. I am surprised if the time to fill your QVector is as large as 10 seconds. I don't do C++ so I can't check. I was going to suggest trying a.resize() or a.reserve() before the loop, but I think the constructor has already done that. One possibility is that (I believe) QVector will do bounds checking each time on your a[i], which the pure C++ example above it will not do, and that will cost some. Instead of indexing by i, try using https://doc.qt.io/qt-5/qvector.html#begin etc. to do the fill via iterators and see if that is quicker? The other possibility is that QVector data is implicitly shared (https://doc.qt.io/qt-5/implicit-sharing.html). The Qt experts here may be able to indicate whether by changing its content in a loop there is an overhead in dealing with the shared storage area, I don't know.

[J.Hilk]

@Please_Help_me_D

I did some test on my own, because its an interesting topic:

#include <iostream>
#include <QVector>
#include <vector>
#include <numeric>
#include <QElapsedTimer>
int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QElapsedTimer et;
    qint64 t1, t2, t3, t4;

    int *pa = new int [100000000];
    et.start();
    for (int i = 0; i < 100000000; i++) {
        pa[i] = i;
    }
    t1 = et.elapsed();

    QVector<int> qva(100000000);
    et.restart();
    for (int i = 0; i < 100000000; i++) {
        qva[i] = i;
    }

    t2 = et.restart();

    std::iota(qva.begin(), qva.end(), 0);

    t3 = et.restart();

    int j(0);
    for( int & i : qva){
        i = j++;
    }

    t4 = et.elapsed();

    qDebug() << "Times:";
    qDebug() << t1 << t2 << t3 << t4;
}

The results are:
Debug build: 456 3889 224 214
Release build: 0 86 58 45

And that's to be expected- Vector is a complex class and therefore lot of debug information /checks, stuff that will slow it down.

I'm however surprised, that in release the good old C loop is so fast. Maybe the compiler optimizes a lot and the loop is not executed at all 🤔

In release, all methods are pretty close, and the iterator methods , not surprisingly, a good but faster than the access method

[JonB]

@J-Hilk said in Simple big-looped program break down:

Maybe the compiler optimizes a lot and the loop is not executed at all 🤔

If he had been filling the elements with a constant value that would have been possible. But because he/you is filling with changing i each time, I can't see any optimization of that is possible, it will have to loop! You could look at the disassembly... :)

[J.Hilk]

@JonB I did, even with only level 1 optimization, the loop is removed 🤷‍♂️

smart things these compilers 😉

[JonB]

@J-Hilk
Noooooo!!!

xor eax, eax
ret

Those two lines just load eax with 0 (quicker than literal "load with 0") and return it as result --- it's an implicit return 0 at the end of main()!

Believe me, somewhere there above it is the code for the loop :) Otherwise, if the compiler has really decided there are no side-effects because pa is never referenced so it will go on strike and do nothing, put a return pa[10] or something after your loop.

[J.Hilk]

@JonB actually, when I change the compiler to MSVC, the loop will never be removed, even with O3
🤨 Windows, am I right!?

[JonB]

@J-Hilk
Yep, that's more like it, good old MSVC! As I said, retry gcc with return pa[10] after the loop and then see?

[J.Hilk]

@JonB said in Simple big-looped program break down:

return pa[10]

you're right by returning pa[10] from main, the release timings look much more like I would have expected:

211 84 52 52

[Please_Help_me_D]

@JonB I'm trying to understand how to adjust QVectror::iterator to assign values to an array. I did the following:

    QVector<int> a = {10, 20, 30, 40, 50};
    QVector<int>::iterator it = a.begin();

    clock_t start_time =  clock();
    while (it != a.end())
    {
        cout << *it << endl;
        it++;
    }

but that just showed me what is the iterator. How to modify the code to fill an empty QVector with numbers?

[JonB]

@Please_Help_me_D said in Simple big-looped program break down:

    int i = 0;
    while (it != a.end())
    {
        *it = i++;
        it++;
    }

[Please_Help_me_D]

@JonB So the execution time for the code:

    QVector<int> a(100000000);
    QVector<int>::iterator it = a.begin();
    int i = 0;

    clock_t start_time =  clock(); // начальное время
    while (it != a.end())
    {
        *it = i++;
        it++;
    }
    clock_t end_time = clock(); // конечное время
    clock_t d_time = end_time - start_time;
    cout << d_time << endl;

is 8-9 seconds
By the way, the same operation in Matlab with single (float) precision accuracy takes 0.3 seconds. That means that Matlab loops not so slow as I thought:

// Matlab code
a = single(zeros(100000000,1)); // preallocate the memory
tic // start time
for n = 1:100000000
    a(n) = single(n); // assign avlue to each position
end
toc // end time

[J.Hilk]

@Please_Help_me_D
If anything, than you should take away from my test, that there's a difference between release and debug builds in c++

build and run your test in release, it will drop way below 1 sec

[JonB]

@Please_Help_me_D
Please do as @J-Hilk has said before, if you're compiling or running for debug there will be a vast difference from release/optimized

Also separately, verify what the MATLAB does in your code with

for n = 1:100000001

[Please_Help_me_D]

@J-Hilk sorry I forgot that!

    int *pa = new int [100000000];
    clock_t start_time =  clock(); 
    for (int i = 0; i < 100000000; i++)
    {
        pa[i] = i;
    }
    clock_t end_time = clock(); 
    clock_t d_time = end_time - start_time;
    cout << d_time << endl;

is 0.14 second

    QVector<int> a(100000000);

    clock_t start_time =  clock();
    for (int i = 0; i < 100000000; i++)
    {
        a[i] = i;
    }
    clock_t end_time = clock(); 
    clock_t d_time = end_time - start_time;
    cout << d_time << endl;

is also about 0.14 second

    QVector<int> a(100000000);
    QVector<int>::iterator it = a.begin(); /
    int i = 0;

    clock_t start_time =  clock(); 
    while (it != a.end())
    {
        *it = i++;
        it++;
    }
    clock_t end_time = clock(); 
    clock_t d_time = end_time - start_time; 
    cout << d_time << endl;

is also about 0.14 second
So Matlab loops is about twice slower :)
Thank you very much! It's good to know such difference in perfomance in Debug and Release mode

[Please_Help_me_D]

@JonB Matlab automatically increases the vector size by 1. But if I dont preallocate the size of the vector and with each iteration the vector size increases then it takes way much time. Here I do the iteration without preallocation and variable "a" is created when first iteration is performed:

// Matlab code
tic
for n = 1:100000000
       a(n) = single(n);
end
toc

35 seconds to preform

[JonB]

@Please_Help_me_D
For presumably similar slowness with QVector, try creating it empty with just QVector<int> a and use a.append(i) in the loop. I anticipate some slowness compared to pre-sizing, but (hopefully) not as bad as it could be:

This operation is relatively fast, because QVector typically allocates more memory than necessary, so it can grow without reallocating the entire vector each time.

[Please_Help_me_D]

@JonB very interesting result as for me:

     QVector<int> a;

     clock_t start_time =  clock();
     for (int i = 0; i < 100000000; i++)
     {
         a.append(i);
     }
     clock_t end_time = clock();
     clock_t d_time = end_time - start_time;
     cout << d_time << endl;

in Debug mode it takes 5.5-6 seconds
in Release mode it takes 0.7 second (while in Matlab as I previously said it takes 35 seconds)
So QVectror is really fast even if it works with <appending mode>
Thank you for this example!

[jsulm]

@Please_Help_me_D said in Simple big-looped program break down:

So QVectror is really fast even if it works with <appending mode>

As far as I know QVector allocates more memory as currently is need and on resizing it allocates again more than "current_size + 1".