Need more help with C++ code syntax - QTConncurent and template

[Pl45m4]

@AnneRanch said in Need more help with C++ code syntax - QTConncurent and template:

The "run" function works as expected, however , the supporting QConcurrent " 1 seconds ticks" needs work.

Yes, because now you cant use your changed futureWatcher the same way as used by the Qt Example.
(spin function doesn't return a QStringList vector)

[Anonymous_Banned275]

@Pl45m4
Yes, I realized that. But I hacked it , at lest for now , by defining another
QFutureWatcher

   QFutureWatcher<QVector<QStringList> > futureWatcher_run;
    // 1 seconds tick 
    // keep curent SIGNAL / SLOT
    QFutureWatcher<void> futureWatcher;   // keep for  SIGNAL etc 1 second ticks 

Now as I suspected - I have lost the "concurrency" - the "run" , as coded , blocks process and I have no "concurrent" 1 second ticks running in QProgressDialog.
More fun work.... perhaps just checking for "result" AFTER the "concurrent " is done will work. QProgressDialog sits on top on the area which eventually will show the results , so no hurry until it is gone from view.

[Anonymous_Banned275]

I need to discuss / clarify ONE more "problem" .

I hope I can explain "the problem" using my interpretations of Qt documentation. It is somewhat dupe of what I have said / posted before.

My "primary objective" was to utilize / write multi threaded code to account for a function which takes abut 10 seconds to execute .
I am using "QtConcurrent "framework " which does support multi threading .
I do "QtConcurrent::run" to run time consuming function in ONE thread AND futureWatcher.setFuture(QtConcurrent::map(vector, spin)); in additional FOUR threads.
As is - It all works as expected.

The "time consuming function" retrieves data for further processing.
I have a lengthy discussion elsewhere about how to pass / retrieve such data for such processing and it DOES work using following code:

QVector<QStringList> returnValue = futureWatcher_run.future().result();

The problem is , as far as I can identify it - the futureWatcher_run.future().result() no longer "runs " as multi threaded / QtConcurrent framework - it appears to run in main thread and is blocking.

Am I wrong ?

I do not have code implemented to actually see / debug the thread execution of the above code.

I do realize my problem is harder to visualize without full code, but how can I verify / change the "returnValue" code to make sure it runs in QtConcurrent framework and in its OWN thread , and not to be blocking?

    QStringList *stringList = new  QStringList(); // temporary pass QStringList
    QElapsedTimer timerHCI;
    timerHCI.start();

    futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList));
    
    
    // test returnValue
    int returnValueIndex = 0;
    int returnValueSize = 0;
    
    qDebug()<< "futureWatcher.setFuture(QtConcurrent::run(HCI, "
               "&HCI_Scan_Dialog::HCI_Scan_Scan,stringList)) elapsed time " <<  timerHCI.elapsed() ;
    
#ifdef BYPASS
    // takes real time
    QVector<QStringList> returnValue = futureWatcher_run.future().result();

[J.Hilk]

@AnneRanch I' don't quite understand your situation, why do you think its not running in its own thread?

Any way, if you want to, take a look at my git repo
https://github.com/DeiVadder/QtThreadExample

In that I have a project that covers all Qt ways of parallel processing, Related functions are number form 1 to 5.

Qt Concurrent is the number 4

[Anonymous_Banned275]

@J-Hilk said in Need more help with C++ code syntax - QTConncurent and template:

@AnneRanch I' don't quite understand your situation, why do you think its not running in its own thread?

Actually - that is silly for me to say.
Of course it is running in its own thread, problem is it is NOT QtConcurrent - one of he expected multi threads .
But I cannot simply prove it - the "debug" is getting too complicated.
I know for sure it is blocking - it is causing the main window to "go gray".
That is from my initial experience - when I was not running QtConcurrent correctly - the main window would turn gray at completion of the function.

I'll take a look at your code.
thanks.

Any way, if you want to, take a look at my git repo
https://github.com/DeiVadder/QtThreadExample

In that I have a project that covers all Qt ways of parallel processing, Related functions are number form 1 to 5.

Qt Concurrent is the number 4

[JKSH]

@AnneRanch said in Need more help with C++ code syntax - QTConncurent and template:

Appreciate all the help , it has been revealing how things can get complex if one does not pay attentions to ALL PARTS of the code.

You're welcome. Indeed, complexity tends to grow exponentially.

futureWatcher.setFuture(QtConcurrent::map(vector, spin));

You started this thread by asking about QtConcurrent::run(), and now you're adding QtConcurrent::map() to the mix. These two functions are very different.

Let's step back a bit. First, please describe in your own words:

• What does QtConcurrent::run() do? What does it return?
• What does QtConcurrent::map() do? What does it return?

perhaps just checking for "result" AFTER the "concurrent " is done will work.

Yes, you should do this.

The reason for the "block" is documented at https://doc.qt.io/qt-5/qfuture.html#result

[Anonymous_Banned275]

@JKSH said in Need more help with C++ code syntax - QTConncurent and template:

• What does QtConcurrent::run() do? What does it return?

I "runs" time consuming function " which scans (HCI inquiry) for nearby bluetooth devices and returns their address and name.

• What does QtConcurrent::map() do? What does it return?

It "runs" QProgressDialog and updates its "progress bar" in roughly 1 second intervals . The interval is not important.

Thanks to QConcurrent these functions run in multiple threads , hence in 'parallel fashion " as far as the user is concerned.

perhaps just checking for "result" AFTER the "concurrent " is done will work.

Yes, you should do this.

The reason for the "block" is documented at https://doc.qt.io/qt-5/qfuture.html#result

I'll check that.

In the mean time - for entertainment purposes - here is somewhat sanitized version of basic code - the "result" code is still under construction.

{
# ifdef TRACE
    qDebug() << "QDEBUG TRACE START initialize time consuming task";
    qDebug() << "file     " << __FILE__;
    qDebug() << "function "<<__FUNCTION__;
    qDebug() << "@line    " << __LINE__;
    //qDebug()<<"TEMPORARY EXIT ";
    //   return;#ifdef BYPASS
#endif
    qDebug() << " SETUP         declare / define (?) consuming function QFutureWatcher<QVector<QStringList> > futureWatcher_run;";
    QFutureWatcher<QVector<QStringList> > futureWatcher_run;
    qDebug() << " SETUP         declare / define (?) QFutureWatcher<void> futureWatcher;1 second ticks";
    QFutureWatcher<void> futureWatcher;   // keep for  SIGNAL etc 1 second ticks


    qDebug()  << "SETUP         # of iterations as vector array ";
    QVector<int> vector;
    //NOTE for now - # of iterations 3*4* estimated number of devices
    //TODO something smarter - use "finished" SIGNAL
    int iterations = 35;               // temp MN
    for (int i = 0; i < iterations; ++i)
    {
        // first vector cSTART initialize time consuming taskontains time consuming function - no delay
        // rest of them  - single delay time (1 S)
    #ifdef PROCESS
        //qDebug()<<" interations loop ?? index " << i;
        qDebug()<<" vector.append(i) index " << i; // LOOP ";
        // another delay "iterations
        //qDebug()<<" another delay iterations (?) ";
    #endif
        vector.append(i);
    }

    qDebug()<<"SETUP            Create QProgressDialog dialog - set paramaters ";
    // this is a default QProgressDialog constructor
    QProgressDialog dialog;
    //    	QProgressDialog(const QString & labelText,
    //    const QString & cancelButtonText, int minimum, int maximum,
    //            QWidget * parent = 0, Qt::WindowFlags f = 0)
    #ifdef BYPASS
    QProgressDialog dialog ( "TEST window title   ",
                             " BUTTON text", 0, iterations ,0 ,0);
    //BUG  did not set to TEST window title  here
    // dialog.setWindowModality(Qt::WindowModal);
    #endif
    //     dialog.labelText(" what is label ?" ); no matching function ??
    dialog.setWindowTitle("TEST window title  ");
    dialog.setFixedWidth(500);                    //W.setWindowTitle("TEST window title  ");
    // set defaulrs  of what ???
    //    dialog.setMaximum(100);
    //    dialog.setRange(10, 1000 );
    //    If set to 0, the dialog is always shown as soon as any progress is set.
    //    The default is 4000 milliseconds.
    //    int	minimumDuration() const
    //    void	setMinimumDuration(int ms)
    dialog.setMinimumDuration(0);
    // sets delay time to ~ 5 seconds
    dialog.setRange(0,0);
    // single shot label only
    //dialog.show(); //This is done querying the number of processor cores

    //NOTE won;t work unitil what is done ???
    qDebug()<< "SETUP        test timer";
    QElapsedTimer timer;
    timer.start();
#ifdef BYPASS
    //TODO later
    dialog.setLabelText(QString("Progressing using %1 thread(s)...\nTODO Elapsed time %2  [mS] ")
                        .arg(QThread::idealThreadCount()).arg(timer.elapsed()));
    dialog.show();          // test show
#endif

    qDebug()<<"SETUP           ALL connect function futureWatcher and dialog connect SIGNAL  SLOT ";
    // # of iterations ?? " << iterations ; // vector.append(i) LOOP ";

    qDebug()<<"SETUP           setup future WatcherSIGNAL(finished()";
    // Resets the progress dialog. The progress dialog becomes hidden if autoClose() is true.
    //QObject::connect(&futureWatcher, SIGNAL(finished()), &dialog, SLOT(reset())); // reset to what ??


    qDebug()<<"SETUP           setup dialog, SIGNAL(canceled()";
    //QObject::connect(&dialog, SIGNAL(canceled()), &futureWatcher, SLOT(cancel())); // TOK

    // how to emulate these ??
    QObject::connect(&futureWatcher, SIGNAL(progressRangeChanged(int,int)), &dialog, SLOT(setRange(int,int)));
    QObject::connect(&futureWatcher, SIGNAL(progressValueChanged(int)), &dialog, SLOT(setValue(int)));


    // add tracking  slot
    //QObject::connect(&futureWatcher, SIGNAL(progressValueChanged(int)), this , SLOT(setValue(int)));




    // create test SLOT to monitor progressValueChanged(int)
    qDebug() << "SETUP        ! create test SLOT to monitor progressValueChanged(int)";
    QObject::connect(&futureWatcher, SIGNAL(progressValueChanged(int)), this, SLOT(on_doTaskButton_2_clicked()));

    //   QObject::connect(&futureWatcher, SIGNAL(progressValueChanged(int)),this, SLOT(on_doTaskButton_3_clicked()));
    QObject::connect(&futureWatcher, SIGNAL(progressRangeChanged(int,int)),this, SLOT(on_doTaskButton_3_clicked()));



    // create test SLOT to monitor progressValueChanged(int)
    qDebug() << "SETUP         create test SLOT to monitor progressValueChanged(int)";
    QObject::connect(&futureWatcher, SIGNAL(progressValueChanged(int)), this,
                     SLOT(on_doTaskButton_2_clicked())); // &dialog.getValue())));





    qDebug() << "SETUP          HCI_Scan_Dialog *HCI = new HCI_Scan_Dialog () with HCI_Scan_Scan  function ";
    HCI_Scan_Dialog *HCI = new HCI_Scan_Dialog ();



    // run HCI_Scan_scan with   QStringList   array as parameter

    qDebug() << "RUN            futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList));";
    QStringList *stringList = new  QStringList(); // temporary pass empty QStringList
    QElapsedTimer timerHCI;
    timerHCI.start();
    // takes no real time - but HCI_Scan_scan does !
    futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList));
    qDebug() << "CZECH          futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList));";
    qDebug() << "CZECH          futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList)) elapsed time "
          <<  timerHCI.elapsed() << " mS" ;

    // NOTE  no progress dialog here , no main window gray either
    // TODO get real thread ID here

    qDebug() << "RUN            futureWatcher.setFuture(QtConcurrent::map(vector, spin));";
    futureWatcher.setFuture(QtConcurrent::map(vector, spin));

The reason for the "block" is documented at https://doc.qt.io/qt-5/qfuture.html#result
    // test real duration of dialog here

    qDebug() << "RUN            dialog.exec(); // run modal dialog;";
    QElapsedTimer timer_wait;
    timer_wait.start();
    //     futureWatcher.progressValue()
    qDebug()<<"TODO   ??  futureWatcher.progressValue() " << futureWatcher.progressValue();  // what is this for ??
    dialog.exec(); // run modal dialog
    qDebug()<< "futureWatcher.setFuture(QtConcurrent::run(HCI, "
               "&HCI_Scan_Dialog::HCI_Scan_Scan,stringList)) elapsed time " <<  timer_wait.elapsed() ;

}

PLEASE NOTE

The reason for the "block" is documented at https://doc.qt.io/qt-5/qfuture.html#result

Unfortunately to mitigate this "blocking / waiting for ANY ( first?) result " , which in my case takes 10 seconds - the "result" code , as suggested , cannot be simply placed willy-nilly .

It is back to "emit" SIGNAL or wait until "finished" SIGNAL is emitted.
I am already setup to emit "single device found" SIGNAL from the function itself, not from the "QConcurretn code ".

It will take some care and time to add all the necessary emits , SIGNAL SLOT....

Thanks for all the forum help I am making some (slow) progress.

[Anonymous_Banned275]

Posting this has given me an idea to take a different approach

    What does QtConcurrent::run() do? What does it return?

I "runs" time consuming function " which scans (HCI inquiry) for nearby bluetooth devices and returns their address and name.

    What does QtConcurrent::map() do? What does it return?

It "runs" QProgressDialog and updates its "progress bar" in roughly 1 second intervals . The interval is not important.

Naturally - I have been coding these (two main ) tasks sequentially , with emphasis on the QtConncurent " functions" - mainly what they do. What I have missed is "block" approach which is little different that just " function returns..." when coding in event driven environment.

I will try to rebuild the code using this "block" philosophy , maybe it will make the code interaction easier to see.
Here is my goal in pseudo-code

{// QtConcurrent block START 
         {#  QtConcurrent::run() block 
         code 
        {#QtConcurrent::run()    block 
         {#   QtConcurrent::map() block 
                   {#   setup  spin vector array  block 
                code 
    ..           {#   setup  spin vector array  block  
              code 
        {# QtConcurrent::map()   block 
     code 

}// QtConcurrent block END 


Of course that still does not help to figure out how to process "the time consuming function results / return". 
What is puzzling - QtConcurennt  is multi threading  framework and by (my) definition one of the advantages of multi threading is NOT to have blocking processes. 
So why is "result" blocking  ? 
(I still cannot prove it, I have no "get actual thread ID" code , but it appears to   "run" in main process thread and if it blocks that thread - no good. ) 
Is it not part of the QtConcurrent framework ?

[Pl45m4]

@AnneRanch

result() is blocking the main thread, because you want to use the result there. If the result is not ready, it would not make any sense to continue with the code. Every use of result data would be invalid. So it waits until there is a result, then, for example assigns it to your local variable and continues.
Same as waitForFinished(), with the only difference that you don't care about the returned values, when using this.

The calculation of your result happens in multiple threads concurrently, but the calling thread (your main thread), where you want to use the result, has to wait.

[Anonymous_Banned275]

@Pl45m4 said in Need more help with C++ code syntax - QTConncurent and template:

@AnneRanch

result() is blocking the main thread, because you want to use the result there. If the result is not ready, it would not make any sense to continue with the code. Every use of result data would be invalid. So it waits until there is a result, then, for example assigns it to your local variable and continues.
Same as waitForFinished(), with the only difference that you don't care about the returned values, when using this.

The calculation of your result happens in multiple threads concurrently, but the calling thread (your main thread), where you want to use the result, has to wait.

I do understand that, however , where is the advantage of QtConcurrent then?

Either way , I still do not see "the connection / advantage " between plain function "return" , or processing the pointer passed to the function and this "result" .

Irregardless which way I access the data - it is not what I expect.
I need to "code it" differently.

I am already using the "map" letting "connect" do simple updating of progress bar, so why not utilize same method to keep "run" from screwing things up "waiting" for result ? I am using wrong method and in wrong place to boot.

The way I have it now - the "spin" is finished first and when it is finished - I should be able to see if "run" is also finished and then get "result" without any blocking.

[JKSH]

@AnneRanch said in Need more help with C++ code syntax - QTConncurent and template:

I am already using the "map" letting "connect" do simple updating of progress bar, so why not utilize same method to keep "run" from screwing things up "waiting" for result ?

QtConcurrent::run() runs your time-consuming function in another thread. When it finishes, your QFutureWatcher emits the finished() signal. Connect this signal to a slot; fetch the result in that slot.

    What does QtConcurrent::map() do? What does it return?

It "runs" QProgressDialog and updates its "progress bar" in roughly 1 second intervals . The interval is not important.

QtConcurrent::map() takes a function and a vector that contains N elements. It runs that function N times -- once per vector element. On a modern CPU, multiple copes of that function can run concurrently (in parallel).

[Anonymous_Banned275]

I am hoping this is my LAST post in this thread.
Even after I double check completion of the "time consuming function"
the "returnValue" takes over 1 second to complete.

First returnValue   QVector<QStringList> = futureWatcher_run.future().result() 
 elapsed time  1533 [mS] 

And I really do not want to know why...

Cheers

EDIT / SOLVED

It takes 1.5 seconds to output the DEBUG message ! DUH !

       QElapsedTimer  timerHCI;
        timerHCI.start();

        QVector<QStringList> returnValue = futureWatcher_run.future().result();

       // should take no significant time
        qDebug()<< "time " <<  timerHCI.elapsed() ;  

       now the time = ZERO !

[JonB]

@AnneRanch
It will not take 1.5 seconds to execute qDebug()<< "time " << timerHCI.elapsed(). However, it may well take 1.5 seconds to get the return result back from futureWatcher_run.future().result(), if that's how long it takes to complete all of the threads.

[SimonSchroeder]

@AnneRanch said in Need more help with C++ code syntax - QTConncurent and template:

I do understand that, however , where is the advantage of QtConcurrent then?

There are two different use cases of QtConcurrent: 1) Similar to what you are doing you can launch multiple tasks in separate runs one after the other and only after all the runs call result(). This would mean that you can compute several tasks in parallel, but it also means that you have a synchronization point when you want to access the results. Furthermore, if you run a whole bunch of tasks using QtConcurrent it will use a thread pool in the background such that your CPU cores are not oversubscribed. This is the main advantage over QThread. 2) The other use case, as already meantioned, is to only call run in your function and connect the finished signals of the QFutureWatcher to a separate slot which will continue from where you want to access the results. Basically, this means that you need to split your function right there where you call result() (and you don't have to call result anymore because it will be the parameter of your function).

Furthermore, map and mapReduce automatically parallelize and call your function on each element. This is the really helpful part of QtConcurrent.

One quick fix to your problem (I haven't made up my mind if this is good or bad design) would be to run your existing function inside a separate thread. Then your separate thread would block instead of the main thread. However, this only works if you don't do any GUI stuff inside this thread. This would work something like this:

QThread::create([=]()
{
     // your whole function code here...
    ...
    futureWatcher_run.setFuture(QtConcurrent::run(HCI, &HCI_Scan_Dialog::HCI_Scan_Scan,stringList));
    ....
    QVector<QStringList> returnValue = futureWatcher_run.future().result(); // now this will block
})->start();

[Anonymous_Banned275]

Thanks for the post.

I seldom RTFM first, but this time I have made an exception.
I have chosen QtConcurrent because it is advertised as to "simplify multi threading". In principle that is what I am comfortable with, and it does the job.

To be clear - as an old fart - I believe there is "more than one way to skin a cat " , so I do not object to alternatives.

I actually got "impressed" by a sample using QProgressDialog and somehow lost my focus on the " time consuming function ".
My " time consuming function " provides intermediate data / result .
So I get intermediate SIGINAL(s) before I get the final "finished".

The MAIN issue was that QtConcurrent automatically utilizes all available CPU cores - so instead of updating the QProgressDialog in 1 seconds intervals it was four times faster....
That "bug" was not that easy to find...even after RTFM again.

Technically - I never bothered to check the actual "run:" threads.
I am happy as long as it is "non blocking".

I am sure QThread would do the job, but why fix it if it is not broken?

[JKSH]

@AnneRanch said in Need more help with C++ code syntax - QTConncurent and template:

The MAIN issue was that QtConcurrent automatically utilizes all available CPU cores - so instead of updating the QProgressDialog in 1 seconds intervals it was four times faster....
That "bug" was not that easy to find...even after RTFM again.

QtConcurrent's raison d'être is to complete tasks as quickly as possible by performing sub-tasks in parallel -- that's why it's called "concurrent". So, QtConcurrent::map() will finish faster when you have more cores and faster CPUs. It is intentionally designed this way.

If you want to update QProgressDialog at fixed intervals, QtConcurrent::map() and QThread are not suitable tools for the job. Instead, you want QTimer.

[Anonymous_Banned275]

@JKSH
Let me clear this - I have no objection to QConcurrent mode of operation.
I generally stick with a task , my - primary directive / objective - in this case multi threading / non blocking due to time consuming function.
It does not matter , to me , how many threads are being used.
Hence the primary task was to have non-blocking operation, secondary ( as I call "bells and whistles" ) some rough indication where is the "time consuming function" . Updating the status at 1 seconds intervals seems adequate.

Missing the "note" about "automatic CPU core selection" WAS MY MISTAKE.

I'll leave configuring QConcurrent to run only ONE CPU to the audience...

Cheers