Concurrent map equivalent
-
Hi guys, there's a class called
ThreadPoolExecutor
on Python that has a function calledmap
, that function accepts a method to be called, and iterables to be passed to the function, for example:def example(self, name, foo, bar): .... # 10 is the total of workers with ThreadPoolExecutor(10) as executor: executor.map( example, self.list_of_names, itertools.repeat(foo, len(self.list_of_names)), itertools.repeat(bar, len(self.list_of_names)) )
Imagine that
list_of_names
is a list of names, imagine thatfoo
andbar
has a constant value, for example,foo
=hello
andbar
=world
, we need an iterable to be passed to themap
function, then theitertools.repeat
will create an iterable repeating the same thing the same size oflist_of_names
. In the end,example
will have as argument each name fromlist_of_names
,foo
andbar
.I was wondering, how to simulate this on Qt? How to create a concurrent call of a method from a
QMap<QString, QString>
and also pass two additional QString as argument? -
Hi
Imagine that list_of_names is a list of names
Yes that totally works for me ;)
It sounds high level and nice java class,
not sure Qtt can match that 100% but it seems in function the same
as
http://doc.qt.io/qt-5/qtconcurrent-index.html
with
http://doc.qt.io/qt-5/qtconcurrent.html#mapBut I didnt see a way to add extra parameters :(
-
I tried that way, but it crashed:
QStringList names = { "john", "jane" }; QString extra = "doe"; auto example = [=] (const QString &name) { qDebug() << name << extra; }; QtConcurrent::map(names, example);
I thought about using
foreach
andQtConcurrent::run
but I will have to callwaitForFinished
.QStringList names = { "john", "jane" }; QString extra = "doe"; auto example = [=] (const QString &name, const QString &extra) { qDebug() << name << extra; }; foreach (QString name, names) { QFuture<void> result = QtConcurrent::run(example, name, extra); result.waitForFinished(); }
I don't know if the way I made can lead me to problems, a problem I can see already is: How can I pause them if it's a loop? For instance...
What you think about that?
-
Hmm that was odd. does look ok.
will names survive the scope?Well the foreach is not bad but dont
result.waitForFinished();block the GUI thread ?
-
@mrjj What you mean with names surviving the scope? I don't know what happens, it just crashes.
The second option seems to be more "the right way" cause I'm passing the values as argument not capturing it in a lambda, also I can create a method and call, the first option I can't call a method.
I tried this:auto example = [=] (const QString &name, const QString &extra) { for (int i = 0; i < 9999; ++i) { qDebug() << name << extra; } };
In the second option to see if it blocks the gui thread but I'm able to move it, I don't know if I'm testing the right way.
edit
I just realized that it does block the gui thread, I incremented the
9999
to9999999
and well... nothing shows.What can I do?
-
@Defohin said in Concurrent map equivalent:
What you mean with names surviving the scope?
He means this from the docs for
QtConcurrent::map
:Calls function once for each item in sequence. The function is passed a reference to the item, so that any modifications done to the item will appear in sequence.
Once you leave the scope of the function, the reference you'd passed is invalid and the program crashes. Perhaps you need
mapped()
instead? -
Hi
I think concurrent will be more fun as it dont block the GUI thread.
as in.void mapFunction(int& n) { qDebug() << n; } void test() { QList<int>* vectorOfInts = new QList<int>; for (int i = 0; i < 1000; i++) vectorOfInts->push_back(i); QFuture<void> future = QtConcurrent::map(*vectorOfInts, mapFunction); } // end of scope
-
@Defohin
Hi , i think you can cancel via
http://doc.qt.io/qt-5/qfuture.html#cancel
http://doc.qt.io/qt-5/qfuturewatcher.html#cancelI think you need to use QFutureWatcher if u dont use waitForFinished.
All credits to @kshegunov :) -
I don't want the pause, continue or the stop function, I just want to run a list of string over a method and pass additional arguments to it in parallel.
UsingQtConcurrent::map
won't work cause I can't pass additional arguments.
UsingQtConcurrent::run
won't work cause I'll need to callwaitForFinished
and it will block the GUI thread... (QFutureWatcher::cancel
orQFuture::cancel
won't work either cause it's not possible to cancel aQFuture
returned byQtConcurrent::run
)...What can I do? I'm out of ideas.
-
like this:
QStringList names = { "john", "jane" }; // global or member variable void YourClass::startJobs() { QString extra = "doe"; auto example = [extra] (QString &name) { qDebug() << name << extra; }; QFuture<void> future = QtConcurrent::map(names, example); }
-
@Defohin said in Concurrent map equivalent:
No need to cancel.
By cancel i mean to terminate before its finished. Like closing program.
In that regards you need "cancel" :) -
@Defohin said in Concurrent map equivalent:
Using QtConcurrent::map won't work cause I can't pass additional arguments.
Just use a function object
struct MapHelper{ MapHelper(){} QString m_additionalArgument1; QString m_additionalArgument2; typedef QString result_type; QString operator()(const QString &val) { return m_additionalArgument1+val+additionalArgument2; } } MapHelper helper; // TODO: make sure to manage its lifecycle helper.m_additionalArgument1 = "Prefix "; helper.m_additionalArgument2 = " Suffix"; QtConcurrent::map(list_of_names,helper);