Question about the futures (and concurrency)
-
Hello everybody,
I'm developing an application which receive a flow of data in input, process them and then call a function which send the results to an endpoint over the net.
This last call is quite expensive and slow down the entire process, so I'm wondering if I could execute it using the concurrent framework and futures.
My doubt is that the number of futures will grow up during execution (there's some kind of guard?), because the producer is faster than the consumer.
Do you know is there a similar solution or a better way to avoid these problems?Thank you.
-
Hello everybody,
I'm developing an application which receive a flow of data in input, process them and then call a function which send the results to an endpoint over the net.
This last call is quite expensive and slow down the entire process, so I'm wondering if I could execute it using the concurrent framework and futures.
My doubt is that the number of futures will grow up during execution (there's some kind of guard?), because the producer is faster than the consumer.
Do you know is there a similar solution or a better way to avoid these problems?Thank you.
@Merlino said in Question about the futures (and concurrency):
I'm wondering if I could execute it using the concurrent framework and futures.
Sending data over the network using QNetworkAccessManager is already asynchronous and concurrent. It performs up to 6 network requests in parallel.
the producer is faster than the consumer.
Can you batch the data (upload multiple results in a single network request)? Or compress your data before uploading? These techniques should speed up your upload.
-
Thank you for your answer. Unfortunately the function I call to send the data is a custom API and I don't know how it manages the data after call. I already pack multiple results in block, but the call is still slow.
@Merlino said in Question about the futures (and concurrency):
the function I call to send the data is a custom API and I don't know how it manages the data after call.
So it doesn't use QNetworkAccessManager? Then check with the API's documentation or its developers to see if the function can be called concurrently.
I already pack multiple results in block, but the call is still slow.
Another options is to decimate your data if that is acceptable (For example, throw away every second result)
