What is best practice for QNetworkAccessManager for high frequency POST requests to API
-
I have been using a single instance of QNAM to connect to an API server endpoint and transmit HTTPS POST requests.
It works nicely but sometimes it is required to transmit a burst of messages just a few milliseconds apart. As the server takes around 50 milliseconds to process requests I frequently encounter problems where my requests are queued due to there only being 6 available threads/connections to the server managed by the QNAM (as far as I understand based on my reading docs and other posts here).
I am clearly able to detect this lag in transmissons by logging request send times (by logging timestamp of return of the QNAM async write() call) and comparing with times the response is received.
My question is really about how I would go about a solution based on the above use case. I can really only think of the following:
- Use multiple instances of QNAM and make each connect to the same server endpoint and share the transmission load across these instances (not sure if this is OK as I have seen many posts suggest that only one QNAM instance is "advised" (this is my current use case))
- Use QNAM instances each with their own execution thread and use QNAM with a synchronous mode (I am not sure if QNAM has a synchronous operation mode)
- Give up and try something like boost::asio and multi-threading.
Any suggestions of a stable way forward would be very welcome. Without a solution I will likely have to abandon Qt as my development environment completely.
Similarly, correction of any misunderstanding I have about how QNAM works internally would be welcome.
-
I have been using a single instance of QNAM to connect to an API server endpoint and transmit HTTPS POST requests.
It works nicely but sometimes it is required to transmit a burst of messages just a few milliseconds apart. As the server takes around 50 milliseconds to process requests I frequently encounter problems where my requests are queued due to there only being 6 available threads/connections to the server managed by the QNAM (as far as I understand based on my reading docs and other posts here).
I am clearly able to detect this lag in transmissons by logging request send times (by logging timestamp of return of the QNAM async write() call) and comparing with times the response is received.
My question is really about how I would go about a solution based on the above use case. I can really only think of the following:
- Use multiple instances of QNAM and make each connect to the same server endpoint and share the transmission load across these instances (not sure if this is OK as I have seen many posts suggest that only one QNAM instance is "advised" (this is my current use case))
- Use QNAM instances each with their own execution thread and use QNAM with a synchronous mode (I am not sure if QNAM has a synchronous operation mode)
- Give up and try something like boost::asio and multi-threading.
Any suggestions of a stable way forward would be very welcome. Without a solution I will likely have to abandon Qt as my development environment completely.
Similarly, correction of any misunderstanding I have about how QNAM works internally would be welcome.
@nerdman
The limitation mentioned relates to one host/port combination in http. That means 6 requests to the same host and port will be parallelized.sometimes it is required to transmit a burst of messages just a few milliseconds apart.
An unusual requirement, to say the least. If "bursts" of parallel POST-requests are needed to achieve a well-meaning goal, maybe improve API design.
Without a solution I will likely have to abandon Qt as my development environment completely
Alarm going off at this very point. If the impossibility to send "bursts" of POST requests within milliseconds is a knock out criterion: Planning a DOS attack?
-
I have been using a single instance of QNAM to connect to an API server endpoint and transmit HTTPS POST requests.
It works nicely but sometimes it is required to transmit a burst of messages just a few milliseconds apart. As the server takes around 50 milliseconds to process requests I frequently encounter problems where my requests are queued due to there only being 6 available threads/connections to the server managed by the QNAM (as far as I understand based on my reading docs and other posts here).
I am clearly able to detect this lag in transmissons by logging request send times (by logging timestamp of return of the QNAM async write() call) and comparing with times the response is received.
My question is really about how I would go about a solution based on the above use case. I can really only think of the following:
- Use multiple instances of QNAM and make each connect to the same server endpoint and share the transmission load across these instances (not sure if this is OK as I have seen many posts suggest that only one QNAM instance is "advised" (this is my current use case))
- Use QNAM instances each with their own execution thread and use QNAM with a synchronous mode (I am not sure if QNAM has a synchronous operation mode)
- Give up and try something like boost::asio and multi-threading.
Any suggestions of a stable way forward would be very welcome. Without a solution I will likely have to abandon Qt as my development environment completely.
Similarly, correction of any misunderstanding I have about how QNAM works internally would be welcome.
@nerdman While I agree with @Axel-Spoerl, that the design needs a rethink, another option is to have the receiver open multiple ports and send your POST requests to these in round robin. The 6 connection limit is per host and port combination.
The way you describe it, your 50ms measure includes the transmission latency and server processing time. This is not all in your program's control. The time taken to establish a TCP connection, encode and send a HTTP POST request (longer for HTTPS), and get a response is likely to take more than a "couple of milliseconds" regardless of what is used to program it. If it works at the moment it is likely because TCP connections are pooled and reused by QNAM. Change the API to send blocks of data less often, use something other than HTTP, consider whether you really need to send this data at all, ... all sorts of options
-
@nerdman
The limitation mentioned relates to one host/port combination in http. That means 6 requests to the same host and port will be parallelized.sometimes it is required to transmit a burst of messages just a few milliseconds apart.
An unusual requirement, to say the least. If "bursts" of parallel POST-requests are needed to achieve a well-meaning goal, maybe improve API design.
Without a solution I will likely have to abandon Qt as my development environment completely
Alarm going off at this very point. If the impossibility to send "bursts" of POST requests within milliseconds is a knock out criterion: Planning a DOS attack?
@Axel-Spoerl Nothing remotely like what you are suggesting (I'm actually a little offended by the suggestion to be honest). It's a legitimate usage of the target API and I thought my question was reasonable and I'd not considered my requirement as "unusual" at all. But thanks for taking the time to reply anyway.
-
@nerdman While I agree with @Axel-Spoerl, that the design needs a rethink, another option is to have the receiver open multiple ports and send your POST requests to these in round robin. The 6 connection limit is per host and port combination.
The way you describe it, your 50ms measure includes the transmission latency and server processing time. This is not all in your program's control. The time taken to establish a TCP connection, encode and send a HTTP POST request (longer for HTTPS), and get a response is likely to take more than a "couple of milliseconds" regardless of what is used to program it. If it works at the moment it is likely because TCP connections are pooled and reused by QNAM. Change the API to send blocks of data less often, use something other than HTTP, consider whether you really need to send this data at all, ... all sorts of options
@ChrisW67 Yes I think the pooling of connections is precisely what is happening because I experience larger lag on my first requests (presumably due to the SSL handshaking process). The problem is simply that I end up having my requests queued until one of them gets freed up. In general it works fine, but requests need sending in response to external data which is unpredictable so sending data less often is not really an option. The vast majority of the time the application is completely idle, but I'll occasionally need to make 20+ requests in a second and thats when I start experiencing the lagging. Thanks for your response anyway.
-
@ChrisW67 Yes I think the pooling of connections is precisely what is happening because I experience larger lag on my first requests (presumably due to the SSL handshaking process). The problem is simply that I end up having my requests queued until one of them gets freed up. In general it works fine, but requests need sending in response to external data which is unpredictable so sending data less often is not really an option. The vast majority of the time the application is completely idle, but I'll occasionally need to make 20+ requests in a second and thats when I start experiencing the lagging. Thanks for your response anyway.
From my pov the used transfer type is not suitable for the task. Are you sure the endpoint can really handle this the way you think?
-
From my pov the used transfer type is not suitable for the task. Are you sure the endpoint can really handle this the way you think?
@Christian-Ehrlicher Yes I know that many people do exactly as I am trying to do, but most of them use python with thread pools and blocking sockets to achieve the same thing during a period of high activity. While the application are is somewhat niche, it is by no means uncommon and the API servers are easily able to deal with the scenarios mentioned and the providers of the API in no way see what we do as "unusual" or malicious in any way. It does seem perhaps that I just need to port my code or write my own HTTPS client thread pools with OpenSSL and pthreads.
-
@Christian-Ehrlicher Yes I know that many people do exactly as I am trying to do, but most of them use python with thread pools and blocking sockets to achieve the same thing during a period of high activity. While the application are is somewhat niche, it is by no means uncommon and the API servers are easily able to deal with the scenarios mentioned and the providers of the API in no way see what we do as "unusual" or malicious in any way. It does seem perhaps that I just need to port my code or write my own HTTPS client thread pools with OpenSSL and pthreads.
@nerdman hi, in that case you might want to check the QThreadPool and QSslSocket classes as indeed, you won't be able to achieve what you currently want with QNAM as is.
-
@nerdman hi, in that case you might want to check the QThreadPool and QSslSocket classes as indeed, you won't be able to achieve what you currently want with QNAM as is.
-
@nerdman hi, in that case you might want to check the QThreadPool and QSslSocket classes as indeed, you won't be able to achieve what you currently want with QNAM as is.
