Solved Multiple requests in the same method
-
I come from languages that are not events driven like Qt and sometimes when I want to create a kind of API wrapper or something like that I can create multiple requests in the same method and wait for their return in order to process the next one. I can request for login, request a page and download a file in the same method, waiting the other requests to finish before going forward to the next one. In Qt its a bit complicated for me as it uses signals and slots, so I was wondering, how can I create a method that can do multiple network actions?
-
Hi
http://doc.qt.io/qt-5/qnetworkaccessmanager.html#details
is already asynchronous so you can have up to 6 request out at same time.
You cannot in any good way use it synchronous so doing it all from same function will prove difficult.
So you better make a small handling class that controls the downloading and
first send off next request when the one before is finished if that is the goal. -
The question is about the possibility to make multiple requests at the same time but the requirement to make one after the other when its finished. For example:
I need to create a class that have a method
search
, in thatsearch
method I need to send a post request to a form, get the returned json (this is one request), with the json in hand, I need to parse that json and get a code, with that code I need to send another request to another url (this is the second request), with that second request I need to parse and return a string to the user. This in the same method.
The thing is, in order to return a string to the user I need to make two requests before that. In PHP, Python or other language I can just assign the return of the request to a variable and we are good to go, but In Qt its more complicated cause its events. -
Hi
So it should have downloaded all request before processing them ?
You need a queue or similar to handle that, so when each one comes in you put the data in a list. Then when list is complete enough, you can hand it off to a thread for processing if its heavy. If its ok to parse in GUI thread ( it will hang while parsing) then no thread needed.
I tried to find sample for this but didn't find any good ones as the the "task" handling is very app dependant. -
I don't care about the method being async, because I can just use the
Qt Concurrent
later with the run method in order to solve the freezing problem of the GUI
I was reading the signal and slots documentation and I saw that I can use lambdas instead of assigning a method as a slot. You think that this would work?
Yes, one request depends on the other and the method needs to return something, so I can't really rely on signals and slots in this case. -
@Mr-Gisa
Hi, using lambdas wont help more than using traditional slots but does the code more compact and local.
You need to structure your code so it does not depends on blocking behaviour as for loops.
So you need a way to send out request, collect data,when comes in and then
process them when data is ready.
I dont see that being in a single function without some kind of state machine running it.Maybe others have some trick to do it synchronous without ugly local event loops and such hacks.
-
Yes, most of the codes I found on the internet is exiting the event loop and then executing again after when its finished. But I think that the best way to go here is lambdas as I can't see another way to create this kind of project. I depend on other information so I really don't know what to do knowing that Qt only works with async requests.
-
Hi
Normally people use a small state machine design so code can work async.
( a queue and some signals to switch states )
Im not sure lambdas will help unless you use the local event loop trick. -
That is so weird, I will make some tests with lambda, if it doesn't work I will have to use the event loop hack thing or use a third party library to make the http requests.
-
The problem is you're trying to impose a blocking style on something that is asynchronous in nature.
If I understand you correctly you're thinking like this:void func() { result = doSomething(); //blocks until finished result2 = doSomethingElse(result); //blocks until finished result3 = doYetAnotherThing(result2) //blocks until finished }
This is doable. You just need to make the calls blocking. You could wrap them in something like this:
QByteArray blockingGet(QNetworkRequest* req) { auto reply = qnam.get(req); while(!reply->isFinished()) qApp->processEvents(); reply->deleteLater(); return reply->readAll(); }
but this is kinda horrible.
The way to do it more Qtish is to think about how can you split the functionalities you need into objects that signal their state. To use your search example. You could make a
Searcher
class like this (simplified):class Searcher : public QObject { Q_OBJECT public: void search(const QString& what); // posts a request and connects `finished()` to `processJson` signals: void searchFinished(const QString& result); private: void processJson(); // reads code from json, posts another request and connects `finished()` to `processSearchResult` void processSearchResult(); // parses the response and emits searchFinished with the result };
Then you would make a connection like:
connect(searcher, &Searcher::searchFinished, some_class, &SomeClass::showSearchResultsToTheUser);
and initialize search (e.g. on user input) like this:
searcher->search(something);
For the simple case of two requests this would be enough. For more complex request schemes you would create a state machine (as @mrjj suggested). You can take a look at QStateMachine.
You're saying you need it in a single function. That's the thing - you probably don't. You're just used to doing it this way and trying to force it here.
This might look like more code than the blocking style but don't be fooled. More code doesn't mean worse code. It decouples your network code from the main thread and separates responsibilities a lot nicer.
-
@Chris-Kawa Its amazing, you made me see this in a whole new way, seriously. And about the state machine thing, I don't know if I will need that, but I will tell you what I want to do.
The class is an API wrapper, meaning that it will have a
login
,logout
andsearch
methods. I can only search if the user is logged in. Thesearch
method is the most complex one that I need to create multiple requests, and it also has pages so I will have to loop over the pages and get requests for each one. Like looping over a pagination and put the results in a list. -
Depending on the api of that service looping over pages is also easy. Just don't think about looping in terms of c++
for
.
It's more like:void requestFirstPage(); //makes a request and connects to parsePage() void parsePage();// parses page, if there's more pages makes another request and connects co parsePage() again, otherwise emits some finishing signal
-
What about the result list, I should put it as a member of the class or emit it together with the signal?
-
If you're getting results as you make more requests you'll need a member anyway to store them. After that either make a finishing signal something like
searchFinished(const QStringList& results)
or justsearchFinished()
and another method likeQStringList searchResults() const
so the caller can retrieve results. Or you can combine both, whatever you like more.If you want to update some ui with the results in realtime you could also add a signal like
searchResultsAvailable
after each request processed so the caller can show partial results as they come. -
Amazing amazing, thank you for making me see this in a whole new way. I will be playing with this in order to learn more.
-
@Chris-Kawa I was playing around and I have two questions:
1 - Is it better for me to connect with the
QNetworkAccessManager
finished in order to send theQNetworkReply
to another method such asprocessJson
or I can have oneQNetworkReply
pointer member and reuse that for other requests?2 - What about error handling for each step?
-
The answer to both questions is "whatever you want or will integrate nicely with the rest of your code".
To give it a little more substance:1 - It's generally suggested to use single QNAM instance. Since you mentioned you can have multiple requests in flight at the same time connecting to the QNAM's
finished
signal might be inconvenient. You would have to somehow distinguish which series of requests it is and which "stage" of it it is. That's of course doable. You could for example do something like this:auto response = qnam.get(request); response->setProperty("request_type", foo); //foo could be an enum indicating login, logout, search etc. response->setProperty("request_stage", bar); // bar could be an enum indicating json retrieval or the following request
and then in the slot connected to QNAM's
finished(QNetworkReply*)
signal you could check these properties and do a big "switch". This would work but can get kinda messy as those switches tend to grow over time and gather unrelated functionalities.Another approach is to connect not to the QNAM's
finished
signal but to the response'sfinished
signal. This way you can "string" the requests together and you don't have to do a big switch to see which one it is. every method knows what it receives and can start the next stage.
To get the response object you can use a little lambda on connection site:void Searcher::search(const QString& what) { auto reply = qnam.get( /* build the request from what*/ ); connect(reply, &QNetworkReply::finished, [=]{ processJson(reply); }); } void Searcher::processJson(QNetworkReply* reply) { ... }
Or, if you want to spare yourself the lambda, you can also do it like this:
void Searcher::search(const QString& what) { auto reply = qnam.get( /* build the request from what*/ ); connect(reply, &QNetworkReply::finished, this, &Searcher::processJson); } void Searcher::processJson() { auto reply = qobject_cast<QNetworkReply*>(sender()); ... }
Some people will tell you that
sender()
is bad and you shouldn't use it (even the docs says that), but if you keep methods that use it private I consider that internal piping and a valid use forsender()
. It is considered bad because you never know how the slot is called if it's public, but if it's private you, as the designer of the class should know and be responsible for calling it right.
Anyway, lambda or not, this is one way to do it.2 - That one is pretty simple really. Apart from connecting to
finished
signal of the reply connect also to theerror
andsslErrors
(if you use https). If you don't care what particular error happened just emitsearchFinished
with an empty list of results. If you do care (e.g. want to show some message to the user) instead of emittingsearchFinished
you could emit something likesearchFailed(QString& reason)
or you could add another param to thesearchFinished
signal e.g. an enum that would indicate what type of error occurred or a message string where empty string would indicate a successful search and non-empty would contain the error message. -
@Chris-Kawa So in this case I can use the same
QNetworkReply
with another request right? But where do I delete the reply? I see in the docs that they usually calldeleteLater
in the finish slot, if I want to reuse the reply where to delete it? -
You can't reuse a reply. A new reply object is created by QNAM and returned to you by each call to
get()
,post()
etc.
You can store and reuse a pointer to it but IMO this can lead to bugs if you mix up your requests. I wouldn't recommend that. It's better to just connect to what you need and be done with it.You can call
deleteLater
in the slot connected tofinished()
. As said before - you can't reuse the reply object, just the pointer to it and this is the trouble I mentioned - it's easy to mess up and delete the wrong thing or at the wrong time. I'd suggest not to store it at all and rely on either function param orsender()
in the slot connected tofinished()
signal. -
@Chris-Kawa Thank you very much