Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Web server started through QProcess is unable to receive GET requests
QtWS25 Last Chance

Web server started through QProcess is unable to receive GET requests

Scheduled Pinned Locked Moved Unsolved General and Desktop
qprocesshttp getserverenvironment
22 Posts 2 Posters 6.0k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R Offline
    R Offline
    Red Baron
    wrote on 22 Feb 2018, 08:35 last edited by
    #7

    Calling

    qDebug() << QString(this->testEnv.readAllStandardError());
    qDebug() << QString(this->testEnv.readAllStandardOutput());
    

    returns "" (aka empty string).

    1 Reply Last reply
    0
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 22 Feb 2018, 08:35 last edited by
      #8

      I missed the "unit" ^^

      You're using it wrong. You have to first create the QSignalSpy objects, then start your process, let your unit test flow and at the end you can check the count value.

      What you do here is like creating a counter initialised at zero and checking right away that it's not zero before starting the for loop that will increment it.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      R 1 Reply Last reply 22 Feb 2018, 08:42
      1
      • S SGaist
        22 Feb 2018, 08:35

        I missed the "unit" ^^

        You're using it wrong. You have to first create the QSignalSpy objects, then start your process, let your unit test flow and at the end you can check the count value.

        What you do here is like creating a counter initialised at zero and checking right away that it's not zero before starting the for loop that will increment it.

        R Offline
        R Offline
        Red Baron
        wrote on 22 Feb 2018, 08:42 last edited by Red Baron
        #9

        Doesn't QSignalSpy::wait() help in this situation? I do the same with the QNetworkReply but have a wait() before I actually count. I tried it for the process' QSignalSpy() (even set the timeout to 10000ms = 10s) but all I got was wait() returning false (for each separately).

        1 Reply Last reply
        0
        • R Offline
          R Offline
          Red Baron
          wrote on 22 Feb 2018, 11:00 last edited by Red Baron
          #10

          facepalm It was working. I had a typo in the code OF MY SERVER at one place, which lead to the whole thing doing not what it was supposed to (on the Qt side). :D:D:D Thanks for taking the time!

          Shoot, I was running the server in the background (a terminal) and that is why it worked. Issue is still unresolved... :-/ The typo (see the dashed text above) was the reason why I wasn't getting the expected results even when running in terminal.

          Since the Python script is executable and has the proper shebang I even tried to pass its name as the first argument of QProcess

          this->hTestEnv.start(this->script, this->scriptArgs);
          

          which is the application that is supposed to run. Still nothing.

          1 Reply Last reply
          1
          • R Offline
            R Offline
            Red Baron
            wrote on 22 Feb 2018, 14:22 last edited by
            #11

            Yet another update: I use QProcess::systemEnvironment() to get the environment for the parent process (same as Qt Creator's) and assign it to the child process just to make sure that there isn't some variable that is missing in the equation. Still nothing...

            1 Reply Last reply
            0
            • R Offline
              R Offline
              Red Baron
              wrote on 22 Feb 2018, 15:23 last edited by Red Baron
              #12

              In addition to all I've written so far I just saw that Connection refused error message is printed (through easylogging++), which is coming from the slot I have connected to the reply's SIGNAL(error(QNetworkReply::NetworkError)), plus the following warning generated by Qt itself:

              QWARN : TestHttpRetriever::testReceiveEmptyReply() QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.

              where testReceiveEmptyReply() is the test slot I'm calling the server. All of this does not happen whenever server runs from external terminal.

              1 Reply Last reply
              0
              • R Offline
                R Offline
                Red Baron
                wrote on 22 Feb 2018, 17:06 last edited by
                #13

                I found out what is happening though I have no idea how to solve the problem.

                Basically I ran my unit test in debug mode and had a breakpoint right on the line where the waiting for the emitted signal from my application (which is generated if the processing of the XML reply has been successful). At that point the request to the server was already "on the way". I went to drink some water and when I came back I continued the debugging session. And a miracle happened - I got the same result as when running the server not as a child process but in a separate terminal.

                I did some testing to verify my assumption and it was confirmed - the problem is in the SPEED at which the code in the unit test is executed. As you have mentioned (with the count() for the QSignalSpy()) things happen too fast and there is not enough time for completing the GET request (and processing the reply).

                Now the question that arises here is: how do I fix the timing problem? I tried QThread::sleep() but even setting it to 10 seconds resulted in a partial or no success at all. I obviously need accurate timings or some sort of synchronization method to keep unit test and the dummy serve in synch.

                1 Reply Last reply
                0
                • S Offline
                  S Offline
                  SGaist
                  Lifetime Qt Champion
                  wrote on 22 Feb 2018, 21:19 last edited by
                  #14

                  Can you describe what your test should do ? Maybe show the code for the complete method you are debugging ?

                  From the looks of it, it seems that you should start your web server as part of the unit test start itself and not in the test.

                  Interested in AI ? www.idiap.ch
                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                  R 1 Reply Last reply 22 Feb 2018, 21:39
                  0
                  • S SGaist
                    22 Feb 2018, 21:19

                    Can you describe what your test should do ? Maybe show the code for the complete method you are debugging ?

                    From the looks of it, it seems that you should start your web server as part of the unit test start itself and not in the test.

                    R Offline
                    R Offline
                    Red Baron
                    wrote on 22 Feb 2018, 21:39 last edited by
                    #15

                    @SGaist I will try to provide some more code if possible.

                    The process (with the server) is needed by all tests for the given test case (tests = private slots). That is why the process is a class member and is instantiated only inside initTestCase(), which runs at the beginning of the whole test case and not before each test.

                    In each test (private slot) I configure the server through some datagrams but it's the same server through and through until cleanupTestCase() is called after all tests have been executed. The server contains a UDP part (HTTP and UDP traffic is handled in two different threads), which can receive some datagrams in a specific format. I need this since in real life I actually have to work with two data sources - the actual web server (that I have no access to) and an onboard unit (a computer for controlling various functions in a bus or a tram). Part of the data that the server generated as sends my way needs to contain data that the onboard unit has given me (a sort of a synchronization). I use the datagrams to also alter the behaviour of the server and trigger generation of different replies with synthetic data (that mimics the real one). This happens in every test and is followed by a GET request from the application I'm testing. After that I use a QSignalSpy to detect a signal (or not) that is generated if the processing of the XML reply has been successful. This signal is (in the actual normal execution of my application) caught by a slot from another component that continues the processing and finally outputs stuff on a display.

                    As you can see it's not a trivial task. One thing I can't understand exactly is why in a child process I have these timing issues but when I have the server running in a terminal - not.

                    1 Reply Last reply
                    0
                    • S Offline
                      S Offline
                      SGaist
                      Lifetime Qt Champion
                      wrote on 22 Feb 2018, 21:55 last edited by
                      #16

                      Are you using QNetworkAccessManager to do the requests for your tests ?

                      Interested in AI ? www.idiap.ch
                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                      R 1 Reply Last reply 23 Feb 2018, 07:13
                      0
                      • S SGaist
                        22 Feb 2018, 21:55

                        Are you using QNetworkAccessManager to do the requests for your tests ?

                        R Offline
                        R Offline
                        Red Baron
                        wrote on 23 Feb 2018, 07:13 last edited by
                        #17

                        @SGaist The application itself is using it. Inside the respective module (that I'm actually testing in this test) I have the slots for the QNetworkReply::finished() and QNetworkReply::readyRead(). The reply itself is created by calling the QNetworkAccessManager::get(QNetworkRequest) method. This is how my test looks like (code is still buggy in terms of cleanup at least):

                        #include "testhttpretriever.h"
                        #include "httpretriever.h"
                        #include "framework/telegramoverip/telegramoveripbroadcaster.h"

                        #include <QNetworkAccessManager>
                        #include <QNetworkReply>
                        #include <QUrl>
                        #include <QUrlQuery>
                        #include <QDebug>
                        #include <QSignalSpy>
                        #include <QProcess>
                        
                        #include <easylogging++.h>
                        
                        using Foo::Network::Bar::HttpRetriever;
                        using Foo::Network::Bar::HttpRequestParameters;
                        
                        using Framework::TelegramOverIp::TelegramOverIpBroadcaster;
                        
                        void TestHttpRetriever::initTestCase()
                        {
                            // Create new HttpRetrieve module that will generate the GET requests and process the reply from the dummy server
                            this->retriever = new HttpRetriever(this);
                            this->retriever ->setServer(serverUrl); // Set server URL (a const QUrl with value "http://127.0.0.1:8090")
                            this->retriever ->setAutoRequestInterval(10); // Set interval (in seconds) for automatically triggering GET requests (here it's 10s)
                            this->retriever ->setRequestIdOffset(900000000);
                            this->retriever ->setRequestParams(HttpRequestParameters()); // Use default parameters for the GET request
                        
                            // Create signal spy for the expected emission of signalConnections(), which is emitted once the XML reply has been processed correctly
                            this->retrieverConnDataSpy = new QSignalSpy(this->retriever, &HttpRetriever::signalConnections);
                        
                            // Get path to Python script. Here it is "/home/user/Projects/Application/build/test-bin/HttpServer/testing.py"
                            this->script = QCoreApplication::applicationDirPath() + "/HttpServer/testing.py";
                            int journeys = 4; // Number of journeys in XML reply
                            // Set IP and port for both the HttpServer (that will handle all GET requests) and the onboard unit client (that handles UDP datagrams and can also change some of the settings of the HttpServer)
                            QString hIp = QString("127.0.0.1");
                            int hPort = 8090;
                            QString obuClientIp = QString("127.0.0.1");
                            int obuClientPort = 8091;
                            this->scriptArgs = QStringList() << "-j" << QString::number(journeys)
                                                             << "-ih" << hIp << "-ph" << QString::number(hPort)
                                                             << "-io" << obuClientIp << "-po" << QString::number(obuClientPort);
                            // Retrieve the parent process' environment and set the child process' with it
                            QProcessEnvironment childEnv;
                            childEnv = QProcessEnvironment::systemEnvironment();
                            this->testEnv = new QProcess(this);
                            this->testEnv ->setProcessEnvironment(childEnv);
                            // Change working directory to where the HttpServer script is (because of logs and a couple of XML template files that are used)
                            this->testEnv ->setWorkingDirectory(QCoreApplication::applicationDirPath() + "/HttpServer");
                            // Run the script - no need for QProcess::waitForFinished() since the server will run forever (until SIGTERM received)
                            this->testEnv->start("python2.7", QStringList() << this->script << this->scriptArgs);
                            // Create a new UDP broadcaster that will be used to 1)configure the HttpServer and 2)parse some special UDP telegrams (containing bus line and run number)
                            this->serverControl = new TelegramOverIpBroadcaster(QHostAddress(obuClientIp), obuClientPort, this);
                        }
                        
                        void TestHttpRetriever::cleanupTestCase()
                        {
                            this->testEnv->close(); // Shutdown child process (and server/UDP client)
                        }
                        
                        void TestHttpRetriever::init()
                        {
                        }
                        
                        void TestHttpRetriever::cleanup()
                        {
                        }
                        
                        void TestHttpRetriever::testReceiveEmptyReply()
                        {
                            quint16 ownLine = 123;
                            quint8 ownRun = 4;
                            // Generate UDP telegrams for configuring the HttpServer with the given the own line and run numbers. This is done
                            // to make sure that the XML reply, generated by the server, actually contains the own journey
                            QByteArray serverOwnLine;
                            serverOwnLine.append('l');
                            QString paddedOwnLine = QString("%1").arg(QString::number(ownLine), 3, QChar('0'));
                            serverOwnLine.append(paddedOwnLine);
                            serverOwnLine.append('\r');
                            serverOwnLine.append('\n');
                            // Configure dummy server for generation of own journey with given line
                            this->serverControl->broadcast(serverOwnLine);
                            QByteArray serverOwnRun;
                            serverOwnRun.append('k');
                            QString paddedOwnRun = QString("%1").arg(QString::number(ownRun), 2, QChar('0'));
                            serverOwnRun.append(paddedOwnRun);
                            serverOwnRun.append('\r');
                            serverOwnRun.append('\n');
                            // Configure dummy server for generation of own journey with given run
                            this->serverControl->broadcast(serverOwnRun);
                        
                            // Configure module to look for given own line and run numbers when parsing the XML reply
                            this->retriever->slotSetVehicleData(ownLine, ownRun);
                            // Trigger a GET request with the stop ID 1 (using the offset converted to 900000001)
                            this->retriever->slotStartAutomaticRequests(1);
                        
                            // Wait for signalConnections() to be emitted
                            QVERIFY(this->retrieverConnDataSpy->wait());
                            QCOMPARE(this->retrieverConnDataSpy->count(), 1);
                        
                            // TODO Validate contents of signalConnections()
                        }
                        
                        INITIALIZE_EASYLOGGINGPP
                        QTEST_MAIN(TestHttpRetriever)
                        

                        The slotStartAutomaticRequests() does nothing more than

                        1. Interrupt a previous GET request (if one is currently being awaited or processed)
                        2. Instantiate module's own QNetworkAccessManager (if one is not already present; class member)
                        3. Generate URL for the get request (using the URL and the query parameters provided during the configuration of the module)
                        4. Create a QNetworkRequest for the given URL and query
                        5. Generate a QNetworkReply (class member) using QNetworkAccessManager::get(QNetworkRequest) with the above mentioned request
                        6. Connect the QNetworkReply's error(QNetworkReply::NetworkError), finished() and readyRead() signals to the module's respective slots and also the QNetworkAccessManager's finished() signal to the QNetworkReply's deleteLater() slot
                        7. Wait for reply from server and process it

                        The steps below can be found in any tutorial on how to do GET requests using Qt's networking tools.

                        1 Reply Last reply
                        0
                        • R Offline
                          R Offline
                          Red Baron
                          wrote on 23 Feb 2018, 07:18 last edited by
                          #18

                          I forgot a really important piece of info - I'm using Qt 5.4.2 (will add this to the initial post). Currently I'm looking into this bug that I hope it's not what I'm suffering from.

                          1 Reply Last reply
                          0
                          • S Offline
                            S Offline
                            SGaist
                            Lifetime Qt Champion
                            wrote on 23 Feb 2018, 07:45 last edited by
                            #19

                            I would add waitForStarted to ensure that your python process is indeed running.

                            Interested in AI ? www.idiap.ch
                            Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                            R 1 Reply Last reply 23 Feb 2018, 08:49
                            0
                            • S SGaist
                              23 Feb 2018, 07:45

                              I would add waitForStarted to ensure that your python process is indeed running.

                              R Offline
                              R Offline
                              Red Baron
                              wrote on 23 Feb 2018, 08:49 last edited by
                              #20

                              @SGaist Same error (connection refused etc.) and not change in outcome. I have even put the call inside QVERIFY to check if true is returned.

                              1 Reply Last reply
                              0
                              • R Offline
                                R Offline
                                Red Baron
                                wrote on 23 Feb 2018, 15:12 last edited by
                                #21

                                I added a couple of more tests plus a 10s QThread::sleep() right after the QProcess::waitForStarted() and it seems that it's working now (did 10 runs and not a single failure!). It is possible that the issue is also coming from the UDP synchronization that I have in place since based on it the element I'm looking for during the parsing of the XML reply will either be there or missing. In the case of it being missing but expected to be there the specific test will fail. I'm thinking of adding a confirmation reply from the UDP part of the dummy server and only after the confirmation is received by the given test, the test can proceed.

                                1 Reply Last reply
                                0
                                • S Offline
                                  S Offline
                                  SGaist
                                  Lifetime Qt Champion
                                  wrote on 23 Feb 2018, 21:49 last edited by
                                  #22

                                  IIRC, you have QTest;:wait for that kind of stuff.

                                  Interested in AI ? www.idiap.ch
                                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                  1 Reply Last reply
                                  0

                                  16/22

                                  22 Feb 2018, 21:55

                                  • Login

                                  • Login or register to search.
                                  16 out of 22
                                  • First post
                                    16/22
                                    Last post
                                  0
                                  • Categories
                                  • Recent
                                  • Tags
                                  • Popular
                                  • Users
                                  • Groups
                                  • Search
                                  • Get Qt Extensions
                                  • Unsolved