Web server started through QProcess is unable to receive GET requests

Red Baron

Ahm, this is part of a unit test for a network feature my application has. :P

I run the Python script in my unit test as follows:

Red Baron

The two QSignalSpys (one for each of the mentioned QProcess signals) don't return anything:

QSignalSpy testEnvReadStdErr(&this->testEnv, &QProcess::readyReadStandardError);
QSignalSpy testEnvReadStdOut(&this->testEnv, &QProcess::readyReadStandardOutput);

if (testEnvReadStdErr.count())
{
    qDebug() << "Child stderr"; // Never gets triggered
}

if (testEnvReadStdOut.count())
{
    qDebug() << "Child stdout"; // Never gets triggered
}

I will try your other suggestion with printing the process' channels.

Red Baron

Calling

qDebug() << QString(this->testEnv.readAllStandardError());
qDebug() << QString(this->testEnv.readAllStandardOutput());

returns "" (aka empty string).

SGaist

I missed the "unit" ^^

You're using it wrong. You have to first create the QSignalSpy objects, then start your process, let your unit test flow and at the end you can check the count value.

What you do here is like creating a counter initialised at zero and checking right away that it's not zero before starting the for loop that will increment it.

Red Baron

Doesn't QSignalSpy::wait() help in this situation? I do the same with the QNetworkReply but have a wait() before I actually count. I tried it for the process' QSignalSpy() (even set the timeout to 10000ms = 10s) but all I got was wait() returning false (for each separately).

Red Baron

facepalm It was working. I had a typo in the code OF MY SERVER at one place, which lead to the whole thing doing not what it was supposed to (on the Qt side). :D:D:D Thanks for taking the time!

Shoot, I was running the server in the background (a terminal) and that is why it worked. Issue is still unresolved... :-/ The typo (see the dashed text above) was the reason why I wasn't getting the expected results even when running in terminal.

Since the Python script is executable and has the proper shebang I even tried to pass its name as the first argument of QProcess

this->hTestEnv.start(this->script, this->scriptArgs);

which is the application that is supposed to run. Still nothing.

Red Baron

Yet another update: I use QProcess::systemEnvironment() to get the environment for the parent process (same as Qt Creator's) and assign it to the child process just to make sure that there isn't some variable that is missing in the equation. Still nothing...

Red Baron

In addition to all I've written so far I just saw that Connection refused error message is printed (through easylogging++), which is coming from the slot I have connected to the reply's SIGNAL(error(QNetworkReply::NetworkError)), plus the following warning generated by Qt itself:

QWARN : TestHttpRetriever::testReceiveEmptyReply() QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.

where testReceiveEmptyReply() is the test slot I'm calling the server. All of this does not happen whenever server runs from external terminal.

Red Baron

I found out what is happening though I have no idea how to solve the problem.

Basically I ran my unit test in debug mode and had a breakpoint right on the line where the waiting for the emitted signal from my application (which is generated if the processing of the XML reply has been successful). At that point the request to the server was already "on the way". I went to drink some water and when I came back I continued the debugging session. And a miracle happened - I got the same result as when running the server not as a child process but in a separate terminal.

I did some testing to verify my assumption and it was confirmed - the problem is in the SPEED at which the code in the unit test is executed. As you have mentioned (with the count() for the QSignalSpy()) things happen too fast and there is not enough time for completing the GET request (and processing the reply).

Now the question that arises here is: how do I fix the timing problem? I tried QThread::sleep() but even setting it to 10 seconds resulted in a partial or no success at all. I obviously need accurate timings or some sort of synchronization method to keep unit test and the dummy serve in synch.

SGaist

Can you describe what your test should do ? Maybe show the code for the complete method you are debugging ?

From the looks of it, it seems that you should start your web server as part of the unit test start itself and not in the test.

Red Baron

@SGaist I will try to provide some more code if possible.

The process (with the server) is needed by all tests for the given test case (tests = private slots). That is why the process is a class member and is instantiated only inside initTestCase(), which runs at the beginning of the whole test case and not before each test.

In each test (private slot) I configure the server through some datagrams but it's the same server through and through until cleanupTestCase() is called after all tests have been executed. The server contains a UDP part (HTTP and UDP traffic is handled in two different threads), which can receive some datagrams in a specific format. I need this since in real life I actually have to work with two data sources - the actual web server (that I have no access to) and an onboard unit (a computer for controlling various functions in a bus or a tram). Part of the data that the server generated as sends my way needs to contain data that the onboard unit has given me (a sort of a synchronization). I use the datagrams to also alter the behaviour of the server and trigger generation of different replies with synthetic data (that mimics the real one). This happens in every test and is followed by a GET request from the application I'm testing. After that I use a QSignalSpy to detect a signal (or not) that is generated if the processing of the XML reply has been successful. This signal is (in the actual normal execution of my application) caught by a slot from another component that continues the processing and finally outputs stuff on a display.

As you can see it's not a trivial task. One thing I can't understand exactly is why in a child process I have these timing issues but when I have the server running in a terminal - not.

SGaist

Are you using QNetworkAccessManager to do the requests for your tests ?

Red Baron

@SGaist The application itself is using it. Inside the respective module (that I'm actually testing in this test) I have the slots for the QNetworkReply::finished() and QNetworkReply::readyRead(). The reply itself is created by calling the QNetworkAccessManager::get(QNetworkRequest) method. This is how my test looks like (code is still buggy in terms of cleanup at least):

#include "testhttpretriever.h"
#include "httpretriever.h"
#include "framework/telegramoverip/telegramoveripbroadcaster.h"

#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <QUrl>
#include <QUrlQuery>
#include <QDebug>
#include <QSignalSpy>
#include <QProcess>

#include <easylogging++.h>

using Foo::Network::Bar::HttpRetriever;
using Foo::Network::Bar::HttpRequestParameters;

using Framework::TelegramOverIp::TelegramOverIpBroadcaster;

void TestHttpRetriever::initTestCase()
{
    // Create new HttpRetrieve module that will generate the GET requests and process the reply from the dummy server
    this->retriever = new HttpRetriever(this);
    this->retriever ->setServer(serverUrl); // Set server URL (a const QUrl with value "http://127.0.0.1:8090")
    this->retriever ->setAutoRequestInterval(10); // Set interval (in seconds) for automatically triggering GET requests (here it's 10s)
    this->retriever ->setRequestIdOffset(900000000);
    this->retriever ->setRequestParams(HttpRequestParameters()); // Use default parameters for the GET request

    // Create signal spy for the expected emission of signalConnections(), which is emitted once the XML reply has been processed correctly
    this->retrieverConnDataSpy = new QSignalSpy(this->retriever, &HttpRetriever::signalConnections);

    // Get path to Python script. Here it is "/home/user/Projects/Application/build/test-bin/HttpServer/testing.py"
    this->script = QCoreApplication::applicationDirPath() + "/HttpServer/testing.py";
    int journeys = 4; // Number of journeys in XML reply
    // Set IP and port for both the HttpServer (that will handle all GET requests) and the onboard unit client (that handles UDP datagrams and can also change some of the settings of the HttpServer)
    QString hIp = QString("127.0.0.1");
    int hPort = 8090;
    QString obuClientIp = QString("127.0.0.1");
    int obuClientPort = 8091;
    this->scriptArgs = QStringList() << "-j" << QString::number(journeys)
                                     << "-ih" << hIp << "-ph" << QString::number(hPort)
                                     << "-io" << obuClientIp << "-po" << QString::number(obuClientPort);
    // Retrieve the parent process' environment and set the child process' with it
    QProcessEnvironment childEnv;
    childEnv = QProcessEnvironment::systemEnvironment();
    this->testEnv = new QProcess(this);
    this->testEnv ->setProcessEnvironment(childEnv);
    // Change working directory to where the HttpServer script is (because of logs and a couple of XML template files that are used)
    this->testEnv ->setWorkingDirectory(QCoreApplication::applicationDirPath() + "/HttpServer");
    // Run the script - no need for QProcess::waitForFinished() since the server will run forever (until SIGTERM received)
    this->testEnv->start("python2.7", QStringList() << this->script << this->scriptArgs);
    // Create a new UDP broadcaster that will be used to 1)configure the HttpServer and 2)parse some special UDP telegrams (containing bus line and run number)
    this->serverControl = new TelegramOverIpBroadcaster(QHostAddress(obuClientIp), obuClientPort, this);
}

void TestHttpRetriever::cleanupTestCase()
{
    this->testEnv->close(); // Shutdown child process (and server/UDP client)
}

void TestHttpRetriever::init()
{
}

void TestHttpRetriever::cleanup()
{
}

void TestHttpRetriever::testReceiveEmptyReply()
{
    quint16 ownLine = 123;
    quint8 ownRun = 4;
    // Generate UDP telegrams for configuring the HttpServer with the given the own line and run numbers. This is done
    // to make sure that the XML reply, generated by the server, actually contains the own journey
    QByteArray serverOwnLine;
    serverOwnLine.append('l');
    QString paddedOwnLine = QString("%1").arg(QString::number(ownLine), 3, QChar('0'));
    serverOwnLine.append(paddedOwnLine);
    serverOwnLine.append('\r');
    serverOwnLine.append('\n');
    // Configure dummy server for generation of own journey with given line
    this->serverControl->broadcast(serverOwnLine);
    QByteArray serverOwnRun;
    serverOwnRun.append('k');
    QString paddedOwnRun = QString("%1").arg(QString::number(ownRun), 2, QChar('0'));
    serverOwnRun.append(paddedOwnRun);
    serverOwnRun.append('\r');
    serverOwnRun.append('\n');
    // Configure dummy server for generation of own journey with given run
    this->serverControl->broadcast(serverOwnRun);

    // Configure module to look for given own line and run numbers when parsing the XML reply
    this->retriever->slotSetVehicleData(ownLine, ownRun);
    // Trigger a GET request with the stop ID 1 (using the offset converted to 900000001)
    this->retriever->slotStartAutomaticRequests(1);

    // Wait for signalConnections() to be emitted
    QVERIFY(this->retrieverConnDataSpy->wait());
    QCOMPARE(this->retrieverConnDataSpy->count(), 1);

    // TODO Validate contents of signalConnections()
}

INITIALIZE_EASYLOGGINGPP
QTEST_MAIN(TestHttpRetriever)

The slotStartAutomaticRequests() does nothing more than

Interrupt a previous GET request (if one is currently being awaited or processed)
Instantiate module's own QNetworkAccessManager (if one is not already present; class member)
Generate URL for the get request (using the URL and the query parameters provided during the configuration of the module)
Create a QNetworkRequest for the given URL and query
Generate a QNetworkReply (class member) using QNetworkAccessManager::get(QNetworkRequest) with the above mentioned request
Connect the QNetworkReply's error(QNetworkReply::NetworkError), finished() and readyRead() signals to the module's respective slots and also the QNetworkAccessManager's finished() signal to the QNetworkReply's deleteLater() slot
Wait for reply from server and process it

The steps below can be found in any tutorial on how to do GET requests using Qt's networking tools.

Red Baron

I forgot a really important piece of info - I'm using Qt 5.4.2 (will add this to the initial post). Currently I'm looking into this bug that I hope it's not what I'm suffering from.

SGaist

I would add waitForStarted to ensure that your python process is indeed running.

Red Baron

@SGaist Same error (connection refused etc.) and not change in outcome. I have even put the call inside QVERIFY to check if true is returned.

Red Baron

I added a couple of more tests plus a 10s QThread::sleep() right after the QProcess::waitForStarted() and it seems that it's working now (did 10 runs and not a single failure!). It is possible that the issue is also coming from the UDP synchronization that I have in place since based on it the element I'm looking for during the parsing of the XML reply will either be there or missing. In the case of it being missing but expected to be there the specific test will fail. I'm thinking of adding a confirmation reply from the UDP part of the dummy server and only after the confirmation is received by the given test, the test can proceed.

SGaist

IIRC, you have QTest;:wait for that kind of stuff.