Why does removal of a.exec() from console app prevent QNetworkAccessManager posting data?

Snorkelbuckle

I created a console app using the Qt template for it. When I run it, I decided to remove "return a.exec();" from the end so that console doesn't hang, waiting around for something that will never happen.

I use a logger that logs to Seq Server [in case anybody interested in Seq: https://datalust.co/seq, I highly recommend it]. This logger sends the logs via QNetworkManager by POSTing to the server's API endpoint.

The strange thing I noticed is that with the "return a.exec()" removed, no logging at all happens! When I ran through debugger step by step, I can see that the logging code is executed properly and there are no exceptions, just nothing POSTED to the server. I add back in the a.exec() just for kicks, and bam!, it starts logging again.

I was under the impression that a.exec() is not really needed in a console app, but I guess I'm wrong. What exactly is happening in a.exec() that allows QNetworkManager to function correctly?

If I need a.exec() for this to work, then what do I need to do to keep this in my code and implement so that when the console app is finished, it actually exits and doesn't hang around waiting for something.

Here is what main looks like:

#include <QCoreApplication>
#include <QString>
#include <seqlogger.h>

int main(int argc, char *argv[])
    {
    SeqLogger *_myLog = SeqLogger::Instance();
    _myLog->SetHost("127.0.0.1");
    _myLog->SetMinimumLogLevel(SeqLogger::LogLevel::InformationLevel);
    _myLog->Information("Starting Main: {application}->{file}", {"Snorky-Perft", QString(__FILE__)});
    QCoreApplication a(argc, argv);

    // Do some stuff
    // ...
    // ...
    // Finish stuff
  
    // IF THIS IS COMMENTED OUT, then no output occurs to the logger at *_myLog
    return a.exec();
    }

Here is where the logging actually occurs and sends to the network, it is all really quite simple, but doesn't POST anything if a.exec() in main.cpp is commented out.

void SeqLogger::PostMessage(QJsonDocument *document)
    {
    QNetworkRequest request;
    request.setUrl(_api_endpoint);
    request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json");
    _network_manager->post(request, document->toJson(QJsonDocument::JsonFormat::Compact));
    }

The question is, why is a.exec() so important to a console app, and in particular, QNetworkManager?

SGaist

As I already wrote in my first message, you don't take into account the asynchronous nature of QNetworkAccessManager.

So what you are currently doing is:

Call information
Fire the request
Emit finished
End the application

So your post request might have started but you kill it before it has ended. As I already suggested, connect the finished signal of either QNetworkReply or QNetworkAccessManager to the quit slot of QApplication. In the absolute you should have an API in your logger for that.

Another very important point, don't create any QObject based class instance before QCoreApplication, it shall be the first object create because it puts in place all the internals needed for proper signal, slot, event, etc handling.

SGaist

Hi,

Because you do not start the event loop. Since QNetworkAccessManager is asynchronous, it won't be able to do its work.

You can connect the finished signal to QCoreApplication::exit to end your application automatically.

Snorkelbuckle

Okay, makes sense! I guess I need to make a few changes and use event loop.

Thanks!

Snorkelbuckle

Okay, well, I took your advice and did the event loop thingy. It is actually worse now. What is going on here?? Now nothing gets logged, no matter what, again, tracing in debugger shows the code of the logger is getting executed and really nothing wrong with it. I believe the issue is something to do with the event loop as you described, but what is the right way to create a console app? This seems to be a persistent issue when I search on the web as many seem to have similar problems. Qt is really great, I like it, but dang, doing a simple console app that wants to use Qt libraries is just annoying as heck.

The pattern I followed is this: https://treyweaver.blogspot.com/2013/02/qt-console-application-template-tutorial.html. My only difference is that I didn't implement "aboutToQuitApp()" because there is no cleanup to be done.

main.cpp

#include <QCoreApplication>
#include <QtCore>
#include <QString>
#include <task.h>
#include <seqlogger.h>

int main(int argc, char *argv[])
    {
    SeqLogger *_myLog = SeqLogger::Instance();
    _myLog->SetHost("127.0.0.1");
    _myLog->SetMinimumLogLevel(SeqLogger::LogLevel::InformationLevel);
    _myLog->Information("Starting Main: {application}->{file}", {"Snorky-Perft", QString(__FILE__)});
    QCoreApplication a(argc, argv);
    Task *task = new Task(&a);
    QObject::connect(task, &Task::finished, &a, &QCoreApplication::quit);
    LOGINFO("Starting exec()", {});
    QTimer::singleShot(0, task, &Task::run);
    return a.exec();
    }

task.h

#ifndef TASK_H
#define TASK_H

#include <QObject>
#include <QString>
#include <QTextStream>

class Task : public QObject
    {
        Q_OBJECT
    public:
        explicit Task(QObject *parent = nullptr);

    public slots:
        void run();

    signals:
        void finished();

    };

#endif // TASK_H

task.cpp

#include "task.h"
#include <seqlogger.h>

Task::Task(QObject *parent) : QObject(parent)
    {
    }

void Task::run()
    {
    SeqLogger *_myLog = SeqLogger::Instance();
    _myLog->Information("Doing something in Task::run()", {});
    emit finished();
    }

Christian Ehrlicher

@Snorkelbuckle said in Why does removal of a.exec() from console app prevent QNetworkAccessManager posting data?:

This seems to be a persistent issue when I search on the web as many seem to have similar problems

Which problems? Do they all forget to run the event loop? Without running the event loop, signals and slots won't work so there is no other way than this.

Snorkelbuckle

@Christian-Ehrlicher

In general most have problems trying to figure out the right way to code a console app in Qt. But that is not my real question, do you have any insight as to why in the above code QNetworkAccessManager won't POST any data? The logger function in Task::run() is simply calling the SeqLogger::PostMessge() function which is only posting a JSON object.

At first, it was thought that it was because the event loop wasn't being used. So I'm only adding the event loop in the manner that seems to be generally accepted, but it just won't post anything to the endpoint, and all the code paths for the logger are working correctly. If I change this to a gui app and use QApplication instead of QCoreApplication, keep the code basically the same, then the logger works. Or, as indicated in my first post, I just do the really simple console app and I must include the a.exec() to allow it to send output, but then the app just hangs because no exit from the loop has occurred. Also, the logging doesn't occur at until after the program is closed forcibly. So I could log hundreds of messages over long time, but until the program is forcibly quit, the logging isn't posted.

This brings me back to the original question, what is the proper way to create a console app that can use QNetworkAccessManager with real-time logging via POST to an API endpoint and the app doesn't hang until forcibly closed. Really simple, and nobody seems to have an answer that I've found. It is certainly not in the docs, I've checked or if there in the docs, very obscure. Is it maybe that QNetworkAccessManager is not meant to be used in non-gui apps? If that is the case, then it should be documented.

Is this a bug? If it is, I'll gladly report it, but if it isn't, please tell me the right way to do it? Anybody?

SGaist

As I already wrote in my first message, you don't take into account the asynchronous nature of QNetworkAccessManager.

So what you are currently doing is:

Call information
Fire the request
Emit finished
End the application

So your post request might have started but you kill it before it has ended. As I already suggested, connect the finished signal of either QNetworkReply or QNetworkAccessManager to the quit slot of QApplication. In the absolute you should have an API in your logger for that.

Another very important point, don't create any QObject based class instance before QCoreApplication, it shall be the first object create because it puts in place all the internals needed for proper signal, slot, event, etc handling.

Christian Ehrlicher

@Snorkelbuckle

It is certainly not in the docs,

It is, see https://doc.qt.io/qt-5/qnetworkaccessmanager.html#details

"QNetworkAccessManager has an asynchronous API."

Snorkelbuckle

@SGaist said in Why does removal of a.exec() from console app prevent QNetworkAccessManager posting data?:

As I already wrote in my first message, you don't take into account the asynchronous nature of QNetworkAccessManager.

So what you are currently doing is:

Call information

Fire the request

Emit finished

End the application

So your post request might have started but you kill it before it has ended. As I already suggested, connect the finished signal of either QNetworkReply or QNetworkAccessManager to the quit slot of QApplication. In the absolute you should have an API in your logger for that.

Another very important point, don't create any QObject based class instance before QCoreApplication, it shall be the first object create because it puts in place all the internals needed for proper signal, slot, event, etc handling.

Well, I guess what I really need is a synchronous method to POST to an endpoint. Since QNetworkAccessManager is not synchronous, I decided to make it act synchronous, this is a hack, but I think it works out okay for my purposes. To fix this in my console app, I do this at the point where the access manager is posting data to the endpoint:

    QEventLoop eventLoop;
    connect(_network_manager, &QNetworkAccessManager::finished, &eventLoop, &QEventLoop::quit);
    _network_manager->post(request, document->toJson(QJsonDocument::JsonFormat::Compact));
    eventLoop.exec();

So it is now practically synchronous. I'm putting a request to the Qt guys to add synchronous ability with QNetworkAccessManager, I think it is an important missing feature. The alternative to those needing synchronous is to build your own qnetworkaccessmanager from the ground up using QTcpSocket.

Christian Ehrlicher

@Snorkelbuckle said in Why does removal of a.exec() from console app prevent QNetworkAccessManager posting data?:

I'm putting a request to the Qt guys to add synchronous ability with QNetworkAccessManager, I think it is an important missing feature

Blocking event handling is bad practice which is what you want here - it's wrong.

Snorkelbuckle

@Christian-Ehrlicher

I understand what you are saying and I agree with you up to a point, but shouldn't that be a decision for the application developer? In my particular case, I don't need async network calls. And I don't need the extra that is needed trying to implement an event loop for a console app to use QNetworkAccessManager which has a very useful methods for POST and GET http requests. If this was a gui-app, then there is no issue, I can utilize the built-in event loop of qt gui app. But with the console app, it seems i have to bend over backwards to do anything with an async API of QNetworkAccessManager. So my choices are build from scratch using QTcpSocket which supports synchronous calls as far as I can tell, or build a glorified logger that spawns its own process with all the extra needed for that.

I just wanted a simple logger that logs to a server at an http endpoint.

So yes, I agree that is bad practice to block an event loop. BUT, I didn't want it in the first place, QNetworkAccessManager forces it if you want to use it in an console app that is meant to be synchronous.