How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication) ?

eupuser00

Hello.

I am running a Qt console app in Windows with a detached console. The app has a server to handle some requests and also to act upon some file-watchers. Internally I call some external executables which print messages to stdout.

I want all the output messages of the app (and executables called internally) to be logged in a file instead of stdout. I have used "qInstallMessageHandler" to capture the logs. But this only logs the messages logged through "qDebug()" and the other messages which are directed to stdout are left out.

I tired using "freopen" and it didn't redirect the stdout to the specified file even though I got proper return ptr value from the call.

Is it possible to redirect all output to stdout and stderr to a file when running an console app using QCoreApplication?

Thank you in advance.

Christian Ehrlicher

@eupuser00 said in How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication) ?:

Internally I call some external executables which print messages to stdout.

See QProcess::readAllStandardOutput() and it's corresponding signals.

JonB

@eupuser00
You can do as @Christian-Ehrlicher has just posted, but that will only affect stdout writing from child processes. I believe you are asking for redirection of anything written within your calling program itself, libraries etc.

I'm not a C++-er, but I think your current problem is: stuff written to C++ cout is not the same thing as stuff written to stdout? See http://www.cplusplus.com/reference/iostream/cout/ and http://www.cplusplus.com/reference/ios/ios_base/sync_with_stdio/

If the streams are synchronized, a program can mix iostream operations with stdio operations, and their observable effects are guaranteed to follow the same order as used in the thread.

I don't know what the "synchronized" & "observable effects" add up to, but it may be that cout uses file handle #1 like stdout does but it does not actually use stdout --- so freopen() won't be enough.

You either ought do your redirection on low-level file handles 0/1/2 (fdopen(), and possibly some dup()-ping, IIRC) instead of just on stdin/out/err, or perhaps cout has its own thing. Googling for something about "redirecting stdout cout" might help you.

eupuser00

@Christian-Ehrlicher

Thanks. But your suggestion works when I am using QProcess to execute the external binaries. And as @JonB says I need to redirect everything from my main program.

Christian Ehrlicher

@eupuser00 said in How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication) ?:

I need to redirect everything from my main program.

Why do you use cout in your own program in the first place? Esp. when it has no console...

eupuser00

No, I am not using cout at all (only qDebug, qWarning,...). But the external tools or the libs use it and they log to stdout and these messages are only visible in Qt app output console(if i run the tool from Qt Creator) and not in my redirected log files.

hskoglund

Hi, what happens if you launch your console app with an ancient "DOS"-style redirection, e.g.

yourconsoleapp.exe > filename.txt

eupuser00

@hskoglund

Not output at all. freopen is not logging anything when i use:
freopen("log_file.txt", "w", stdout);
or
freopen("log_file.txt", "w", (FILE*)1);

Also Qt keeps complaining for freopen :

main.cpp:16:1: warning: 'freopen' is deprecated: This function or variable may be unsafe. Consider using freopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
stdio.h:242:20: note: 'freopen' has been explicitly marked deprecated here
vcruntime.h:255:55: note: expanded from macro '_CRT_INSECURE_DEPRECATE'
vcruntime.h:245:47: note: expanded from macro '_CRT_DEPRECATE_TEXT'

hskoglund

Same if you omit all freopen() calls?

JonB

@eupuser00
Is there any reason you don't want to look at low-level e.g. fdopen() when I've already explained that you need to do that to achieve what you want, and why stdout does not work for you? Makes me wonder what the point of answering was.

Christian Ehrlicher

@eupuser00 said in How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication) ?:

But the external tools or the libs use it and they log to stdout and these messages

So how do you call them when not with QProcess? And when you use something else - why not use QProcess?

eupuser00

@hskoglund
Command line style redirection didn't work when I used "2>&1 > file" as one of the arguments to the QProcess::start().
With "freopen()" even the qDebug messages which were logged through my custom msg handler was failing. (Without "freopen()" the qDebug messages were logged.)

@Christian-Ehrlicher
I was calling the external programs through the static function in "QProcess::execute()". But now switched to use the "QProcess::start()" so that I can use "QProcess::setProcessChannelMode()" as shown below.

@JonB
I was able to get it working with File Handles and dup(). Here is the code snippet:

// Global scope
// Setting same file for stdout & stderr (and qDebug from custom msg handler 
// func, "customMessageOutput")
QFile outFile("log_file.txt");

execProg(){
  ...
  QProcess extTool;
  extTool.setProcessChannelMode(QProcess::ForwardedChannels);
  extTool.start(program, arg);
  extTool.waitForFinished();
  ...
}  

redirect_stdout_stderr() {
   outFile.open(QIODevice::WriteOnly | QIODevice::Append)
  fout = outFile.handle();
 
  // redirect stdout 
  _dup2(fout, _fileno(stdout))

  // redirect stderr 
  _dup2(_fileno(stdout), _fileno(stderr))
}

main(){
  redirect_stdout_stderr();    
  // Installing a custom message handler
  // Note: Definition for "customMessageOutput" is not shown here.
  qInstallMessageHandler(customMessageOutput);
  ...
  execProg();
  ...
}

For brevity, I have not included error handling.
I had trouble to redirect both stdout and stderr to the same file. I was opening the same file twice and trying to use 2 file descriptors/ handles. This got solved when I used:
_dup2(_fileno(stdout), _fileno(stderr))

Thank you all for the suggestions and pointing me towards the right direction to achieve a solution.

I would like to know whether there is a better/elegant solution in Qt? If yes, then I would like to adapt to it.

PS: I will mark this thread as "solved" once I get some or no suggestions.

JonB

@eupuser00
Well done! For heeding the advice, and for getting the code right :)

Yes, dup/dup2 is for merging stdout & stderr.

From a GUI I did something like the following during start up in old code

close(0);
close(1);
close(2);
open("/dev/null", O_RDONLY);
open(logfile, O_WRONLY);
dup(1);

I like having stdin (0) opened from a null device, so any attempt to read from it delivers EOF.

Qt isn't going to provide a better solution. This operates at the lowest level.

You might have to check your code on Windows. Hopefully it works, I suspect I did that stuff via native calls as their will be some situation where it's better.

David K. Hess

Late to the party, but figured out some more of this situation that may help others.

The straight duping to fd 1 or 2 and/or use of _fileno() with stdout and stderr don't work when the app is launched normally within Windows by say a double-click in File Explorer (and thus has no console). The reason why is in that situation stdout and stderr both have an fd of -2 which I suppose indicates they are open for purposes of not breaking calling code but are not open in terms of actually doing anything when used. In other words, sort of a CRT /dev/null setup.

More info on that situation here: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fileno?view=msvc-160

So, this general pattern worked for me:

        FILE *loggingStreamStdout;
        FILE *loggingStreamStderr;
        QFile loggingFile("app.log");

        loggingFile.open(QIODevice::WriteOnly | QIODevice::Append | QIODevice::Unbuffered);
        int fd = loggingFile.handle();

        freopen_s(&loggingStreamStdout, "\\\\.\\NUL", "w", stdout);
        setvbuf(loggingStreamStdout, NULL, _IONBF, 0);
        freopen_s(&loggingStreamStderr, "\\\\.\\NUL", "w", stderr);
        setvbuf(loggingStreamStderr, NULL, _IONBF, 0);
        _dup2(fd, _fileno(stdout));
        _dup2(fd, _fileno(stderr));

        qInstallMessageHandler(messageOutput);

That uses a QFile to handle your log file and then freopens stdout and stderr on Window's version of /dev/null. This gives them valid file descriptors (instead of the -2). Then you can _dup2() the log file fd to them. I also like to set the streams as non-buffered so that tools like Baretail.exe show real-time output from the exe.

Developer123

@David-K-Hess It says identifier "_dup2" is undefined

Developer123

@eupuser00 it's almost useless for other people if you paste incomplete code...

Christian Ehrlicher

@Developer123 said in How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication) ?:

it's almost useless for other people if you paste incomplete code...

_dup2() is a function, a function is defined in a header (as all functions not defined in the same source). Since the op did not post the includes you have to search the correct one by yourself which shouldn't be too hard. First google link: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/dup-dup2?view=msvc-170

Developer123

@Developer123 I found a solution to print everything to a file, if anyone else needs, works out of the box

https://stackoverflow.com/a/11202102/2076973

JonB

@Developer123
The code answers there all implement sending Qt error messages within an application to a file. Perfectly useful in itself, but nothing to do with the topic title here of "How to redirect stdout & stderr to a file in a Qt console App (QCoreApplication)".