Getting GTest unit tests to integrate correctly with Qt Creator

GarryJ

Had a quick browse around the forums and couldn't find this particular issue, so here's the lowdown; we have:

Ubuntu
Qt Creator 4.7.0
A CMake project (using CMake 3.8)
Googletest (GTest) unit test binaries

Qt Creator detects the unit tests correctly after opening the project, and shows them in the test pane. But, trying to run them gives the following error:

FATAL: Test for project "the-project-name" failed to produce any expected output."

When running the test binaries using the run button, the expected GTest output is shown in the application output window. Similarly, when running the unit test binary outside of Qt in a shell, the output is shown as expected.

I've had a quick look at the Qt Creator source - the error comes from line 254 in plugins/autotest/testrunner.cpp, but it's not immediately clear from the source exactly how the mechanism works; i.e., how does Qt Creator parse and validate the output from the unit tests. As such, I can't tell if it's a bug, or if my setup is somehow incorrect.

Any ideas?

SGaist

Hi and welcome to devnet,

From a quicklook, you should check the GTestOutputReader class. There might be something that has changed with the version of GTest you are using.

GarryJ

Thanks for the tip - after building Qt Creator from source and inserting some debug statements, I found the issue; it was because I had GTEST_COLOR=1 set in my environment.

Explanation, just in case anyone else has this problem in future:

GTEST_COLOR is an environment variable you can set so that test output in a shell is coloured according to the test result.

When GTEST_COLOR is not set, Qt Creator sees and processes test output like this (in this case, in GTestOutputReader::processOutput():

outputLine = "[==========] Running 1 test from 1 test case."

If, however, GTEST_COLOR=1 is set, Qt Creator sees the colour formatting, which it can't parse:

outputLine = "\x1B[m\x1B[0;32m[==========] \x1B[mRunning 1 test from 1 test case."

It'd be nice if Qt Creator could parse this, as there are cases where Qt Creator needs to be run from a shell where it's handy to also be able to run the unit tests standalone (in our case, it's a cross-compilation environment using Yocto), so I'll make a suggestion on the Qt Creator JIRA.

SGaist

Good catch !

Don't forget to post the link to the suggestion.

It would also be nice to add an example of test output with and without GTEST_COLOR set to the report.

aha_1980

@GarryJ

The fastest solution I could imagine would be to un-set the environment variable within Creator, e.g. in the build environment.

I don't know if the colored test output is visible in Creator, if not clearing the variable would also be the easiest fix in code.

Regards

GarryJ

@aha_1980

Yeah, that's one option!

It's not ideal in my particular case, though, because of the way Qt Creator works with CMake projects (it creates CMakeLists.txt.user on the fly). In the situation where you have to detect CMake and the compiler from the environment variables (because, for example, you have multiple targets requiring different toolchains), the CMakeLists.user.text file can get messy with Imported Kits pretty quickly, so I often find myself having to delete the file so I can keep track of which Kits are which. Deleting the file would, of course, remove any project-specific environment settings. It'd be real nice if Qt Creator had a command line interface which would allow adding a Kit, but for now detection from the environment is the easiest way.

The option I went for is to avoid setting the environment variable completely, and then specify the --gtest_color=yes option when adding a test in CMake:

add_executable(unittest-binary
      test-source.cpp
      )

add_test(
    NAME unittest-name
    COMMAND unittest-binary --gtest_color=yes
    )

This way, running tests via the shell using ctest outputs in colour.

GarryJ

For posterity: the JIRA issue is here: https://bugreports.qt.io/browse/QTCREATORBUG-21012.