Autotest scanner doesn't find multiple QtTest test cases

pkeller

Hi,
My first post, so please bear with me. My question is similar to this one: https://forum.qt.io/post/413777 (unsolved), but simpler.
I hacked up a template project created by QtCreator; my main() looks like so:

#include <QCoreApplication>
#include <QtTest>

#include "tst_testcase1.h"
#include "tst_testcase2.h"

int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);
    app.setAttribute(Qt::AA_Use96Dpi, true);
    QTEST_SET_MAIN_SOURCE_PATH

    int status = 0;
    {
    TestCase1 tc;
    status |= QTest::qExec(&tc, argc, argv);
    }
    {
    TestCase2 tc;
    status |= QTest::qExec(&tc, argc, argv);
    }

    return status;
}

where tst_testcase1.cpp/h and tst_testcase2.cpp/h are just the pieces of the template project generated by Qt Creator. The test scanner finds only TestCase2. The scanner reliably catches tests that I add to TestCase2, but TestCase1 is ignored no matter what I do.

Based on my experience with Google tests in Qt Creator, I even tried keeping the declarations and definitions together, and including the whole shmear in my main() file. Same result.

Thanks in advance for any ideas!

pkeller

OK, I found a simple solution:

Structure TestCase1 and TestCase2 exactly as the Qt creator wizard would do it: #includes, test case declaration, test case definition, QTEST_MAIN, and moc-file include, each in a separate .cpp file.
Manually expand the QTEST_MAIN macro, and change main() to main1() for TestCase1, and main2() fo TestCase2.
Add another .cpp file, with the actual main(), as shown below:

int main1(int argc, char *argv[]);
int main2(int argc, char *argv[]);

int main(int argc, char *argv[])
{
    return
            main1(argc, argv) |
            main2(argc, argv);
}

Bingo presto, the whole thing links OK, AND the Autotest parser likes it!

pkeller

Update: my simple solution unfortunately doesn't work the way I want it to, due to the command-line arguments that the test runner gives to the test program:

If you want to run all the tests, the test runner passes no test names. The test cases are run one after the other, and all is well.
If you want to run just one test, or a selection of tests, the test runner passes a list of names of the tests to run. If none of those tests happen to be in TestCase1, I would have expected QTest::qExec to just return without doing anything; unfortunately, the stupid thing actually quits the application! Which means that no tests get run. I got around that by prefixing all my tests with something like "TestCase1_", and putting an if statement in front of each execution of Test::qExec, which parses the arguments and checks to see whether any tests for this test case are in the list.
The problem with that is that selecting an entire group of tests behaves like running all the tests, i.e. the test runner passes no test name at all, and all test cases are run. Grrr...

As you can see, I have gone around the block on this a couple of times, and I have come to believe that the only real solution is the documented one, as per https://doc.qt.io/qtcreator/creator-autotest.html:

Declare a single test object in a single .h file.
Create separate .cpp files for the method definitions that correspond to the different test cases, and place them in separate subfolders. Each .cpp must of course include the .h test object definition. Select the QtTest "Group" option, as described in "Selecting Tests to Run" of the documentation, above.
Finally, main() is in yet another .cpp file: just a standard QTEST_MAIN(...), but without the #include "xxx.moc" (since our test object is declared in a .h file).

Unfortunately, I'm unable to try out my own advice on my present project, since I need different initTestCase() and cleanupTestCase() methods for my different test cases. I'm back to separate projects, one for each test case... :-(

aha_1980

@pkeller said in Autotest scanner doesn't find multiple QtTest test cases:

Unfortunately, I'm unable to try out my own advice on my present project, since I need different initTestCase() and cleanupTestCase() methods for my different test cases. I'm back to separate projects, one for each test case... :-(

Have you set them up as SUBDIRS project? Then you should be able to run all tests or selected tests. The Qt autotests itself are organized that way.

Regards