Testing with multiple QTEST_APPLESS_MAIN
-
Hello,
I have multiple test classes that build into an executable. QTEST_APPLESS_MAIN() seems to be the right choice to setup a main function for my tests to run. Using that macro at the end of the test class works. But when I have multiple test classes this does not work anymore because of multiple main functions.
I used following guide https://alexhuszagh.github.io/2016/using-qttest-effectively/, but this results in me not being able to run test classes seperately. They all get executed.
I worked on a big software project that had the same test structure as mine and they made it work (with work I mean multiple test classes that build in to one exe that can be debugged seperately). No access to that project anymore.
Anyone got any ideas?
-
@Redman
I just found this post:
https://forum.qt.io/post/448807
Paul Colby addressed all my concerns and provided a solution.This is a bit dated, tough. Is this still the way to go?
Hi @Redman said in Testing with multiple QTEST_APPLESS_MAIN:
@paul-colby addressed all my concerns and provided a solution.
This is a bit dated, tough. Is this still the way to go?Speaking just from personal experience / opinion (and as the author of the old linked solution), I'd say as long as you have CMake as an option now, I would not do it this way (and I don't anymore). That said, if you do really want multiple test classes in one executable, then I'd say the linked approach is still the best way to go.
What I wanted to solve with that multiple-test-classes-per-executable approach, was to have an easy way to run all, or a sub-set of tests. And be able to get combined coverage results for all tests, together. But CMake makes that's so easy for me now, that I would (and do) adhere to the recommended one-test-class-per-executable, which prevents tests cross-polluting each other (and allows them to run in parallel).
Basically, I use CMake's add_test() for each test. So, for example, if I have have
testClassAtotestClassZ, then, once built, I can run tests like:make test # or nmake.exe, etc. Will run all tests ctest # portable; will run all tests ctest --verbose # will run all tests, in their verbose modes ctest -R ClassA # will run tests for ClassA ctest -R 'Class[A-E]' # run tests for ClassA through E; can use any regex pattern.Or, my favorite:
ctest --output-on-failureWhich will run all the tests, with nice succinct output, but will give the full output only for tests that failed.
Using individual CMake tests also means they appear as individual tests in Qt Creator as part of Qt Creator's CMake integration, which is less important to me (most of my tests run from the terminal), but can be very handy when you're living in the IDE.
So, here's a more modern example of how I do Qt project unit tests today: https://github.com/pcolby/dokit/tree/main/test/unit
I'm sure there's plenty I could be doing better there, but I do find it much better than my old all-in-one-executable approach you linked.
Cheers.
-
Hello,
I have multiple test classes that build into an executable. QTEST_APPLESS_MAIN() seems to be the right choice to setup a main function for my tests to run. Using that macro at the end of the test class works. But when I have multiple test classes this does not work anymore because of multiple main functions.
I used following guide https://alexhuszagh.github.io/2016/using-qttest-effectively/, but this results in me not being able to run test classes seperately. They all get executed.
I worked on a big software project that had the same test structure as mine and they made it work (with work I mean multiple test classes that build in to one exe that can be debugged seperately). No access to that project anymore.
Anyone got any ideas?
-
@Redman
I just found this post:
https://forum.qt.io/post/448807
Paul Colby addressed all my concerns and provided a solution.This is a bit dated, tough. Is this still the way to go?
Hi @Redman said in Testing with multiple QTEST_APPLESS_MAIN:
@paul-colby addressed all my concerns and provided a solution.
This is a bit dated, tough. Is this still the way to go?Speaking just from personal experience / opinion (and as the author of the old linked solution), I'd say as long as you have CMake as an option now, I would not do it this way (and I don't anymore). That said, if you do really want multiple test classes in one executable, then I'd say the linked approach is still the best way to go.
What I wanted to solve with that multiple-test-classes-per-executable approach, was to have an easy way to run all, or a sub-set of tests. And be able to get combined coverage results for all tests, together. But CMake makes that's so easy for me now, that I would (and do) adhere to the recommended one-test-class-per-executable, which prevents tests cross-polluting each other (and allows them to run in parallel).
Basically, I use CMake's add_test() for each test. So, for example, if I have have
testClassAtotestClassZ, then, once built, I can run tests like:make test # or nmake.exe, etc. Will run all tests ctest # portable; will run all tests ctest --verbose # will run all tests, in their verbose modes ctest -R ClassA # will run tests for ClassA ctest -R 'Class[A-E]' # run tests for ClassA through E; can use any regex pattern.Or, my favorite:
ctest --output-on-failureWhich will run all the tests, with nice succinct output, but will give the full output only for tests that failed.
Using individual CMake tests also means they appear as individual tests in Qt Creator as part of Qt Creator's CMake integration, which is less important to me (most of my tests run from the terminal), but can be very handy when you're living in the IDE.
So, here's a more modern example of how I do Qt project unit tests today: https://github.com/pcolby/dokit/tree/main/test/unit
I'm sure there's plenty I could be doing better there, but I do find it much better than my old all-in-one-executable approach you linked.
Cheers.
-
R Redman has marked this topic as solved on
-
Hi @Redman said in Testing with multiple QTEST_APPLESS_MAIN:
@paul-colby addressed all my concerns and provided a solution.
This is a bit dated, tough. Is this still the way to go?Speaking just from personal experience / opinion (and as the author of the old linked solution), I'd say as long as you have CMake as an option now, I would not do it this way (and I don't anymore). That said, if you do really want multiple test classes in one executable, then I'd say the linked approach is still the best way to go.
What I wanted to solve with that multiple-test-classes-per-executable approach, was to have an easy way to run all, or a sub-set of tests. And be able to get combined coverage results for all tests, together. But CMake makes that's so easy for me now, that I would (and do) adhere to the recommended one-test-class-per-executable, which prevents tests cross-polluting each other (and allows them to run in parallel).
Basically, I use CMake's add_test() for each test. So, for example, if I have have
testClassAtotestClassZ, then, once built, I can run tests like:make test # or nmake.exe, etc. Will run all tests ctest # portable; will run all tests ctest --verbose # will run all tests, in their verbose modes ctest -R ClassA # will run tests for ClassA ctest -R 'Class[A-E]' # run tests for ClassA through E; can use any regex pattern.Or, my favorite:
ctest --output-on-failureWhich will run all the tests, with nice succinct output, but will give the full output only for tests that failed.
Using individual CMake tests also means they appear as individual tests in Qt Creator as part of Qt Creator's CMake integration, which is less important to me (most of my tests run from the terminal), but can be very handy when you're living in the IDE.
So, here's a more modern example of how I do Qt project unit tests today: https://github.com/pcolby/dokit/tree/main/test/unit
I'm sure there's plenty I could be doing better there, but I do find it much better than my old all-in-one-executable approach you linked.
Cheers.
@Paul-Colby Thank you for that detailed answer. I'm using qmake so I guess for now I will stick with that "old" structure. But your answer is bookmarked for future use :)
