[Solved] Unit test failure notification

deanz · wrote on 11 Oct 2010, 19:52

I’m using QMAKE_POST_LINK to run a unit test automatically for each build. My test script returns 0 on success and 1 on failure. The script is working and being automatically run OK.

When I build in Qt Creator and the unit test fails, no notification comes up in the “Build Issues” pane. Is there a way to cause the post-link test failure to get flagged in “Build Issues”?

deanz · wrote on 11 Oct 2010, 20:06

Further on this query.

I find that if I write failure descriptive test to stderr rather than stdout, Qt Creator does take notice in that it displays the text in red rather than black in the "Compile Output" pane. But still nothing in the "Build Issues" pane.

? This user is from outside of this forum · wrote on 12 Oct 2010, 05:34

deleted the duplicate thread that was created here http://developer.qt.nokia.com/forums/viewthread/1234/

lyuts · wrote on 12 Oct 2010, 08:07

As far as I understand it is up to QtTestLib to decide whether to print to stderr or stdout. Do you redirect output in your script?

deanz · wrote on 12 Oct 2010, 12:17

Yes, I redirected the output text from the test script to stderr and Qt Creator then printed the text in red instead of black in the compile output pane but still nothing in Build Issues pane.

deanz · wrote on 12 Oct 2010, 12:18

Also, I'm not using QtTestLib; perhaps that is the problem?

tobias.hunger · wrote on 15 Oct 2010, 20:17

deanz: Qt Creator parses the compile output to generate the build issues. It does not expect unit test results when building, so it does not parse them and thus nothing ends up in the build issues.

Sorry, there is no workaround available that I am aware of. We want to add proper unit test integration, but so far nobody found the time to actually do it:-/

deanz · wrote on 15 Oct 2010, 20:56

Thanks, Tobias.

tobias.hunger · wrote on 16 Oct 2010, 15:58

Hmmm... thinking about this: With a bit of hacking you could get your test results displayed in the build issues:-)

I added a TaskList plugin a while back which grabs a file in CSV format and puts it into the build issues plane. So maybe you can wrap your unit test runner into a script that turns the output into such an CSV file... Creator does watch the file and updates the build issue pane whenever the file changes.

Here is my "announcement mail":http://lists.trolltech.com/pipermail/qt-creator/2010-August/007574.html with some information on the syntax accepted. I think this information is actually already in the documentation, but I am not sure where;-)

deanz · wrote on 18 Oct 2010, 12:31

Great!

Thanks again, Tobias.

I can easilly change my unit tester output to CSV format and your plug-in can take it from there. Looks like problem solved.

deanz · wrote on 19 Oct 2010, 19:03

Tobias:

What is the name of your new plugin? How do I download and install it?

tobias.hunger · wrote on 20 Oct 2010, 07:54

deanz: It is called TaskList and is included in Qt Creator 2.1 beta. Sorry, I should have mentioned that:-)

Oh, for now you need to open the .tasks file (once) via File->Open. When opening it any other way you get it displayed in an editor:-(

deanz · wrote on 20 Oct 2010, 12:11

OK, thanks again, Tobias. I will wait for the 2.1 release but it sure sounds like a very useful addition to an already excellent product.

tobias.hunger · wrote on 20 Oct 2010, 13:03

Yes, writing that plugin was 2h well spend;-)

Actually I only did it since I wanted to get our count of krazy (some static code analysis tool) issues down and was too lazy to hop to each place manually. Good to hear that it works for other use-cases as well;-)

tbscope · wrote on 25 Oct 2010, 06:17

bq. Actually I only did it since I wanted to get our count of krazy (some static code analysis tool) issues down

Ohh pooh :-(
Just when I'm about to finish my own Static Code Quality plugin :-)

tobias.hunger · wrote on 25 Oct 2010, 07:44

tbscope: Being able to hack up a script to import an arbitrary list of issues into Creator is not exactly a "static code quality plugin":-)

I am looking forward to see your plugin! I am sure it will be way more user friendly than my hack.

markus.liebe · wrote on 13 Jan 2011, 12:35

[quote author="Tobias Hunger" date="1287579812"]Actually I only did it since I wanted to get our count of krazy (some static code analysis tool) issues down and was too lazy to hop to each place manually. Good to hear that it works for other use-cases as well;-)[/quote]

Hi Tobias.
we plan to use your plugin to display all the tasks generated by a static code analysis tool.
It works great, there is just one thing left:
For each task there exists documentation in the form of a html file. We want to be able to somehow open the html file via some handy action (right click on the task->open documentation or sth.)

Do you think it would be possible to add such a functionality to your plugin?
Well perhaps it is not only your plugin that would have to be extended but also the BuildIssues Implementation...

Regards,
Markus

tobias.hunger · wrote on 13 Jan 2011, 13:38

Hi Markus!

There is no such code in creator yet.

We do have the concept of TaskHandlers though: Just create one that can accept your task and then displays the help if it is able to match the task to one of your HTML files.

This should be very easy to implement in a plugin. There should be no need to meddle with existing Qt Creator code for this.

Best Regards,
Tobias

markus.liebe · wrote on 13 Jan 2011, 13:48

Thanks for your response.
I just finished a TaskHandler that does what I want =) Now the next step is to put this into a plugin.

I have to say, I really like the QtCreator code. Each time I need some extra functionality it is very easy to do that. The codebase is very friendly towards extending it!

tobias.hunger · wrote on 13 Jan 2011, 14:33

Took you less than 11min to write that TaskHandler... I am impressed!

Thanks for the compliments on our code. Positive feedback is always appreciated:-)