I have been adding a steady amount of improvements to the TestNG Eclipse plug-ins over the past weeks, so I thought I’d stop for a minute and put together a quick summary of the upcoming features.
Better tree
The first improvement is the appearance of the tree. You will notice that it now mirrors faithfully the hierarchy that is found in testng.xml: Suites, Tests, Classes and then Methods. Failures are captured in the “Failed Tests” tab for easier look up:
This tab allows you to focus on the tests that fail and ignore all those that passed.
Search
Another new feature is the Search box. When you have hundreds of tests running, finding a specific one is not always easy, so you can now type a few letters of the test method or its parameters and the content of the tree will automatically narrow down to methods matching your search. Note in the screen shot above that the search also works on parameters provided by @DataProvider.
Summary tab
There is now a new tab called “Summary”. This tab gives you statistics on your test run, such as the timings, the test names, the number of methods and classes, etc… Since the results are shown in a table, you can also sort on any criterion you like for easier parsing. This is especially handy when you are trying to determine what tests take the longest time.
Of course, the search box works in this view as well, and note that in the screen shot below, the Time column is sorted in decreasing order:
This view is still a work in the progress and I’m planning to add more information there in the future, such as statistics on individual classes, time line, etc… Let me know if you have suggestions.
Better conversions from JUnit 3
I also improved the automatic conversion of JUnit tests and added support for JUnit 4 conversions. The screen shot above shows how deep the JUnit 3 quick fix now goes. Notice that your code is now correctly updated so that the converted file will compile without additional changes on your part.
… and JUnit 4
When I started writing the conversion fixes, I made support JUnit 4 a non-goal since I was under the impression that developers would be converting from JUnit 3 but that JUnit 4 users would probably not see the need to convert to TestNG.
Well, I was wrong, and since I started receiving requests from JUnit 4 users to help them convert over to TestNG, I went ahead and added quick fixes for JUnit 4 classes. The screen shot above illustrates this.
There are also a lot of other more subtle changes, but I’ll stop here. With these improvements, I’m hoping that the TestNG Eclipse plug-in will become even more appealing as the central hub for all your testing needs.
I will be pushing these new features to the update site very soon.
#1 by Mohamed Mansour on November 2, 2010 - 7:34 pm
droools (cleanly)! Awesome work Cedric
Je suis très content!
#2 by Jonathan O'Connor on November 2, 2010 - 11:18 pm
Wow. Your UI is easily the best I’ve seen for testing. I hope JetBrains steals your idea for RubyMine, which is now my everyday dev tool.
#3 by João Assunção on November 2, 2010 - 11:47 pm
Very cool!!!
Thank you Cedric
#4 by Michał Gruca on November 3, 2010 - 12:58 am
Hi Cedirc.
Great that plugin is developed and getting better
Taking occasion, I’ll request small feature. Is it possible to turn off pop-up console view after running tests? I’m preferring just your ng view for summary which test passed and which haven’t
Thanks for plugin
Michał
#5 by Ruslan Zenin on November 3, 2010 - 4:46 am
Cedric,
Great news. Another reason to use TestNG…its always cutting edge and innovative!
Would be nice to place in the top of Summary tab some useful statistics like this example:
Tests: 26, Failures: 9, Errors: 13, Success rate: 15.38% Time: 16.422
Looking forward for this update!
regards,
Ruslan
#6 by Eugene Kuleshov on November 3, 2010 - 7:40 am
Nice quick fixes.
But I think those tabs are using too much space. I think I’d still prefer JUnit’s “Show Failures Only” toolbar button. As for Summary tab, I’d say it belong to its own view, which could be linked to selection in the test tree.
And above all, the biggest improvement would be show the stack trace into the Console view instead of showing it in the split panel. That would allow to use screen real estate more efficiently.
Finally, we really need a generic view for various test runners. For instance there is already plugins that shows Hudson test results in regular JUnit view. So, why can’t we have the view extended with those nice TestNG features and have one thing to rule them all?
#7 by José on November 3, 2010 - 11:11 am
Everything just looks great ! I’ve been using TestNG for several years now, and all those improvments in the Eclipse plugin look very nice.
I’m also using Netbeans more and more, mainly because of the great JEE6 built in support, but the TestNG plugin for Netbeans is not really on par with the Eclipse one, do you know of any plan to improve it also ?
Anyway, many thanks to you for this amazing tool !
J.
#8 by Frankie on November 4, 2010 - 9:23 pm
That’s great! Thanks for bringing it to us.
However, one question: are there any ways to show all ignored test cases? The reason is that sometime we want to take a look at all ignored test cases and to see why they are ignored and try to fix them. But it seems that it was very hard.
Frankie
#9 by raszi on November 5, 2010 - 3:37 am
unfortunately the new version could not work with a vertical pane. the failure exception is overlapping with the test tree.
and I could not downgrade to the previous version because it was deleted from the site
Pingback: Developing in the meta « Otaku, Cedric's blog
Pingback: One click test conversions « Otaku, Cedric's blog
Pingback: Announcing TestNG 6.0 « Otaku, Cedric's blog
#10 by Chris J. on April 17, 2011 - 5:06 pm
I’m somewhat disappointed with V6 of the plugin compared with v5.4 with the biggest issue for me being the fact that I can no longer see the test suite progress on a method by method basis. The only progress indicator is the single bar that applies to all tests globally at the top. The earlier plugin version allowed you to see the test results dynamically as they proceeded. Maybe I’m missing a button or a switch to turn that on? If not, please point me to the prior version download link so I can reinstall that. Thanks.
#11 by Cedric on April 18, 2011 - 1:42 pm
Hi Chris,
This is a compromise I had to make in order to improve the performance the plug-in. Until I figure out exactly how to update the tree live without impacting the overall performance, I just pushed a fix that only updates the tree with the failures. The passed tests will be added in bulk at the end of the run.
#12 by Chris J. on April 18, 2011 - 2:32 pm
I really appreciate your quick response and action, Cedric. That update works just fine for me. Thanks!
#13 by Patricio on July 19, 2012 - 12:43 am
Hi Cedric, great plugin. I’m using TestNG for the first time (instead of JUnit) for a ATDD Project (using Selenium). The plugin is great! I have seen that @Test annotation (in the scope of a method or the test class) has a testName parameter. I wonder if this attribute may be used instead of the full package or the test method name, in order to present the tests in a colloquial way. Is this logical? We think that is important that every person (technical or not) can see the results and understand.
Thanks a lot!