I was chatting with Hani last night and two interesting ideas came up:
- Repeating failed tests.
When a test run shows several failed tests, the first thing that
developers want to do is rerun only those tests that failed, and not the
entire suite. An easy way to implement this with TestNG is to generate
a testng-failed.xml file that contains all the methods that failed.
If you need to rerun these failed tests, just invoke TestNG with this XML
file and you are done.
The only challenge here is that this file also needs to include all the
methods that the failed tests depend upon, but that’s pretty easy since
TestNG already supports this.
This feature also illustrates the importance of separating your test static
model (business logic) from the runtime model (which tests are run).
If you wanted to provide a similar feature with JUnit, you would need to
generate some Java code creating a TestSuite containing all the failed tests
(making sure you get the imports right, etc…) and compile this suite, or
by generating an ant file with the correct decorator.
- Concurrent testing.
The original idea was to be have the testing framework invoke a test method
several times with concurrent threads, but I’m thinking that this should be
extended to entire groups. You put all your methods in a certain group and
you specify that this group should be run by a thread pool a certain number
I think the methods being invoked concurrently would not
necessary contain the asserts and that the real testing would be done by a
method later (which, obviously, will depend on that group).
Another thing is that this specification is a runtime concern, so I think it
belongs in testng.xml, not in the annotations.