Otaku, Cedric's weblog

JUnit 4 overview

It appears that Kent Beck has been working in stealth mode on JUnit 4, and if it wasn't for a small slip in an interview a couple of months ago, nobody would know.  It turns out that there already is some new code checked into their CVS depot and Alexandru and I took a quick look there.

Here is what we found so far:

• Annotations.  As we observed with TestNG, annotations let you do away with inheritance of a base class and make more sense than trying to cram test information into method names.
   
• The @Test annotation has an expected attribute (probably similar to @ExpectedExceptions, but it's hard to tell at this point)=.  I also note they only allow one class to be expected while TestNG lets you specify more than one).
   
• There is a new @Ignore annotations, which we decided to specify as an attribute of @Test in TestNG (e.g. @Test(enabled = false)).
   
• TestNG's timeout and invocationCount also seem to be planned for JUnit.
   
• Finer grained configuration access.  JUnit only allowed to setUp/tearDown around test methods, you can now do this at the suite level (TestNG offers even greater flexibility: around classes and around groups).
   
• Suites are being deprecated and they are looking for a way to describe collections of tests (quite a surprising decision, but may I suggest a file called...  I don't know, junit.xml?).
   
• There will be a converter (I invite the JUnit team to start off with com.beust.testng.JUnitConverter and tweak the imports it generates :-)).
   
• Run failures first (TestNG generates a testng-failures.xml for this).
   
• Quoted from the todo file:  "be able to specify the order of tests (ewwww....)".  Now that is very interesting...  They are just falling short of specifying test dependencies but as much as I find this feature crucial for non-unit testing, I question its use for unit testing.
   
• Groups!  Yes, that's right.  A lot of people criticized the need for groups when they were introduced in TestNG a year ago, but we have definitely seen users like this feature and use it a lot.  It's good to see JUnit follow the same path.

For more information, Alexandru posted a more detailed list of the various features offered by JUnit 4 with his own comments.

A few comments now...

• This is a pretty good start, and I am not surprised to see that dependent test methods are not planned for JUnit (this feature probably doesn't belong to a unit-testing framework anyway).
   
• I didn't notice any new tests, so they are not using Test-Driven Development...  makes you wonder, uh?
   
• There is still no separation of static and dynamic models.
  
  Having test groups is not very useful you can't specify which groups to execute at runtime.  I hope JUnit will provide a less awkward way to invoke tests than putting them in a suite() method.  As you probably know by now if you are a regular reader of this blog, I still firmly believe that I shouldn't have to recompile Java code if I want to run a different set of tests, which is the reason behind the existence of the testng.xml file in TestNG.

All in all, this is very exciting but I feel disappointed that no email was sent to the JUnit mailing-list nor request for feedback posted on any blog:  the JUnit team seems to have decided to work behind closed doors on this.

The JUnit mailing-list itself has 5800 subscribers (!) and would probably be an invaluable way for them to get feedback.  I'm not quite sure what is the driving vision behind JUnit 4 (besides TestNG :-)), but I certainly hope they will be coming out of stealth mode and discuss it publicly soon.

At any rate, I am quite happy to see that things are finally moving in the testing area!