Groboclown and I have had an interesting exchange about suite() on his weblog.

The idea of using JUnit ant tasks to work around the limitations of JUnit is sadly very common, to the point that nobody notices even more that this is a fundamental flaw in JUnit. In this particular example, you can use <batchtest> to invoke explicitly several Test classes (something you can’t even do with the basic TestRunner!) and also to use ant’s powerful wildcard matching.

This is useful, but it has two limitations:

  • It requires ant. In heterogeneous environments that build more than just Java code, ant is not always an option. Besides, I really think that such features should be available at the simple Java level.
  • Your level of granularity is still the class, not the method.

This last point is important. I find myself very often needing to run an individual test method, either to track a test that fails or more simply while I am coding the said test. The only way you can achieve this with JUnit is by commenting out all the methods in your class that you want JUnit to ignore (actually, I was renaming them “_test”). And then of course, you need to recompile your test.

That’s a lot of work when all you need is to tell your test framework to run a specific test method.

Groboclown quotes a book on this topic:

In Testing Objet-Oriented Software: Life Cycle Solutions, by Imran Bashir and Amrit L. Goel, they argue that the unit of test has moved from the function (in structured programming) to the class (in object-oriented programming). Using this argument, I’ll claim that there’s no need to get a lower granularity than at the test class level.

I disagree. The class level still fails to capture three very important levels of granularity in testing:

  • Individual methods.
  • Groups of methods.
  • Sequences of methods that depend on each other.

These notions should look very familiar to you: they are at the very core of build tools such as ant or make.

You declare dependencies between targets and you can group these dependencies together, creating an acyclic graph of clusters representing your run sequence where some processes are run sequentially while others can be run in parallel.

Tests are no different, and whenever you use a framework that doesn’t give you this flexibility, you are making your life unnecessarily hard.