March 29, 2008
Test method priorities in TestNG
A TestNG user recently requested an interesting feature: method priorities.
The idea is to be able to assign a priority to a test method (or an entire class) to guarantee that it will be run before other methods with a higher priority. Note that this is slightly different from dependent methods, since it's just about ordering: if a method with a priority 2 fails, methods with higher priorities will still be run.
As it turns out, this is very easy to implement with one of TestNG 5.8 newest features: method interceptors.
TestNG lets you declare a class implementing the interface IMethodInterceptor, whose signature is:
public interface IMethodInterceptor extends ITestNGListener {
List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context);
}
TestNG passes the list of methods it's about to run so you get a chance to reorder them. Implementing method priorities with this interface is very simple.
Let's start with what we want this feature to look like. In the following example, I declare three kinds of priorities:
- Priorities at the method level.
- A priority at the class level.
- No priority. When no priority is defined on a method, we want to look for a priority on the class first, and if none can be found, the priority will receive a default value of 0.
@Priority(10)
public class PriorityTest {
@Test
public void b1() { System.out.println("Default priority");}
@Priority(-3)
@Test
public void a1() { System.out.println("Priority -3");}
@Priority(-3)
@Test
public void a2() { System.out.println("Priority -3");}
@Priority(3)
@Test
public void c1() { System.out.println("Priority 3");}
@Priority(3)
@Test
public void c2() { System.out.println("Priority 3");}
@Priority(4)
@Test
public void d1() { System.out.println("Priority 4");}
}
Next, we implement the @Priority annotation:
@Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
@Target({METHOD, TYPE})
public @interface Priority {
int value() default 0;
}
Fairly straightforward: we make the annotation valid on methods and types only, and we use the shortcut value() annotation so we can specify @Priority(1).
Now comes the implementation of the IMethodInterceptor:
public class PriorityInterceptor implements IMethodInterceptor {
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
Comparator<IMethodInstance> comparator = new Comparator<IMethodInstance>() {
private int getPriority(IMethodInstance mi) {
int result = 0;
Method method = mi.getMethod().getMethod();
Priority a1 = method.getAnnotation(Priority.class);
if (a1 != null) {
result = a1.value();
} else {
Class cls = method.getDeclaringClass();
Priority classPriority = cls.getAnnotation(Priority.class);
if (classPriority != null) {
result = classPriority.value();
}
}
return result;
}
public int compare(IMethodInstance m1, IMethodInstance m2) {
return getPriority(m1) - getPriority(m2);
}
};
IMethodInstance[] array = methods.toArray(new IMethodInstance[methods.size()]);
Arrays.sort(array, comparator);
return Arrays.asList(array);
}
}
The heart of this implementation is our Comparator, which tries to find a @Priority annotation the method it receives in parameter. If it can't find any, it looks on the declaring class, and failing to find one there as well, assigns it a default value. The intercept() method simply sorts the methods with this comparator and returns it.
Now, let's run our test without using that interceptor:
<suite name="Example">
<test name="Method interceptor example" >
<classes>
<class name="priority.PriorityTest" />
</classes>
</test>
</suite>
As expected, the output is fairly random:
Priority 3 Default priority Priority 3 Priority -3 Priority -3 Priority 4Now we declare our interceptor in the testng.xml file:
<suite name="Example">
<listeners>
<listener class-name="priority.PriorityInterceptor" />
</listeners>
<test name="Method interceptor example" >
<classes>
<class name="priority.PriorityTest" />
</classes>
</test>
</suite>
And we get the expected outcome, with our method without any @Priority receiving the class priority (10) since it didn't define any itself:
Priority -3 Priority -3 Priority 3 Priority 3 Priority 4 Default priorityTo verify that our class priority implementation works, we can remove the @Priority class annotation. In this case, our method without any @Priority should receive a priority of 0:
Priority -3 Priority -3 Default priority Priority 3 Priority 3 Priority 4As you can see, Method Interceptors allow you to replace TestNG's ordering of test methods with your own. It's particularly powerful if used in conjunction with annotations, as shown in this example, but it's also possible to imagine other external configuration factors to customize TestNG's running engine to your liking.
March 23, 2008
PowerPoint feature request
I'm currently writing a brand new presentation (more on this later) and I find myself in need of a feature I had never thought of before.
I'd like to create two versions of this presentation: one that fits in forty-five minutes and one in two hours, the longer version being a superset of the shorter one: it just contains a few additional slides here and there.
Is there any other way for me to do this besides having two separate files with a lot of slides duplicated?
Ideally, I'd like to be able to put my slides in groups: some will be in the "short" group and other will be in both the "short" and "long" groups. When I start the presentation, I tell the software which group I am showing today (this idea will undoubtedely sound familiar to TestNG users :-)).
The closest I could find in PowerPoint is by hiding/showing individual slides, which is really impractical.
Does anyone have a better solution with either PowerPoint, Keynote or even Google Presentation?
March 03, 2008
TDD leads to an architectural meltdown around iteration three
These are the words from software guru Jim Coplien, who penned some of the most influential C++ books in the 90's. The exact quote is:
TDD done strictly from the YAGNI principle leads to an architectural meltdown around iteration three.It's very refreshing to see the anti-TDD movement gather up some momentum, and I particularly enjoyed the discussion between Jim and Rob Martin, the notorious TDD extremist who thinks that every software developer who doesn't TDD every single line of code she writes is unprofessional.
Jim and Bob had an interesting debate recently, which you can find transcribed at InfoQ. I have to give credit to Jim for his civility as he faces Rob, whose arguments never suffered much tolerance for people who don't agree with his views.
I have been bothered by TDD for a long time now, and my skepticism and even opposition to some of its effects can be found either in the presentation I made with Alexandru at InfoQ called Designing for Testability or more recently, in the book I co-authored with Hani Suleiman.
Having said that, don't expect me to become Rob Martin's counterpart. I certainly don't believe you should never use TDD, but I do believe that the benefits associated to TDD are vastly exaggerated and that the software community needs to keep in mind that TDD is just a tool, and that as a tool, it's not adapted to all situations. I especially dislike the efforts of TDD extremists who are trying to make developers feel bad whenever they don't use TDD, or lead them to think that something is wrong with the way they write their software.
If anything, it seems to me that most of these TDD extremists have been talking to conferences for too long and have become way too comfortable writing Stack classes or code that calculates the score of a bowling game. These toy applications are easy and they make TDD shine, but don't be surprised if your audience leaves the room scratching their heads and wondering how they can apply it to their real job. Good luck using TDD to write a mobile application or to interface with a mainframe that needs to handle hundreds of thousands of two-phase commit transactions spreading over three continents while thread contention remains to a minimum.
I don't know about you, but I'm getting a bit tired of fear mongering in the software community, whether it comes from TDD fanatics or from people who claim they wouldn't hire someone who doesn't use a Mac for development.
When it comes to testing, I live by the following rules of thumb:
- "Tests first" or "tests last" is unimportant as long as there are tests.
- Try to think about testing as early as possible in your development process.
- Don't let one liners contradict your experience. For example, don't listen to people who tell you to write "the simplest possible thing that could possibly work", also known as YAGNI. If your experience tells you you're going to need this extra class in the future even if it's not needed right now, follow your judgment and add it now.
- Keep in mind that functional tests are the only tests that really matter to your users. Unit tests are just a convenience for you, the developer. A luxury. If you have time to write unit tests, great: they will save you time down the road when you need to track bugs. But if you don't, make sure that your functional tests cover what your users expect from your product.
- Don't feel bad if you are not doing Test-Driven Development. There are a lot of factors that make this practice a bad fit for a lot of projects and developer personalities (aspects which are very often never mentioned by TDD extremists).
Don't let extremists make testing a chore, because it can truly be one of the most rewarding aspects of our profession if you exercise it with judgment.