October 30, 2006
Announcing TestNG 5.3
I'm pleased to announce the immediate availability
of TestNG 5.3.
The main feature in this new release is the introduction of Annotation
Transformers, that I described in a previous
post. The full documentation for Annotation
Transformers can be found here.
As you can see below, the change log is quite big as well, so
if we failed to fix a bug you reported or an improvement you requested,
please let us know and we'll try to get it in for the next release!
Thanks to all who helped make 5.3 a reality!
Core
- Fixed: use a single instance of bsh.Interpreter
- Added: @Before/@AfterMethod can declare a java.lang.reflect.Method parameter to be informed about the @Test method
- Fixed: super classes must not be listed in testng-failures.xml
- Fixed: parallel attribute must not appear if empty or null in testng-failures.xml
- Fixed: parsing for javadoc annotations is done on request only
- Added: improved multiple suite summary page report
- Added: -target option deprecated in favor of -annotations javadoc|jdk
- Fixed: filesets in the ant task didn't work if the paths have spaces in them
- Fixed: Before/After Suite were behaving wrong in parallel execution
- Added: A generic/extensible RemoteTestNG was added to the core
- Fixed: Before/AfterGroup-s were behaving wrong when using invocationCount, dataProvider and threadPoolSize
- Fixed: improved support for running different annotation type tests in the same suite
- Fixed: testng-failed.xml was generated even if there were no failures/skipps
- Fixed: -usedefaultlisteners was wrongly passed to JVM instead of TestNG options
- Added: Attribute dataProviderClass for @Test and @testng.test
- Fixed: Forgot to account for cases where both invocationCount and DataProviders are present
- Fixed: AfterGroups were invoked out of order with invocationCount and DataProviders
- Fixed: Reporter.getOutput() returned an empty array if a timeOut was specified
- Added: testng.xml now supports <suite-files>
- Added: ant task can receive several listeners
- Fixed: TESTNG-109 Skipped tests with expected exceptions are reported as failures
- Added: ant task can now select the parallel mode for running tests
- Fixed: ant task correctly deals with empty groups and excludedgroups parameters
- Added: ant task can override default suite and test names
- Added: comand line support for setting parallel mode, suite and test names
Eclipse plug-in
- Added: Support for configuring per project usedefaultlisteners
- Added: Contextual drop-down menu on failures tab of the TestNG view to enable running/debugging method failure only
- Added: Suppport for configuring per project TestNG jar usage (project provided one or plugin provided one)
October 29, 2006
A view from below...
Click on the image for full size (2 megs)
An underground subway runs between the two major Alliance cities Stormwind and Ironforge. For a long time, I noticed that this Tram goes through an underwater section about halfway between the two cities, but I never bothered to find out. It always intrigued me, though, so a few days ago, I took the long walk to see for myself, and I wasn't disappointed...
Through the
waving reflections of the force field that keeps the water from
flooding the tunnel, you can see a cute Deep Run Diver walking on the
sandy bottom, a sunken pirate ship and a mermaid. But the
best is on the other side of the track...
And as for all World of Warcraft screenshots, I strongly recommend to click on both of these pictures, they are worth it...
October 18, 2006
Annotation transformers in Java
Ever since annotations officially made it officially in the Java language, the debate has been raging about where configuration should go: in annotations or XML files.
The pros and cons of each approach are easy to summarize: annotations allow you to put your configuration system close to the Java source it applies to, but changing it requires Java knowledge and a recompilation. On the other hand, XML files are usually easy to modify and they can be reread at runtime (sometimes even without relaunching your application), but they are very verbose and the edition can be error prone.
As usual in such situations, the correct course of action is to use a mix of these approaches. For example, a rule of thumb that I have been following is that "any configuration parameter that refers to a Java element (package, class, method) should be specified in an annotation" (e.g. @Test or @Transaction) and "any configuration parameter that applies to your application as a whole probably belongs to an XML file" (e.g the name of an output log file or a target web server).
Another approach that blends these two techniques has also been under scrutiny recently: being able to override Java annotations with XML files. I haven't come across an implementation of this idea that can claim it's the defacto standard, and at this point, I suspect that this approach is unlikely to take off because it suffers from the "worst of both worlds" syndrome.
Here is why.
Consider the simple following code:
public class Mytest {
@Test(invocationCount = 10)
public void verify() {
// ...
}}
This instructs TestNG to invoke the verify() test
method ten times.If I want to override this value at runtime, I find myself having to write a fairly complex piece of XML code in order to pinpoint the exact annotation I'm trying to override. It would probably look like this:
<override-annotation> <package name="org.foo.tests"> <class name="MyClass"> <method name="verify"> <annotation name="org.testng.Test"> <attribute name="invocationCount" value="15" /> </annotation> </method> </class> </package> </override-annotation>Of course, you might want to come up with a way to capture the override (the annotation/attribute part) so you can reuse it somewhere else in your XML file, in case you want this override to apply to more than just one method. This would probably be achieved with the definition of an "override-ref" that you would define at the top of your XML file and that you would repeatedly use throughout your XML file (exactly like ant's classpath-ref, for example).
That's already a lot of work (and it's quite hard to read), but it's also extremely fragile since it will most likely break if you decide to rename your class or your method name (IDE's are beginning to extend refactorings to non-Java files, but we're not quite there yet).
Because of these shortcomings, I have been working on a slightly different approach with TestNG that lets you specify this runtime override in Java. Again, it's not a silver bullet and suffers from some of the compromises expressed above, but if you can live with the idea that you have to recompile your override if you modify it (but not the annotated code you are trying to override, so it's already a progress), it's actually fairly simple to achieve.
TestNG introduces the following interface:
public interface IAnnotationTransformer {
public void transform(ITest annotation, Class testClass,
Constructor testConstructor, Method testMethod);
}
and you use it as follows:public class MyTransformer implements IAnnotationTransformer {
public void transform(ITest annotation, Class testClass,
Constructor testConstructor, Method testMethod)
{
if ("verify".equals(testMethod.getName())) {
annotation.setInvocationCount(15);
}
}
}
Here are a few observations about this technique:- The idea is that whenever TestNG parses a @Test
annotation,
it will run it through the annotation transformer before using it.
- The three extra parameters of the method let you know where
this annotation was found (on a class, a constructor or a method).
Only one of these three parameters will be non-null.
- Notice that the parameter to the transform()
method is an
ITest. This is because TestNG
supports both
standard and JavaDoc annotations, so ITest is a
simple
façade object that represents the @Test
(JDK) or
@testng.test (JavaDoc) annotation.
In a more general
framework, the parameter would probably directly be the type of the
annotation, although there are still a few problems to solve with this
(see the next point).
- This approach requires write access to the annotation, which is not possible with the current JDK implementation. You can work around this limitation by using a façade, which could possibly be generated as a dynamic proxy.
Annotation Transformers are alreay implemented in the TestNG Subversion depot, and they will be available publicly when TestNG 5.3 comes out (very soon).
October 09, 2006
FireFox 2: Please fix this before it's too late
Compare the 1.5 version:
This is very wrong for two reasons:
- The close button now changes places all the time. I just never know where
to find it: I need to first find what tab is currently highlighted (not very easy to do with MacOS because the difference between the active and inactive tabs is subtle) and then click it. After two weeks using FireFox 2, I still find myself closing the wrong tab once in a while. Actually, I've sometimes even closed the tab I wanted to jump to by accident because I clicked on its close button when I only intended to activate it...
- More than once now, my muscle memory have led my mouse cursor toward the rightmost close button, thereby expecting to close the tab I'm currently reading while, of course, closing one I haven't read yet.
Here is the rationalization offered on the Mozilla Web site:
Power users who open more tabs than can fit in a single window will see arrows on the left and right side of the tab strip that let them scroll back and forth between their tabs.Well, sorry, but that doesn't hold water, and I'm sure that if they had conducted even only one usability study, they would have found that there is a vast majority of users who just think the older UI is more functional.
But I'm open-minded, and I certainly understand the value of evolving user interfaces over time, but when you are altering something that is so fundamental to basic navigation, the least you can do is offer a switch to users who prefer the older behavior (like Eclipse wisely did). Unfortunately, such a setting is nowhere to be found in the Tabs configuration.
I hope the FireFox developers do the right thing and introduce this setting before 2.0 goes final.
October 06, 2006
TestNG is getting even more Groovy
The indefatigable Andy Glover strikes again with his copacetic post "TestNG is so Groovy".
We all know that Groovy compiles to standard .class files, and in this post, Andy comes up with a technique that, from now on, will let him write his tests in any combination of
- TestNG / JUnit and
- Java / Groovy
Write once, test anywhere!
October 03, 2006
Mac OS background madness
I like simple, uncluttered desktops, and I realized a long time ago that I was among a very small group of users for doing so. Most of the computer background desktops that I see are usually filled with documents, links and programs to a point that I wonder how these users ever find what they're looking for.
Well, I'm sure they find their documents eventually, but do they really visually search through the rows of 10x10 icons on their desktop to do so, or do they rely on the Explorer or Google Desktop for this? On top of this, most users typically have the picture of a sunset, a waterfall or their sons playing soccer, further degrading the readability of whatever text they used to name their documents.
If you have more than twenty icons on your desktop, I am betting that you are only clicking on four or five of them regularly.
For all these reasons, I like to keep my desktop relatively uncluttered. All the programs are neatly gathered in the Quick Launch bar and the few icons that you can see on my background are usually temporary files that I need to refer to either a lot, or just for the current week.
With that in mind, I recently set out to modify my Mac Book Pro desktop in order to show my favorite background: solid black. And I mean black as in 0x000000, which is the only background color that allows me to identify icons and texts without any impediments.
So I right-clicked on the background, selected "Change Desktop Background..." and I was presented with a dialog box offering me various options, among which "Solid colors", which I promptly selected. The dialog shown above appears, and here is my first surprise: it looks like I have a choice of ten preset backgrounds, but solid black is not one of them. "No problems", I think, I'll just find the color chooser and I'll set the color myself.
Problem is... there is no such thing.
That's right: no color chooser.
Mac OS X won't let you set your background to an arbitrary solid color: you have to choose one that Apple decided for you. And by the way, did you notice the dotted rectangle above? It represents a solid white background. The trouble is... this dotted line is not shown in reality (it was added manually to this picture). The background is there for you to select, but it's completely invisible because shown as white on white. How silly is that?
Anyway, back to my problem.
I can't select my own background color. That's pretty retarded. You know, I can almost imagine what went on in the meeting room when this decision was made:
Steve Jobs: "Okay, I want the following ten choices for background colors: dark gray, solid aqua blue, ...."
UI designer: "Great, we'll give the user a quick way to select the colors that work best with our Aqua theme. Where shall we put the color chooser?"
"There shall be no color chooser."
"Sir?"
"There shall be no color chooser."
"But... What if they don't like any of these preset colors?"
"I don't care. The Mac OS desktop is the quintessence of beauty. I don't want any of our tasteless users to ruin it with their own crappy colors."
<uncomfortable silence, then forced grin> "Ah... good one sir... Very funny. Now, seriously...?"
"I was being serious."
So you can't choose your own background color on Mac OS. That's the way it is. Of course, there are workarounds, such as creating a 1x1 image and use it as a tiled background as explained here, but if I hear one more time that Mac OS is the most advanced desktop currently, I'll simply challenge the zealot to set the background color of their desktop to solid magenta and watch her suffer in agony while I enjoy my black Windows desktop which took me all of three seconds to configure.
October 01, 2006
Dynamic language, refactoring IDE. Pick one.
In a recent entry, Tim Bray revisits the old debate about refactoring dynamically typed languages:
IDEs all have a “Find Usages” feature, which in itself is handy (“I’m thinking of nuking this stupid method, where is it used?”) but if you think closely, is also vital too pretty well every other refactoring operation. If you’re going to rename a method or a class or extract an interface, it’s absolutely vital that the IDE know all the places where that code is used; otherwise refactoring equals breakage.As always in this kind of debate, a commenter was quick to point out that "Smalltalk already did it":
Of course, the problem is harder with dynamic languages, because variables aren’t declared, and because you can screw around with the insides of classes at runtime,
As with everything else: it's all be done before:First of all, that commenter is wrong. All the Smalltalk IDE did for renaming was search and replace, which is, frankly, the best it could do. And clearly unacceptable today.
Smalltalk. Infact refactoring was pioneered (like many other things, oop, gui, unit testing, agile/xp ...) in Smalltalk!
Smalltalk has 30 years of experience making the Ruby object model fast.
It's simple, really: dynamic languages that are not statically typed (i.e. let you get away with not typing variables) simply *cannot* do certain refactorings, among which "renaming".
How could they? They just don't have the type information to know exactly which types are being used to invoke your methods.
A few months ago, I offered the following code snippet to the author of the Ruby Refactoring Browser:
def f1(o)
o.init
end
def f2(o)
o.init
end
class C
def init
...
end
end
And I asked him: "If I rename C.init to C.init2, how do you know which o.init must be renamed in f1 and f2?".
His response was unequivocal:
This problem is difficult for dynamically typed language. I think computer can't determine whether those must be renamed or not.This shouldn't come as a surprise. One of the main reasons why people like dynamically typed languages is precisely because they don't have to specify these types. Is it really so shocking that this convenience comes at the price of not being able to perform any operation that requires type information on this code?
Therefore Ruby Refactoring Browser provides two functions, one is renaming all methods that have same name, and another is renaming only methods and calls that cleary belong the class. The former renames o.init in f1 and f2, and the latter doesn't rename them.
As for Tim's solution to this problem, which consists in having the IDE gain this knowledge by running your code instead of analyzing it statically: it's clearly not going to work either.
In order for this approach to work, you would need to have 100% coverage with your tests which, I claim, is probably achieved by 0.001% of the projects out there, all of which are simple "Hello world" programs. And without this, who wants an IDE that performs a correct refactoring "most of the time"?
So is there any way out of this conundrum? I think there is: dynamic languages that allow you to type your variables if you feel like it, and the only language that I can think of that does that at the moment is Groovy. This is clearly the best of both worlds, because you only need to type your variables once (i.e. in your method signature) and type inference will save you the trouble of duplicating this information from that point on.