Archive for September, 2004

Marillion wrap-up

So I saw Marillion at the Fillmore last night, and despite my doubts, I thoroughly enjoyed the show.

The band used an unusual formula: the show was split in two parts and the first part was entirely dedicated to the new material. They played pretty much the entire Marbles album except for a couple of songs which they saved for the encores. The second part featured some of their standard live songs and unfortunately, they didn’t play any Fish-era songs at all.

I can understand this decision (after all, Fish was only with Marillion for a few years and that was more than fifteen years ago), but there is no doubt that quite a few of the fans present last night would have loved to hear some Fish material. And after all, even Genesis was still covering some Peter Gabriel oldies in their last tours, so this is not unheard of in the prog circles.

The part I liked most was in the encores (there were two). After the first song, Steve Hogarth took he microphone and said:

“We were chatting in the dressing room and we couldn’t decide which song we were going to play now… Afraid of Sunlight, or The Great Escape

Of course, the crowd was quick to indicate its preference vociferously and Steve teased us for a little while, switching from one to the other and back. Finally, he shrugged and said:

“Okay, fine. Both.”

The audience went berserk at this point, as you can imagine, and I happily joined in the screaming, as these two songs are definitely some of my favorites. Then he turned to Steven Rothery and asked him:

“What guitar are you using right now?”

Rothery mouthed “A-O-S”, Steve acknowledged and then, Marc Kelly started playing the hypnotic first bars of Afraid of Sunlight… Drive the road… to your surrender…

The Great Escape is always spectacular on stage, especially when Steve’s voice is up to it, and he fully delivered last night. Finally, the band closed the last encore with Easter, which Steve described as “one of the very first songs we wrote together, when I showed up circa 1989″.

A fitting closing act for a fine performance of progressive rock delight.

ant, make and tests: same fight

Groboclown and I have had an interesting exchange about suite() on his weblog.

The idea of using JUnit ant tasks to work around the limitations of JUnit is sadly very common, to the point that nobody notices even more that this is a fundamental flaw in JUnit. In this particular example, you can use <batchtest> to invoke explicitly several Test classes (something you can’t even do with the basic TestRunner!) and also to use ant’s powerful wildcard matching.

This is useful, but it has two limitations:

  • It requires ant. In heterogeneous environments that build more than just Java code, ant is not always an option. Besides, I really think that such features should be available at the simple Java level.
  • Your level of granularity is still the class, not the method.

This last point is important. I find myself very often needing to run an individual test method, either to track a test that fails or more simply while I am coding the said test. The only way you can achieve this with JUnit is by commenting out all the methods in your class that you want JUnit to ignore (actually, I was renaming them “_test”). And then of course, you need to recompile your test.

That’s a lot of work when all you need is to tell your test framework to run a specific test method.

Groboclown quotes a book on this topic:

In Testing Objet-Oriented Software: Life Cycle Solutions, by Imran Bashir and Amrit L. Goel, they argue that the unit of test has moved from the function (in structured programming) to the class (in object-oriented programming). Using this argument, I’ll claim that there’s no need to get a lower granularity than at the test class level.

I disagree. The class level still fails to capture three very important levels of granularity in testing:

  • Individual methods.
  • Groups of methods.
  • Sequences of methods that depend on each other.

These notions should look very familiar to you: they are at the very core of build tools such as ant or make.

You declare dependencies between targets and you can group these dependencies together, creating an acyclic graph of clusters representing your run sequence where some processes are run sequentially while others can be run in parallel.

Tests are no different, and whenever you use a framework that doesn’t give you this flexibility, you are making your life unnecessarily hard.

TestNG's @Factory

I have extolled the virtue of external runtime configuration for testing with TestNG for a while now. More simply put: when you decide to run a different set of tests, you shouldn’t have to recompile any Java code. This kind of information is dynamic and Java is therefore not the best way to express it.

Having said that, there is some undeniable power in the ability to express this configuration from a programming language. JUnit offers this with the concept of Test Suites (and the method suite()), and for this, TestNG has the @Factory annotation.

Imagine the following situation: you have a class that validates an XML file and you want to run this class on several files. The problem is: you don’t know in advance what these XML files are (they might be “all the XML files in this directory”). Obviously, a programmatic approach is needed for this.

Enter @Factory.

public class XMLFilesTest {
@Factory
public Object[] factory() {
List<Object> vResult = new ArrayList<File>();
File[] files = // read files from the directory
for (File f : files) {
vResult.add(new XMLValidatorTest(f));
}
return vResult.toArray(new Object[vResult.size()]);
}
}

Once you have a test class that has a @Factory method, all you need to do is specify this method in your testng.xml:

<suite>
<test name="XML" >
<classes>
<class name="test.sample.XMLFilesTest" />
</classes>
</test>
</suite>

TestNG will pick you your @Factory method, invoke it, retrieve all the objects you returned and consider each of them as a separate TestNG test class.

Interestingly, I initially added this feature to allow tests to provide multiple instances of themselves. This is handy when you are testing something like the reentrancy of your class, or when you want to stress-test a server (write a simple test, create one thousand instances of that class and run TestNG on it). As it turns out, this feature can be used to return any kind of instances, not just instances of class the factory belongs to.

Can you think of other potential uses for @Factory?

Marillion in San Francisco

Marillion is coming to San Francisco next week. Since I moved to the US, I haven’t had a lot of opportunities to watch my favorite progressive rock bands on stage, since most of them are European. So far, only Yes, Camel, Spock’s Beard and the Flower Kings have made it to the West Coast, so I am quite happy to get a chance to watch Marillion on stage again after so many years (last time was ten years ago in France).

Marillion has always had a special place in my musical portfolio, because they were the second progressive band I completely fell for after reeling from the shock caused by the music of Genesis. My first Marillion album was Misplaced Childhood and I was sold right away. Clutching at straws is a solid favorite Fish-era album of mine and it’s hard not to be haunted by the title track of Script for a Jester’s tear.

I was quite sad to see Fish leave and the next two albums that followed with the new singer Steve Hogarth left Marillion fans worried that their favorite band had abandoned them. And then, Brave came out, and it absolutely blew me away. Even to date, Brave is one my top five favorite progressive albums of all times, next to Genesis’ The lamb lies down on Broadway.

The next album, Afraid of Sunlight, was also a real pleasure, but since then, I have to say that Marillion seems to have drifted again away from my personal tastes. This strange engine, Radiation, marillion.com and Anoraknophobis didn’t do much for me.

With their latest album, Marbles, Marillion seems to have come back to a kind of music that I find more inspired and melodious, varying themes both musically and lyrically.

At any rate, I am more than happy to watch them on stage again, even if I feel I have somewhat wandered away from the music they produce these days. They remain an outstanding set of musicians and I am quite looking forward to discovering their new set list (which I didn’t look up, because I like to be surprised).


Listen as the syllables of slaughter cut with calm precision

Patterned frosty phrases rape your ears and sow the ice incision

Apocalyptic alphabet casting spells the creed of tempered diction

Adjectives of annihilation bury the point beyond redemption

Wanna work at Google? Email me.

I’m serious.

If you are reading this weblog, you are probably the kind of developer Google would like to hire, so if you want to get a chance to work here, all you need to do is email me your r

JDK 5.0 in practice, part 2

Autoboxing

I have stayed away from autoboxing so far, probably because I have a vague feeling of losing control of the performance of my code. I don’t think it is justified, though, so autoboxing can come in handy and make your code a little bit more readable. I think I would encourage developers to flag their code when such autoboxing is happening, and I am pretty sure that IDE’s will soon be able to do the same.

Generics

Where to start?

Well, first of all, nobody can dispute that Generics are a solid concept that tends to improve the robustness of your code. The reason why they are so usually controversial regardless of the language is because of their implementation. And for having been a member of the C++ committee for many years , I can definitely vouch for the difficulty of getting them right.

In a nutshell, I have this to say about Java generics: my code feels more robust, but it’s harder to read.

So what’s the problem?

Redundancy.

First of all, I have always had a hard time with the redundancy introduced by the necessity of casting in general. For example, instead of writing:

Map accounts = new HashMap();  // no generics
...
Account a = (Account) accounts.get("Cedric");

why can’t I just write:

Map m = new HashMap();  // no generics
...
Account a = m.get("Cedric");

and let the compiler introduce a silent cast, since obviously, it’s an object of type Account that I am trying to retrieve from the Map?

Obviously, Generics don’t solve this problem entirely but they make a decent job at alleviating it somewhat. But they also make it worse in some other ways:

Map<String, List<Account>> accounts =
new HashMap<String, List<Account>>();

Ouch.

Not only is the code significantly harder to read, but it fails to obey the DRY principle (“Don’t repeat yourself”). What if I need to change the value type of this map from List<Account> Collection<Account>? I need to replace all these statements everywhere in my code. While IDE refactoring will help, it’s still an awful lot of code for a modification of this kind that hardly impacts the semantics of this code.

Admittedly, there is no nice way to avoid this syntax when you are creating a new object, but what I am driving at is that I think Generics would have been better off if typedefs had been introduced with them.

Or so I thought at first.

But after thinking about it more, I realized that typedefs were the wrong solution to the problem, because simply put, they don’t add anything to the use of a separate class to define your complex Generic type.

class AccountMap extends HashMap<String, List<Account>> {
...
}

Except for the fact that you need to extend an implementation (HashMap, and not Map, obviously), this solution is probably better than introducing typedef, which has its own quirks.

I haven’t gone to this trouble so far, but my recommendation would be: do it if you write the type more than three times (twice in the initialization and you use it more than once in your code).

Except for this little annoyance, I am quite happy with Generics overall and I particulary enjoy reading the TestNG Javadocs so nicely typed.

Conclusion

I am very happy with the new features of JDK 5.0 and I am quite proud to have had a chance to influence it with my participation in JSR 175 and JSR 201. Like all radical evolutions, not all of the new features will be popular with everyone, but as long as most developers find some of these features useful and that backward compatibility is preserved, I think JDK 5.0 is a very solid step toward more solid Java code.

JDK 5.0 in practice

I have been writing JDK 5.0 code for over six months now, so I thought I would take some time to reflect on my experience and draw a few conclusions on the features that were introduced.

Enhanced for loop

The undisputed winner. I can’t even begin to describe how good it feels to use the new for loop everywhere (well, almost everywhere). I mentally cringe the few times when I am forced to use the old for loop, typically when I need the index or that I want the Iterator to be visible outside the loop.

The code is much more readable and feels less cluttered with noise (e.g. indices when you don’t need them or incrementation exposing the underlying implementation). This latter point was an unexpected benefit of the new loop, by the way. Imagine that you have:

String[] names = ...;
for (String name : names) {
// ...
}

and you decide that you want to change the type of names to a Collection. How do you modify your code?

List<String> names = ...;
for (String name : names) {
// ...
}

That’s right, just one line. It doesn’t get better than that.

Annotations

Obviously, I am partial to annotations since they are at the heart of TestNG but I am a firm believer that annotations are going to change the way we build software in Java. We have been relying for far too long on reflection hacks to introduce meta-data in our programs, and annotations are finally going to provide an excellent solution to this problem.

Also, I haven’t felt the need to use some of the predefined annotations such as @Override, so I haven’t formed an opinion on them yet.

It seems inescapable to me that in a couple of years, most of the Java code that we will be reading and writing will contain annotations.

Static imports

I hardly use them at all, except in my annotations type for Retention and Target. I am still not convinced that the original intent that motivated the addition of this feature (discourage the anti-pattern of implementing an interface to be able to reuse its constants without having to qualify them) justifies the introduction of a new language feature, but time will tell.

I guess that in some way, the use of an IDE in my day-to-day programming makes imports absolutely obsolete, so I can’t really get myself to feel strongly about this feature anyway.

Variable-length arguments

I haven’t had the need for this feature at all so far. It might come in handy once in a while but I’m really not convinced it warranted a change in the language.

Enums

While I definitely give Enums a theoretical nod of approval, I haven’t really converted my code to them yet, and I haven’t acquired the reflex to use them either. I believe that when I do, I’ll be happy with the result and it will make my code a tad more robust.

Generics

I left the best for the end… but since this entry is getting a bit long, I will save the Generics discussion for tomorrow.

Game of the week

It’s Sunday, you’re not supposed to be on a computer, much less read this weblog. But since you obviously are here anyway, here my choice for game of the week: Zuma.

It’s a very addictive variation of Bubble Bobble. Ideal for ten minutes of mindless fun whenever you want to take a break.

It's not about one-liners

I have received quite a few comments on the previous entry so I thought I would clarify something. Jonas Galvez says:

The Python version is pretty much a one-liner too:
files = filter(lambda s: re.match("gz$", s), sorted(os.listdir(dir))[::-1])

I don’t care about the size of the code (I find that ten lines of Java are usually more readable that one line of Perl).

I do care about readability, which usually boils down to:

  • Ease of reading. This is usually a factor of how many non-alphabetical characters are used for a statement, but it also depends on keywords and on the names of the methods, which must carry the intent of the code very clearly. The code above doesn’t score very well on this scale.
  • Consistency of the API. In the Python code above, we have a mix of lambda, method invocation, string parameters, cryptic parameters (::-1) and arbitrary ordering of such parameters. I am sorry, I don’t find this readable but worse, I wonder if I would remember this magic invocation in six months.

  • Flow of the statements. What I like about the Ruby example is that it’s basically a piped sequence of method invocations: pipe the output of the previous invocation into the input of the next one. All methods, one object, no fuss (with the exception of the closure). Again, the Python code above seems to require object method invocation, imperative method invocations and parameters that alter the meaning of these methods.

Ruby scores pretty high on each of these points: the line can almost be read as natural language and I find it “very” object-oriented.

Why I love Ruby

zippedFiles =
Dir.new(dir).entries.sort.reverse.delete_if { |x| ! (x =~ /gz$/) }

’nuff said.

Okay, there is more to say.

First of all, what does this line of code do? It goes through every file in the given directory, sort them in reverse order while excluding any file that doesn’t end in “.gz”.

This code ported to Java is quite intimidating:

List<String> result = new ArrayList<String>();
File f = new File(directory);
for (String fileName : f.list()) {
if (fileName.endsWith(".gz")) {
result.add(fileName);
}
}
Collections.sort(result, new Comparator<String>() {
public int compare(String o1, String o2) {
return o2.compareTo(o1);
}
public boolean equals(Object o) {
return super.equals(o);
}
});

This code comes from a log analyzer utility that I wrote some time ago. It goes through the Apache log of my Web server and allows me to easily plug-in listeners to collect various kinds of statistics. This utility has provided me with a flexible log analyzer framework into which I have plugged various additional loggers these past months.

Since I hadn’t taken a look at this code in a few months, I was quite happy to realize that it passes the “six-month readability test”. A language that has never passed this test for me is Perl. Perl might be a powerful language but if you stop using it for six months, you will need a book to reread your own code and a personal trainer just to modify it.

So I was quite happy to understand my old Ruby code right away, even in the most idiomatic sections such as the one I pasted above. The code carries its intent clearly thanks to aptly-name methods and closures are, as usual, as pleasant to use as they are powerful.

There is a problem with my log analyzer, though, which is the reason why I am revisiting it today: it’s pretty slow. It takes about five minutes to run through a month of logs, which I find unacceptable. Therefore, I want to port it to a different language.

While I love Ruby, I have to say I like Groovy even more, because it gives me the same flexibility as Ruby with the familiar Java syntax on top of it. However, I have had some bad experiences with the current versions of Groovy and as far as I can tell from the mailing-list, the stability of the compiler still leaves a lot to be desired.

Exit Groovy (for now). So it will probably be Java or C#. I am hoping the poor performance comes from the Ruby interpreter and not from my code, but I will find out soon enough.