September 29, 2004
Marillion wrap-up
So I saw Marillion at the Fillmore last night, and despite my doubts, I thoroughly enjoyed the show.
The band used an unusual formula: the show was split in two parts and the first part was entirely dedicated to the new material. They played pretty much the entire Marbles album except for a couple of songs which they saved for the encores. The second part featured some of their standard live songs and unfortunately, they didn't play any Fish-era songs at all.
I can understand this decision (after all, Fish was only with Marillion for a few years and that was more than fifteen years ago), but there is no doubt that quite a few of the fans present last night would have loved to hear some Fish material. And after all, even Genesis was still covering some Peter Gabriel oldies in their last tours, so this is not unheard of in the prog circles.
The part I liked most was in the encores (there were two). After the first song, Steve Hogarth took he microphone and said:
"We were chatting in the dressing room and we couldn't decide which song we were going to play now... Afraid of Sunlight, or The Great Escape"Of course, the crowd was quick to indicate its preference vociferously and Steve teased us for a little while, switching from one to the other and back. Finally, he shrugged and said:
"Okay, fine. Both."The audience went berserk at this point, as you can imagine, and I happily joined in the screaming, as these two songs are definitely some of my favorites. Then he turned to Steven Rothery and asked him:
"What guitar are you using right now?"Rothery mouthed "A-O-S", Steve acknowledged and then, Marc Kelly started playing the hypnotic first bars of Afraid of Sunlight... Drive the road... to your surrender...
The Great Escape is always spectacular on stage, especially when Steve's voice is up to it, and he fully delivered last night. Finally, the band closed the last encore with Easter, which Steve described as "one of the very first songs we wrote together, when I showed up circa 1989".
A fitting closing act for a fine performance of progressive rock delight.
September 28, 2004
ant, make and tests: same fight
Groboclown and I have had an interesting exchange about suite() on his weblog.
The idea of using JUnit ant tasks to work around the limitations of JUnit is sadly very common, to the point that nobody notices even more that this is a fundamental flaw in JUnit. In this particular example, you can use <batchtest> to invoke explicitly several Test classes (something you can't even do with the basic TestRunner!) and also to use ant's powerful wildcard matching.
This is useful, but it has two limitations:
- It requires ant. In heterogeneous environments that build more than just Java code, ant is not always an option. Besides, I really think that such features should be available at the simple Java level.
- Your level of granularity is still the class, not the method.
That's a lot of work when all you need is to tell your test framework to run a specific test method.
Groboclown quotes a book on this topic:
In Testing Objet-Oriented Software: Life Cycle Solutions, by Imran Bashir and Amrit L. Goel, they argue that the unit of test has moved from the function (in structured programming) to the class (in object-oriented programming). Using this argument, I'll claim that there's no need to get a lower granularity than at the test class level.I disagree. The class level still fails to capture three very important levels of granularity in testing:
- Individual methods.
- Groups of methods.
- Sequences of methods that depend on each other.
You declare dependencies between targets and you can group these dependencies together, creating an acyclic graph of clusters representing your run sequence where some processes are run sequentially while others can be run in parallel.
Tests are no different, and whenever you use a framework that doesn't give you this flexibility, you are making your life unnecessarily hard.
September 27, 2004
TestNG's @Factory
I have extolled the virtue of external runtime configuration for testing with TestNG for a while now. More simply put: when you decide to run a different set of tests, you shouldn't have to recompile any Java code. This kind of information is dynamic and Java is therefore not the best way to express it.
Having said that, there is some undeniable power in the ability to express this configuration from a programming language. JUnit offers this with the concept of Test Suites (and the method suite()), and for this, TestNG has the @Factory annotation.
Imagine the following situation: you have a class that validates an XML file and you want to run this class on several files. The problem is: you don't know in advance what these XML files are (they might be "all the XML files in this directory"). Obviously, a programmatic approach is needed for this.
Enter @Factory.
public class XMLFilesTest {
@Factory
public Object[] factory() {
List<Object> vResult = new ArrayList<File>();
File[] files = // read files from the directory
for (File f : files) {
vResult.add(new XMLValidatorTest(f));
}
return vResult.toArray(new Object[vResult.size()]);
}
}
Once you have a test class that has a @Factory method, all you need to do is specify this method in your testng.xml:
<suite>
<test name="XML" >
<classes>
<class name="test.sample.XMLFilesTest" />
</classes>
</test>
</suite>
TestNG will pick you your @Factory method, invoke it, retrieve all the objects you returned and consider each of them as a separate TestNG test class.
Interestingly, I initially added this feature to allow tests to provide multiple instances of themselves. This is handy when you are testing something like the reentrancy of your class, or when you want to stress-test a server (write a simple test, create one thousand instances of that class and run TestNG on it). As it turns out, this feature can be used to return any kind of instances, not just instances of class the factory belongs to.
Can you think of other potential uses for @Factory?
September 26, 2004
Marillion in San Francisco
Marillion is coming to San Francisco next week. Since I moved to the US, I haven't had a lot of opportunities to watch my favorite progressive rock bands on stage, since most of them are European. So far, only Yes, Camel, Spock's Beard and the Flower Kings have made it to the West Coast, so I am quite happy to get a chance to watch Marillion on stage again after so many years (last time was ten years ago in France).
Marillion has always had a special place in my musical portfolio, because they were the second progressive band I completely fell for after reeling from the shock caused by the music of Genesis. My first Marillion album was Misplaced Childhood and I was sold right away. Clutching at straws is a solid favorite Fish-era album of mine and it's hard not to be haunted by the title track of Script for a Jester's tear.
I was quite sad to see Fish leave and the next two albums that followed with the new singer Steve Hogarth left Marillion fans worried that their favorite band had abandoned them. And then, Brave came out, and it absolutely blew me away. Even to date, Brave is one my top five favorite progressive albums of all times, next to Genesis' The lamb lies down on Broadway.
The next album, Afraid of Sunlight, was also a real pleasure, but since then, I have to say that Marillion seems to have drifted again away from my personal tastes. This strange engine, Radiation, marillion.com and Anoraknophobis didn't do much for me.
With their latest album, Marbles, Marillion seems to have come back to a kind of music that I find more inspired and melodious, varying themes both musically and lyrically.
At any rate, I am more than happy to watch them on stage again, even if I feel I have somewhat wandered away from the music they produce these days. They remain an outstanding set of musicians and I am quite looking forward to discovering their new set list (which I didn't look up, because I like to be surprised).
Listen as the syllables of slaughter cut with calm precision
Patterned frosty phrases rape your ears and sow the ice incision
Apocalyptic alphabet casting spells the creed of tempered diction
Adjectives of annihilation bury the point beyond redemption
September 22, 2004
Wanna work at Google? Email me.
I'm serious.
If you are reading this weblog, you are probably the kind of developer Google would like to hire, so if you want to get a chance to work here, all you need to do is email me your résumé.
This offer is not limited to Java developers. C++, Python, C#, Windows, Linux, etc... all are welcome. Also, Mountain View and New York City are not our only offices, check out our Web site if you want more details.
Here are the rules:
- It's okay if I don't know you. You are welcome to write a few lines to introduce yourself if you feel like it, but ultimately, your résumé is the only thing that counts.
- Please don't send me a Word document. If I don't know you, I'm not going to click on your .doc. Attach your résumé as a text file to your email or better, post it on a Web site and send me the URL.
- Don't expect an answer or an acknowledgement from me. I am not quite sure what kind of response this post will receive, but I can't promise I'll have time to respond personally to each of you.
That's all.
As for my email address, let's just say that figuring it out will be your first interview question, but please use this mailto link so I can identify your email by its Subject.
Don't be shy!
Update: If you don't live in the US, please specify in your email what office (e.g. Zurich) you would like to work in.
Update 2 : This offer does not expire in time. If you are reading it months after I initially posted it, feel free to email me your résumé anyway.
Update 3 : Google just opened a new office in Kirkland, Washington.
JDK 5.0 in practice, part 2
Autoboxing
I have stayed away from autoboxing so far, probably because I have a vague feeling of losing control of the performance of my code. I don't think it is justified, though, so autoboxing can come in handy and make your code a little bit more readable. I think I would encourage developers to flag their code when such autoboxing is happening, and I am pretty sure that IDE's will soon be able to do the same.
Generics
Where to start?
Well, first of all, nobody can dispute that Generics are a solid concept that tends to improve the robustness of your code. The reason why they are so usually controversial regardless of the language is because of their implementation. And for having been a member of the C++ committee for many years , I can definitely vouch for the difficulty of getting them right.
In a nutshell, I have this to say about Java generics: my code feels more robust, but it's harder to read.
So what's the problem?
Redundancy.
First of all, I have always had a hard time with the redundancy introduced by the necessity of casting in general. For example, instead of writing:
Map accounts = new HashMap(); // no generics
...
Account a = (Account) accounts.get("Cedric");
why can't I just write:
Map m = new HashMap(); // no generics
...
Account a = m.get("Cedric");
and let the compiler introduce a silent cast, since obviously, it's an object of type Account that I am trying to retrieve from the Map?
Obviously, Generics don't solve this problem entirely but they make a decent job at alleviating it somewhat. But they also make it worse in some other ways:
Map<String, List<Account>> accounts = new HashMap<String, List<Account>>();Ouch.
Not only is the code significantly harder to read, but it fails to obey the DRY principle ("Don't repeat yourself"). What if I need to change the value type of this map from List<Account> Collection<Account>? I need to replace all these statements everywhere in my code. While IDE refactoring will help, it's still an awful lot of code for a modification of this kind that hardly impacts the semantics of this code.
Admittedly, there is no nice way to avoid this syntax when you are creating a new object, but what I am driving at is that I think Generics would have been better off if typedefs had been introduced with them.
Or so I thought at first.
But after thinking about it more, I realized that typedefs were the wrong solution to the problem, because simply put, they don't add anything to the use of a separate class to define your complex Generic type.
class AccountMap extends HashMap<String, List<Account>> {
...
}
Except for the fact that you need to extend an implementation (HashMap, and not Map, obviously), this solution is probably better than introducing typedef, which has its own quirks.
I haven't gone to this trouble so far, but my recommendation would be: do it if you write the type more than three times (twice in the initialization and you use it more than once in your code).
Except for this little annoyance, I am quite happy with Generics overall and I particulary enjoy reading the TestNG Javadocs so nicely typed.
Conclusion
I am very happy with the new features of JDK 5.0 and I am quite proud to have had a chance to influence it with my participation in JSR 175 and JSR 201. Like all radical evolutions, not all of the new features will be popular with everyone, but as long as most developers find some of these features useful and that backward compatibility is preserved, I think JDK 5.0 is a very solid step toward more solid Java code.
September 21, 2004
JDK 5.0 in practice
I have been writing JDK 5.0 code for over six months now, so I thought I would take some time to reflect on my experience and draw a few conclusions on the features that were introduced.
Enhanced for loop
The undisputed winner. I can't even begin to describe how good it feels to use the new for loop everywhere (well, almost everywhere). I mentally cringe the few times when I am forced to use the old for loop, typically when I need the index or that I want the Iterator to be visible outside the loop.
The code is much more readable and feels less cluttered with noise (e.g. indices when you don't need them or incrementation exposing the underlying implementation). This latter point was an unexpected benefit of the new loop, by the way. Imagine that you have:
String[] names = ...;
for (String name : names) {
// ...
}
and you decide that you want to change the type of names to a Collection. How do you modify your code?
List<String> names = ...;
for (String name : names) {
// ...
}
That's right, just one line. It doesn't get better than that.
Annotations
Obviously, I am partial to annotations since they are at the heart of TestNG but I am a firm believer that annotations are going to change the way we build software in Java. We have been relying for far too long on reflection hacks to introduce meta-data in our programs, and annotations are finally going to provide an excellent solution to this problem.
Also, I haven't felt the need to use some of the predefined annotations such as @Override, so I haven't formed an opinion on them yet.
It seems inescapable to me that in a couple of years, most of the Java code that we will be reading and writing will contain annotations.
Static imports
I hardly use them at all, except in my annotations type for Retention and Target. I am still not convinced that the original intent that motivated the addition of this feature (discourage the anti-pattern of implementing an interface to be able to reuse its constants without having to qualify them) justifies the introduction of a new language feature, but time will tell.
I guess that in some way, the use of an IDE in my day-to-day programming makes imports absolutely obsolete, so I can't really get myself to feel strongly about this feature anyway.
Variable-length arguments
I haven't had the need for this feature at all so far. It might come in handy once in a while but I'm really not convinced it warranted a change in the language.
Enums
While I definitely give Enums a theoretical nod of approval, I haven't really converted my code to them yet, and I haven't acquired the reflex to use them either. I believe that when I do, I'll be happy with the result and it will make my code a tad more robust.
Generics
I left the best for the end... but since this entry is getting a bit long, I will save the Generics discussion for tomorrow.
September 19, 2004
Game of the week
It's Sunday, you're not supposed to be on a computer, much less read this weblog. But since you obviously are here anyway, here my choice for game of the week: Zuma.
It's a very addictive variation of Bubble Bobble. Ideal for ten minutes of mindless fun whenever you want to take a break.
September 17, 2004
It's not about one-liners
I have received quite a few comments on the previous entry so I thought I would clarify something. Jonas Galvez says:
The Python version is pretty much a one-liner too:
files = filter(lambda s: re.match("gz$", s), sorted(os.listdir(dir))[::-1])
I don't care about the size of the code (I find that ten lines of Java are usually more readable that one line of Perl).
I do care about readability, which usually boils down to:
- Ease of reading. This is usually a factor of how many non-alphabetical characters are used for a statement, but it also depends on keywords and on the names of the methods, which must carry the intent of the code very clearly. The code above doesn't score very well on this scale.
- Consistency of the API. In the Python code above, we have a mix of lambda, method invocation, string parameters, cryptic parameters (::-1) and arbitrary ordering of such parameters. I am sorry, I don't find this readable but worse, I wonder if I would remember this magic invocation in six months.
- Flow of the statements. What I like about the Ruby example is that it's basically a piped sequence of method invocations: pipe the output of the previous invocation into the input of the next one. All methods, one object, no fuss (with the exception of the closure). Again, the Python code above seems to require object method invocation, imperative method invocations and parameters that alter the meaning of these methods.
September 16, 2004
Why I love Ruby
zippedFiles =
Dir.new(dir).entries.sort.reverse.delete_if { |x| ! (x =~ /gz$/) }
'nuff said.
Okay, there is more to say.
First of all, what does this line of code do? It goes through every file in the given directory, sort them in reverse order while excluding any file that doesn't end in ".gz".
This code ported to Java is quite intimidating:
This code comes from a log analyzer utility that I wrote some time ago. It goes through the Apache log of my Web server and allows me to easily plug-in listeners to collect various kinds of statistics. This utility has provided me with a flexible log analyzer framework into which I have plugged various additional loggers these past months.List<String> result = new ArrayList<String>(); File f = new File(directory); for (String fileName : f.list()) { if (fileName.endsWith(".gz")) { result.add(fileName); } } Collections.sort(result, new Comparator<String>() { public int compare(String o1, String o2) { return o2.compareTo(o1); } public boolean equals(Object o) { return super.equals(o); } });
Since I hadn't taken a look at this code in a few months, I was quite happy to realize that it passes the "six-month readability test". A language that has never passed this test for me is Perl. Perl might be a powerful language but if you stop using it for six months, you will need a book to reread your own code and a personal trainer just to modify it.
So I was quite happy to understand my old Ruby code right away, even in the most idiomatic sections such as the one I pasted above. The code carries its intent clearly thanks to aptly-name methods and closures are, as usual, as pleasant to use as they are powerful.
There is a problem with my log analyzer, though, which is the reason why I am revisiting it today: it's pretty slow. It takes about five minutes to run through a month of logs, which I find unacceptable. Therefore, I want to port it to a different language.
While I love Ruby, I have to say I like Groovy even more, because it gives me the same flexibility as Ruby with the familiar Java syntax on top of it. However, I have had some bad experiences with the current versions of Groovy and as far as I can tell from the mailing-list, the stability of the compiler still leaves a lot to be desired.
Exit Groovy (for now). So it will probably be Java or C#. I am hoping the poor performance comes from the Ruby interpreter and not from my code, but I will find out soon enough.
September 14, 2004
AOP still not there
I wish Dion were right when he says that AO is gaining use, or at least interest. Indeed, we have seen a significant pick up in interest in AOP these past two years, but unfortunately, not much progress has been made since then in terms of visibility.
The sad truth is that the only times AOP makes an appearance is in bleeding edge conferences such as No Fluff Just Stuff or The ServerSide Symposium. Are you seeing it around you in your every day job? I certainly don't, and actually, most of the developers I talk to don't even know what it is.
What went wrong? Failure to communicate or simple inadequacy to the problems software developers are facing every day?
Is AOP doomed to be a niche technology reserved to a tiny fraction of the developers?
September 11, 2004
Better reporting in TestNG
I have improved the reporting made by TestNG to give you more information on your last test run, and more precisely:
-
The list of groups with their methods. When you write your test methods and assign them to certain groups, you might lose track of which method belongs to which groups since these groups can span over different classes. This view allows you to quickly see the list of groups and which methods are in them, which makes it easy to see, for example, which tests are in the "broken" group (and therefore are in need to be fixed) or how many methods your "functest" group really contains.
-
The sequence of method invocations. This is a list of all the methods that were invoked during your test, sorted chronologically. This is a good way to make sure that your configuration methods are being invoked correctly or that your inclusion and exclusion of test methods is as you expect it. As you can see from this view, configuration class methods are invoked before the instantiation of your test class and after the last test method has run, while configuration methods methods are being invoked "around" each test method.
September 09, 2004
suite() : good or bad?
I have a love-hate relationship with JUnit's suite() feature.
Quick reminder: a JUnit class can either have test methods (starting with test) or a suite() method such as this:
public static Test suite() {
TestSuite suite= new TestSuite();
suite.addTest(new MoneyTest("testMoneyEquals"));
suite.addTest(new MoneyTest("testSimpleAdd"));
return suite;
}
In which case, the Tests returned by your suite() method will be inspected recursively for either test methods or more suites.
This is a powerful concept that allows you to define an entire hierarchy of tests easily. Typically, you can end up with a "root" suite such as:
public static Test suite() {
TestSuite suite= new TestSuite();
suite.addTest(new BackEndSuite());
suite.addTest(new FrontEndSuite());
return suite;
}
and in turn, have BackEndSuite() contain more suites (DataBaseSuite, SecuritySuite, UserProfileSuite, etc...) and FrontEndSuite() contain things such as HTMLSuite, TemplateSuite, PostSuite, etc...
The problem comes from debugging a failed test. Suddenly, it becomes really hard to pinpoint exactly what test is failing because, well, you simply can't tell right away which tests are active, enabled, and what suite they belong to.
In short, there is not a single file that you can look at and get an overall view of your test structure.
Confronted with this problem with TestNG, I am not sure what to do. I like the idea of the hierarchy but I wonder if it doesn't have more downsides than upsides.
For now, I have decided to try the "one file only" approach, and that file is your build.xml. The
Thoughts?
<testng outputDir = "${basedir}/test-output">
<fileset dir="${basedir}">
<include name="back-end/database/testng.xml" />
<include name="back-end/security/testng.xml" />
<include name="front-end/html/testng.xml" />
...
</fileset>
</testng>
If I wanted to introduce the concept of a hierarchy of tests, I would simply allow testng.xml to reference other testng.xml files.
September 01, 2004
Announcing TestNG 1.0!
I am happy to announce the availability of TestNG 1.0. A lot of changes have been made since the early releases thanks to the recent addition of Jolly Chen to the team.
As a quick reminder, TestNG is a testing framework using annotations to provide a more powerful way to test your code in various ways: unit, regression, functional, integration, etc... TestNG allows you to clearly separate your Java code from the way your tests are run. You never need to recompile any of your classes if you decide to run a different set of tests or suites.
TestNG also provides a very flexible configuration mechanism allowing you to specify arbitrary methods to be invoked at particular moments during your test run, such as "before every test method" or "after all the test methods have run".
Here is a quick example of what TestNG code looks like:
import com.beust.testng.annotations.*;
public class SimpleTest {
@Configuration(beforeTestClass = true)
public void setUp() {
// code that will be invoked when this test is instantiated
}
@Test(groups = { "functest" })
public void itWorks() {
// This method will be invoked if the current run
// includes the group "functest"
}
}
Some of the major changes since the previous versions include:
- XML. The property files were easy to work with in the beginning but it was quickly obvious that they wouldn't scale. Now, the information for a suite run (which contains several tests, each test containing several classes) is conveniently located in one XML file.
- Dependent methods. With this feature, you can guarantee that certain groups of methods are always invoked before others. If the dependents didn't run or fail, the method will be marked as a SKIP, which can be seen as less critical than a FAILURE since SKIPs will typically be automatically resolved when the FAILURE that caused them is fixed.
- Groups of groups. Once you have specified that your test methods belong in specific groups, you can specify bigger groups, giving you a lot of flexibility on exactly what tests are run each time.
- Better reporting. Not only does TestNG have its own HTML reporting, it is also compatible with JUnitReport. You can see an example here.
- Open API. TestNG exposes its engine with a convenient set of listeners which allow you to plug-in your own reporting mechanism or for any other purpose (as a matter of fact, the JUnitReport code is such a listener). It is also trivial to invoke TestNG programmatically, thus skipping the XML phase and providing TestNG with direct Java objects.
- Parameters, which override the Suite parameters.
- Groups, made of two parts, definitions and runs.
- Classes, which define which classes are going to be part of this test run
- Definitions, when you want to create groups that contain other groups.
- Runs, which specify the groups to be run.
Download TestNG and try it, you won't regret it!