I just attended Test-Driven Development presentation which represents everything that is wrong about the way Agile advocates are trying to evangelize their practices.  I don’t have anything against the presenter in particular, but it’s really time for Agilists to rethink the way they communicate with the real world.

Here are a few comments on his presentation.

One of the first slides that deeply troubled me claimed the following:

  • Tests are (executable) specs.
  • If it’s not testable, it’s useless.

First of all, tests are not specs.  Not even close.  Somebody in the audience was quick to give a counter-example to this absurd claim by using a numeric example ("how do you specify an exponentiation function with a test?") but my objection to this claim is much broader than that.  Relying on tests as a design specification is lazy and unprofessional because you are only testing a very small portion of the solution space of your application (and of course, your tests can have bugs).  Tests also fall extremely short of having the expressiveness needed to articulate the subtle shades that a real specification need to cover to be effective.

This claim is part of a broader and more disturbing general Agilist attitude that is usually articulated like "Your code is your spec", along with some of its ridiculous corollaries such as "Documentation gets out of date, code never does".

Anyone who claims this has never worked on a real-world project.  And I’m setting the bar fairly low for such a project:  more than five developers and more than 50,000 lines of code.  Try to bring on board new developers on this project and see how fast they come up to speed if all they have to understand the code base is… well, just code.  And tests.

I am currently getting acquainted with a brand new project that is not even very big, and while I understand Java fairly well, there is no doubt in my mind that for ten minutes I spend trying to understand how a certain part of the application works, a five-line comment would have given me this knowledge in ten seconds.

The second claim, "If it’s not testable, it’s useless" is equally ludicrous and a guarantee that at this point, the audience you are talking to is already looking at you as a crackpot.

Software is shipped with untested parts every day, and just because it’s not entirely tested doesn’t mean it’s bad software or that the untested parts are "useless".

Agilists just don’t understand the meaning of calculated risk.

Early in the development cycle, it’s perfectly acceptable to go for a policy of "zero bugs" and "100% tests".  But as the deadline looms, these choices need to be reconsidered all the time and evaluated while keeping a close eye of the final goal.  Very often, Agilists simply forget that their job is to produce software that satisfies customers, not software that meets some golden software engineering scale.

Anyway, let’s go back to the presentation, which then proceeded with the implementation of a Stack class with TDD.  Before spending thirty minutes on a live demo of the implementation of a Stack class (are you impressed yet?), the presenter warned the increasingly impatient audience that they should "not pay too much attention to the Stack example itself but to the technique".

And that’s exactly the wrong thing to do.

Look, we "get" TDD.  We understand it.  Frankly, it takes all of ten minutes to explain Test-Driven Development to a developer who’s never heard of it:  "Write a test that fails and doesn’t compile.  Make it compile.  Then make it pass.  Repeat".

The hard part is applying it to the real world, and showing the implementation of a Stack will soon have everyone leave the room with the thought "Cute, but useless.  Now let’s go back to work". 

It was even worse than that, actually:  The presenter kept taking suggestions from the crowd but he declined all those that didn’t fit in the neat script that he had in hands at all times.  These suggestions were good, by the way: 

"What should we test now?"

"How about:  if we pop an empty stack, we get an exception"

<a bit embarrassed> "Mmh, no, let’s not do that" <hand waving, look down at notes and proceeds while happily ignoring the other raised hands>

To be honest, I am becoming quite suspicious of Agile practices for that reason:  all the presentations I have attended and books that I have read are always using toy implementations as examples.  Stack, List, Money, Bowling…  enough already!  Let’s talk about TDD for code that interacts with clustered databases on laggy connections built on 500,000 lines of code that was never designed to be tested in the first place (and:  yes, I read Michael Feathers’ book, it has some good and some bad, but it’s not germane to Java and TDD so I won’t expand on it here).

And please, avoid smug and useless answers such as:

"A lot of the classes I have to test are hard to isolate, do you have any advice regarding mocks?"

"Well, if you had started with TDD in the first place, you wouldn’t be having this problem today".

Fundamentally, I am disturbed by the Agilists’ dishonesty when it comes to presenting their arguments.  They offer you all these nice ideas such as Test-Driven Development and Pair Programming but they never — ever — disclose the risks and the downsides.  To them, Agility is a silver bullet that is applicable in all cases with no compromises.

The truth is that these practices come at a price, and for a lot of organizations, the price gets high very quickly.  Agile development will never go far if its proponents keep ignoring these organizations and make condescending comments to its members.

I like Test-Driven Development.  I really do, and I’m fortunate enough to work on a project that lets me use TDD most of the time.  But the truth is:  at times, I don’t do TDD because implementing a feature quickly is more important than a fuzzy feeling.  And I’m also aware that TestNG is an open source project with less than five developers, all of them on the bleeding edge and aware of the latest advances in software engineering.

And this is my main beef with Agilists:  I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software have to go through to organize huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias.

So here is my advice to Agilists:  get real now, or you will become irrelevant soon.

Update: four years after I posted this article, reddit picked it up.