Robert (sorry, couldn’t figure out his name) wrote a
few
interesting criticisms
about TestNG:

JUnit’s got huge traction in the Java community because of the tool support.

I disagree.  I think JUnit got traction because there was nothing else
when it came out.  It was simple, lightweight, easy to use, so everybody
adopted it.  Fair enough.  Then its limitations started showing and
since its development was pretty much stopped then, people started building
tools to work around its limitations.

And here is where I think the problem is with JUnit:  the JUnit tools
fix deficiciencies in its design.  For example, the ant task allows you to launch
the runner on a wildcard of classes or suites, which you can’t do with JUnit. 
A lot of the value added by these tools should have been retrofitted in JUnit by
now.  But they haven’t, because JUnit is basically a dead project by now.

If the only way you can easily launch tests remains from the command-line,
it’s not going to work very well.

I am not so sure about that either.  Running tests from your IDE is
definitely a plus (and I plan on writing an Eclipse plug-in once Eclipse
understands JSR 175 annotations), but I believe that most of unit testing happens in
automated builds.  A good testing framework needs to be extremely flexible
from the command line, that’s a requirement.  Which, as I said above and
in
previous
postings, JUnit fails to achieve in many ways.

Case in point: how can I easily launch, in TestNG, both my older JUnit tests
and the new-fangled ones?

It’s pretty easy to do, but you will need to run the launcher twice. 
This is just because I made the "JUnit compatibility flag" per-suite instead of
per-test.  It seems to make more sense this way, but this is pretty easy to
modify.

I hate the way the only log format for TestNG is a
HTML
file.

I agree.  This was just a proof of concept to get you started. 
Right out of the box, TestNG gives you an easy-to-read HTML page that the most
novice user can read and understand.  But TestNG is fairly extensible and
easy to modify.  Since it’s such an early stage, I haven’t had the time to
polish that part yet, but there is an important part of TestNG that you might
have overlooked:  it gives you an API to inspect its running process.

For example, you can add listeners that will keep you informed when a test
starts, when it ends, if it succeeded, if it failed, why, etc…  As you
can see, adding your own logger and report generator is straightforward. 
In comparison, JUnit is a big black box.

There’s not a single Decorator in sight in TestNG.

Let’s take this offline, I’m not quite sure what you are asking for here.

Not to be nitpicky, but… I absolutely loathe the use of the ‘I’ to
mark an interface. Can’t stand it. Makes me shudder and have flashbacks to when
I used to write Microsoft
COM
code in C.

Ah, well, sorry about that.  It’s a widely-used standard, though
(Microsoft, Eclipse and WebLogic Workshop use it, and they certainly know their
stuff…).  You’ll get used to it :-)

At any rate, thanks for the feedback.  Contrary to what a lot of people
think, I didn’t write TestNG to kill JUnit but to make people think.  So
many toolkits and frameworks are considered for granted these days that nobody even
thinks of improving them any more (e.g JUnit, Velocity or XDoclet to name a
few).  If TestNG causes JUnit to be improved, I’m all for it!

That being said, I have had a few more ideas to improve TestNG recently,
which will show how powerful annotations can be compared to the "old way". 
Stay tuned, I’ll blog about this very soon.

Robert, feel free to email me so we
can continue the discussion!