Recently, several TestNG users started asking for more control in the way test methods are selected.  For example, here is one way you specify which groups to run:

<groups>
  <run>
  <include name="nopackage"/>
  <exclude name="broken" />
  </run>
</groups>

The include/exclude syntax is fairly flexible, but it doesn’t help when you want to perform more complex expressions, such as ‘run the test methods that belong to the group "g1" and "g2" but not "g3"’.

There are many ways to solve this problem, and the most obvious one to me is to support a scripting language inside TestNG.

After thinking about it for a while, I decided to go for BeanShell for two reasons:

  • It’s Java.
  • It’s small (the core interpreter is 140k).

I was tempted to pick Groovy for a short while but I decided that the learning curve and the sheer size of its jar file made it a non-starter.  BeanShell is cleary designed to be embedded and has a solid reputation, so it was an easy choice.

  And so far, I don’t regret my choice:  integrating BeanShell in TestNG took less than one hour, and you can now specify things such as:

<test>
 <eval><![CDATA[
       groups.containKey("group1") &&
     method.getName().equals("test1")
     ]]
 </eval>
 ...
</test>

Since I never miss an opportunity to do something silly, I tried the following expression:

0 == JOptionPane.showConfirmDialog(
  new JFrame(),
  "Run this method?\n" + method,
  "TestNG",
  JOptionPane.YES_NO_OPTION);

which promptly displayed the following dialog for every single test method:

This way, I can hand pick the test methods one by one for every run.  Not exactly practical, but you get the idea.

The possibilities offered by embedding a scripting language inside TestNG are quite exciting and I’m looking forward to more twisted ways to abuse this feature
:-)