JUnit 4.8 contains a nice new feature called "Categories" that allows you to group certain kinds of tests together. This is very useful, e.g. to have separate test runs for slow and fast tests. I know the stuff mentioned in JUnit 4.8 release notes , but would like to know how I can actually run all the tests annotated with certain category.
The JUnit 4.8 release notes show an example suite definition, where SuiteClasses annotation selects the tests from certain category to run, like this:
@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@SuiteClasses( { A.class, B.class }) // Note that Categories is a kind of Suite
public class SlowTestSuite {
// Will run A.b and B.c, but not A.a
}
Does anyone know how I could run all the tests in SlowTests category? It seems that you must have the SuiteClasses annotation...
Source: Tips4all
I found out one possible way to achieve what I want, but I don't consider this to be the best possible solution as it relies on ClassPathSuite library that is not part of JUnit.
ReplyDeleteI define the test suite for slow tests like this:
@RunWith(Categories.class)
@Categories.IncludeCategory(SlowTests.class)
@Suite.SuiteClasses( { AllTests.class })
public class SlowTestSuite {
}
AllTests class is defined like this:
@RunWith(ClasspathSuite.class)
public class AllTests {
}
I had to use ClassPathSuite class from ClassPathSuite project here. It will find all the classes with tests.
Here are some of the main differences between TestNG and JUnit when it comes to groups (or categories, like JUnit calls them):
ReplyDeleteJUnit's are typed (annotations) while TestNG's are strings. I made this choice because I wanted to be able to use regular expressions when running tests, for example "run all the tests that belong to the group "database*". Also, having to create a new annotation whenever you need to create a new category is annoying, although it has the benefit that an IDE will tell you right away where this category is used (TestNG shows you this in its reports).
TestNG separates very clearly your static model (the code of your tests) from the runtime model (which tests get run). If you want to run the groups "front-end" first and then "servlets", you can do this without having to recompile anything. Because JUnit defines groups in annotations and you need to specify these categories as parameters to the runner, you usually have to recompile your code whenever you want to run a different set of categories, which defeats the purpose in my opinion.
Not a direct answer to your problem, but maybe the general approach could be improved...
ReplyDeleteWhy are your tests slow? Maybe the set-up lasts long (database, I/O etc.), maybe the tests are testing too much? If this is the case I would seperate the real unit-tests from the "long-running" ones, which often indeed are integration tests.
In my setups I have staging env, where unit-tests are run often and integration-tests constantly but more rarely (e.g. after each commit in version control). I have never worked with grouping for unit tests, because they should be loosely coupled alltogether. I only work with grouping and relationship of test-cases in integration-test setups (but with TestNG).
But good to know that JUnit 4.8 introduced some grouping features.
I am not sure, what exactly your problem is.
ReplyDeleteJust add all the tests to a suite (or hirachy of suites). Then use the Categories Runner and Include/ExcludeCategory annotation, to specify the categories you want to run.
A good idea might be to have one suite containing all the tests, and a couple of seperate suites referring to the first one, specifying the different set of Categories you neeed.