JUnit runs test methods in whatever order it wants, which is generally fine as long as all your test methods are independent of each other. This, though, may not be the case when you’re writing integration tests, where you could have a sequence of operations like
login(credentials);
requestProject("foo");
createNewItem("bar");
logout();
in a situation like this, you always want testLogout() to run after testLogin(), and createNewItem() to run after requestProject()!
Yes, you could group everything in a single test method, but it may become huge and very hard to maintain.
Quite surprisingly, JUnit doesn’t have a built-in solution for this. You can run test methods sorted by name using the @FixMethodOrder(MethodSorters.NAME_ASCENDING) tag, but then you need to artificially name your methods so that they appear in the order you want.
I found some alternative solutions in this stackoverflow question, but they require annotating your methods with tags to specify the order in which you want them to run. What I’d like them to do is just run in the same order as they appear in the source code for the test class. Among the answers, I just found this blog post that achieves the same result as I did, only it looks somewhat more complicated (it involves writing/including several classes).
My solution is fairly simple, but there’s 2 warnings:
- it uses Javassist, so if you don’t want to add libraries, there’s that
- it only works as long as you don’t have test classes that extend other test classes, and you don’t override all @Test-annotated methods in the subclass (I’ve never done that, but I guess as tests get complicated, you may have that); this can be fixed quite easily though, you just need to add the logic for what should come first according to your needs
On with the code!
You can grab the source straight from this pastebin, or copy/paste it from here (I added the MIT License, I think it should be the most permissive.. my intent is to say “do whatever the heck you want with this code”)
/*
* Copyright (C) <2014> <Michele Bonazza>
*
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
/**
* A test runner that runs tests according to their position in the source file
* of the test class.
*
* @author Michele Bonazza
*/
public class OrderedTestRunner extends BlockJUnit4ClassRunner {
/**
* Creates a new runner
*
* @param clazz
* the class being tested
* @throws InitializationError
* if something goes wrong
*/
public OrderedTestRunner(Class<?> clazz) throws InitializationError {
super(clazz);
}
/*
* (non-Javadoc)
*
* @see org.junit.runners.BlockJUnit4ClassRunner#computeTestMethods()
*/
@Override
protected List<FrameworkMethod> computeTestMethods() {
// get all methods to be tested
List<FrameworkMethod> toSort = super.computeTestMethods();
if (toSort.isEmpty())
return toSort;
// a map containing <line_number, method>
final Map<Integer, FrameworkMethod> testMethods = new TreeMap<>();
// check that all methods here are declared in the same class, we don't
// deal with test methods from superclasses that haven't been overridden
Class<?> clazz = getDeclaringClass(toSort);
if (clazz == null) {
// fail explicitly
System.err
.println("OrderedTestRunner can only run test classes that"
+ " don't have test methods inherited from superclasses");
return Collections.emptyList();
}
// use Javassist to figure out line numbers for methods
ClassPool pool = ClassPool.getDefault();
try {
CtClass cc = pool.get(clazz.getName());
// all methods in toSort are declared in the same class, we checked
for (FrameworkMethod m : toSort) {
String methodName = m.getName();
CtMethod method = cc.getDeclaredMethod(methodName);
testMethods.put(method.getMethodInfo().getLineNumber(0), m);
}
} catch (NotFoundException e) {
e.printStackTrace();
}
return new ArrayList<>(testMethods.values());
}
private Class<?> getDeclaringClass(List<FrameworkMethod> methods) {
// methods can't be empty, it's been checked
Class<?> clazz = methods.get(0).getMethod().getDeclaringClass();
for (int i = 1; i < methods.size(); i++) {
if (!methods.get(i).getMethod().getDeclaringClass().equals(clazz)) {
// they must be all in the same class
return null;
}
}
return clazz;
}
}
to use this, you need to add Javassist to your classpath; if you have a Maven project, it’s incredibly easy to do so, just add this to your POM:
<dependency>
<groupId>javassist</groupId>
<artifactId>javassist</artifactId>
<version>3.12.1.GA</version>
</dependency>
and annotate your JUnit test class with @RunWith(OrderedTestRunner.class), like this:
import wherever.you.put.OrderedTestRunner;
import org.junit.Test;
import org.junit.runner.RunWith;
@RunWith(OrderedTestRunner.class)
public class MyTestClass {
@Test
public void testZero() {
System.out.println("test zero run");
}
@Test
public void testOne() {
System.out.println("test one run");
}
}
That’s it!
About warning 2. above, in case you have something like MyTestClass extends BaseTestClass and you have methods annotated with @Test in BaseTestClass that aren’t overridden by MyTestClass, OrderedTestRunner will just fail printing the message you can see at line 50 above on System.err. I did this because I don’t think there’s a well-defined order in that case (should all methods from the super class run first? Should that go all the way up in the class hierarchy?), so you can adjust it to fit your particular needs.
something I did not know 