Order of tests in python unittest

Question

I was looking at similar questions and I couldn't find an answer to my problem.

I wrote Tests in a python class that derives from unittest.TestCase

class TestEffortFormula(unittest.TestCase)

I need to give an order to the tests (please, do not tell me that I shouldn't rely on test's order, I just do).

Before I needed to give order to the tests the command I used to run the tests was:

unittest.main(testRunner=TeamcityTestRunner())

Then I wanted to make the order dissappear, so I tried the following:

loader = unittest.TestLoader()
loader.sortTestMethodsUsing(None)
loader.loadTestsFromTestCase(TestEffortFormula)
suite = loader.suiteClass()

but from here I don't know how to run the tests, specially with testRunner=TeamcityTestRunner() as I did before.

Appreciate your help

Answer 1

Option 1.

One solution to this (as a workaround) was given here - which suggests writing the tests in numbered methods step1, step2, etc., then collecting and storing them via dir(self) and yielding them to one test_ method which trys each.

Not ideal but does what you expect. Each test sequence has to be a single TestClass (or adapt the method given there to have more than one sequence generating method).

Option 2.

Another solution, also in the linked question, is you name your tests alphabetically+numerically sorted so that they will execute in that order.

But in both cases, write monolithic tests, each in their own Test Class.

P.S. I agree with all the comments that say unit testing shouldn't be done this way; but there are situations where unit test frameworks (like unittest and pytest) get used to do integration tests which need modular independent steps to be useful. Also, if QA can't influence Dev to write modular code, these kinds of things have to be done.

Answer 2

I've searched a long time to solve this problem myself.
One of the answers in this question does exactly what you need.

Applied to your code:

ln = lambda f: getattr(TestEffortFormula, f).im_func.func_code.co_firstlineno
lncmp = lambda _, a, b: cmp(ln(a), ln(b))
unittest.TestLoader.sortTestMethodsUsing = lncmp

suite = unittest.TestLoader().loadTestsFromTestCase(TestEffortFormula)
unittest.TextTestRunner(failfast=True).run(suite)

Unfortunately, setting unittest.TestLoader.sortTestMethodsUsing=None does not work, although it is documented that this should avoid sorting the tests alphabetically.