Have a single test method return multiple test results

Question

Before I get you all confused, let me clarify: I'm NOT asking about running a single test method with different arguments. All clear? Then let's go:

I have a test in Python (Django, but not relevant) that basically...

• starts a http server,
• starts Selenium, opens a web page on this server,
• via Selenium loads and runs a suite of JavaScript tests (via Jasmine)
• collects the results and fails if any test failed

I'd like to make the output of the each Jasmine spec visible as a separate entry in Python unit test output (with its own name)? Extracting it from Javascript via Selenium is the easy part, but I don't know how to connect it with the UnitTest machinery.

Expected code would look something like (pseudocode):

class FooPageTest(TestCase):

    def setUp(self):
        # start selenium, etc

    def run(self, result):
        self.run_tests()
        for test_name, status, failure_message in self.get_test_results():
            if status:
                result.add_successful_test(test_name)
            else:
                result.add_failed_test(test_name, failure_message)

Expected output:

$ python manage.py test FooPageTest -v2
first_external_test ... ok
second_external_test ... ok
third_external_test ... ok

A gotcha: The number and names of test cases would be only known after actually running the tests.

Is it possible to bend unittest2 to my will? How?

Answer 1

It sounds like you have multiple external tests to run, and you want to have the results of each test reported individually through Python unit test. I think that I would do something like:

class FooPageTest(TestCase):
    @classmethod
    def setUpClass(cls):
        # start selenium, etc
        cls.run_tests()

    @classmethod
    def getATest(cls, test_name):

        def getOneResult(self):
            # read the result for "test_name" from the selenium results
            if not status:
                raise AssertionError("Test %s failed: %s" % (test_name, failure_message)
        setattr(cls, 'test%s' test_name, getOneResult)

for test_name in get_test_names():
    FooPageTest.getATest(test_name)

This approach does a couple of things that I think are nice:

• It runs the tests when the tests would be run by test discovery, not on module import
• Each selenium test generates to a Python test.

To use this, you'll get to define get_test_names(), which reads the names of the tests that will be run. You'll also need a function to read each individual result from the selenium results, but it sounds like you must already have a way to do this (your get_test_results() method).