How emphasize restrictions for input data in unit testing?

Question

I have parametrized test, for example:

import pytest
from datetime import time


def get_next_hour(time_str: str):
    t = time(*map(int, time_str.split(':')))
    new_t = time((t.hour + 1) % 24, t.minute)
    return new_t.strftime('%H:%M')


@pytest.mark.parametrize(
    "t_str, result_str",
    [
        ('10:30', '11:30'),
        ('11:30', '12:30'),
    ]
)
def test_good_input(t_str, result_str):    
    result = get_next_hour(t_str)
    assert result == result_str

Test test_good_input must work only with valid time strings (for invalid data I have another test test_bad_input). How I can emphasize it - in docstring, or using assert for input data?

With docstring

def test_good_input(t_str, result_str):
    """for t_str allowed only time-valid strings"""

    result = get_next_hour(t_str)
    assert result == result_str

With validation input

def test_good_input(t_str, result_str):

    assert ':' in t_str, 'input data is not time'

    result = get_next_hour(t_str)
    assert result == result_str

Or there is another ways?

Answer 1

I'd say a fat warning in the comment or the test function docstring should be enough. This is not an untrusted user input that should be validated; also, it's better to keep the tests as simple as possible. If some dev misuses the test inputs without reading the docs first, it's his own fault now.

However, test args validation is surely possible with pytest (e.g. to inform the devs that there's nothing wrong with the tested function and they are using the test wrong). I would do an implicit args validation by using the indirect parametrization. In the below example, each arg from mark.parametrize will be first passed to a fixture with the same name where you can do preprocessing before the test starts:

def validate(input):
    try:
        datetime.strptime(input, '%H:%M')
    except ValueError as e:
        pytest.fail('Your test parametrization is wrong. The test argument is erroneous: {}'.format(e))


@pytest.fixture
def t_str(request):
    validate(request.param)
    return request.param


@pytest.fixture
def result_str(request):
    validate(request.param)
    return request.param


@pytest.mark.parametrize(
    "t_str, result_str",
    [
        ('10:30', '11:30'),
        ('11:30', '12:30'),
        ('10:30', 'bar'),
    ],
    indirect=True
)
def test_good_input(t_str, result_str):
    ...

Now the third test will fail with a descriptive error message:

test_spam.py::test_good_input[10:30-11:30] PASSED
test_spam.py::test_good_input[11:30-12:30] PASSED
test_spam.py::test_good_input[10:30-bar] ERROR

============================================= ERRORS ==============================================
__________________________ ERROR at setup of test_good_input[10:30-bar] ___________________________

...

input = 'bar'

    def validate(input):
        try:
            datetime.strptime(input, '%H:%M')
        except ValueError as e:
>           pytest.fail('Your test parametrization is wrong. The test argument is erroneous: {}'.format(e))
E           Failed: Your test parametrization is wrong. The test argument is erroneous: time data 'bar' does not match format '%H:%M'

test_spam.py:15: Failed
================================ 2 passed, 1 error in 0.05 seconds ================================

More info on indirect parametrization: Deferring the setup of parametrized resources

Answer 2

You certainly can add extra comments or the like. But, you already have some natural places to express your intent: Your test function can be given a more descriptive name than test_good_input, the arguments of your test function can be given more descriptive names, and variables, helper functions etc. all can have descriptive names.

For example, you could name your test function test_getNextHour_withValidTimeString_shouldGiveOneHourLaterTimeString. I would argue that this says everything. If you rename t_str to valid_time_str or something similar, this additionally conveys the message also at the top of the parameter field.

I would not add assertions or whatever - giving an invalid time string as input will lead to a failing test case anyway.