Use unittest’s subtest helper
This is part of a series of posts I’m doing as a sort of Python/Django Advent calendar, offering a small tip or piece of information each day from the first Sunday of Advent through Christmas Eve. See the first post for an introduction.
Python testing frameworks
The Python standard library ships with the unittest module for writing tests. The first thing I want to mention about it is that it gets a lot of hate it doesn’t necessarily deserve. Often, the API of unittest is criticized for being “un-Pythonic”, especially for its use of method names like setUp() or assertEqual() which don’t follow the usual style guide for Python naming.
But the reason for this is that unittest wasn’t really written to be a “Pythonic” unit-test library; it began as a third-party package called PyUnit, and was a direct port of the “xUnit” style of testing framework into Python. Other languages have their own ports: JUnit in Java, NUnit for .NET, and so on. The unittest module is consistent with the “xUnit” style and it was written to be familiar to anyone who’d used an xUnit-style testing framework. It also was added to the Python standard library over 20 years ago, and after this long it’d be pretty pointless (and a violation of the PEP 8 style guide!) to break a bunch of code just for the sake of renaming parts of its API.
Many people nowadays prefer the third-party pytest testing framework; personally I have a slight preference for the unittest style of testing, but that’s neither here nor there, because the thing I want to talk about today is the unittest equivalent of a popular pytest feature. And if you’re using Django, its default testing tools are all built on top of unittest, so you have to do some work (and install several more third-party packages) to get equivalent functionality on top of pytest.
Consider a test
Suppose you need to test a bunch of values as inputs to some particular function, and assert that it does the right thing. Here’s a silly example:
import unittest
from other_module import is_even
class EvenNumberTest(unittest.TestCase):
def test_is_even(self):
for number in (2, 4, 6, 8, 10):
assert is_even(number)
This works, but it has some issues: if the assertion fails on any of the inputs, it will fail the whole test without giving you much diagnostic information to know which input it failed on. In the pytest world you can do this instead:
import pytest
from other_module import is_even
@pytest.mark.parametrize("number", [2, 4, 6, 8, 10])
def test_is_even(number):
assert is_even(number)
The parametrize decorator in pytest will run the test function multiple times, each time with one of the values (or sets of values — you can parametrize multiple arguments) from the given list being passed to the test function as an argument. This way if only one of them fails, you see that the others succeeded and you get feedback on which one failed.
While it’s not completely equivalent to parametrize, the key use case — running the test for all the inputs, and reporting exactly which one(s) failed — is, as I recently found out, supported in unittest and has been since Python 3.4, via the subTest() feature. Here’s the above test rewritten with subTest():
import unittest
from other_module import is_even
class EvenNumberTest(unittest.TestCase):
def test_is_even(self):
for number in (2, 4, 6, 8, 10):
with self.subTest(number=number):
assert is_even(number)
Similar to the parametrize example for pytest, this will run the assertion once per input, will run it for all the inputs even if one of them fails, and will report the failures individually, along with the input that caused the failure.