Let’s talk about documentation

June 21, 2008 Django, Pedantics, Programming, Python

One of the most active threads on reddit’s programming section right now discusses things people look for when reviewing someone else’s code; the article being discussed treats this as a great interview question and points to things like algorithm choices and object-oriented design as good responses. While these are important considerations, I’ve found I tend to make snap judgments long before I get to that level of analysis, and they’re almost always based on one key factor: documentation.

Of course, I have the luxury of mostly reviewing Python code, and Python makes it ridiculously easy to embed useful documentation. What’s more, automated API-documentation generators don’t seem to be as popular in Python as they are in some other languages, which means the lazy attitude of “a big list of function names and argument signatures should be all the documentation you need” thankfully is not prevalent in Python.

So when I’m looking at someone’s code, the first thing I typically notice is how well it’s documented. And I’ve found that the presence of useful documentation is quite often an indicator of a particular developer’s quality; in my experience, it’s rare to find bad code with good documentation. So I’d like to run through a few documentation-related things that I usually look for when I’m reading a piece of code for the first time.

Two types of documentation

Python, like a few other languages (e.g., members of the Lisp family), actually offers two ways to document any given piece of code. One is by inserting comments — prefixed, in Python’s case, with the # marker — at relevant places throughout the code. The other is via docstrings, which allow longer, free-form documentation to be associated with (typically — in theory, any Python object can have a docstring) a particular function, class or module. This is interesting for two reasons:

Unlike comments — which are simply discarded — docstrings can be programmatically accessed, meaning that you can extract documentation from Python code using nothing more than standard Python features. This is in stark contrast to some languages where specialized tools have to be written to parse source code and look for specially-formatted comments.
The fact that there are two methods of supplying documentation, although it seems at first to go against Python’s philosophy of “There should be one— and preferably only one —obvious way to do it”, encourages developers to think about the distinction between two important types of documentation they can insert into code.

In certain circles, comments in code are frowned upon, if not actively discouraged, because the presence of a comment is seen as an indicator that the code by itself was not clear enough to communicate its purpose and workings to the reader. Thus, the need to provide explanatory comments is occasionally seen as a weakness which should be fixed by writing better code.

I’d really rather not get into that debate here, but I do think that Python’s solution of offloading documentation into a tailor-made language feature rather than overloading the comment mechanism draws a clear distinction between documentation which explains what a piece of code does and occasionally how it does it, and documentation which explains why a particular technique was chosen. In my opinion, the first two items — “what” and “how” — generally belong in docstrings, and the third — “why” — typically should either be made clear by the code or explained in a comment.

Since this is a somewhat nebulous thing to say, let me take a real-world example. In django-registration, a new, inactive user account has an activation key associated with it; this key is a salted SHA1 hash, and to activate the account the new user must visit a page with the activation key in the URL. The method which generates this key explains what is happening and how:

def create_profile(self, user):
    """
    Create a ``RegistrationProfile`` for a given
    ``User``, and return the ``RegistrationProfile``.
    
    The activation key for the ``RegistrationProfile`` will be a
    SHA1 hash, generated from a combination of the ``User``'s
    username and a random salt.
    
    """

But the default URL pattern which routes to the activation view — and thus which needs to match the activation key to pass it as an argument to the view — looks like this:

url(r'^activate/(?P<activation_key>\w+)/$',
    activate,
    name='registration_activate'),

This is a bit strange. The activation key, as we already know, is a SHA1 hash, which serializes to a 40-digit hexadecimal number. The regular expression which matches an activation key, then, is [a-fA-F0-9]{40}, but the regular expression used in this URL pattern is \w+, which will match any sequence of word characters. There’s a good reason for this: if the pattern only matched things that look exactly like SHA1 hashes, it could easily result in failed matches for things like incorrect copying and pasting from an activation email, returning a “404 not found” instead of a more useful error message saying that the activation key was invalid.

Now, this is a detail that really doesn’t need to be mentioned in high-level documentation, but someone casually reading the code could easily spot this and wonder if it’s a bug and there’s not really any way to rewrite the code so that the reasoning behind it will be clear. Thus, an explanation of why this URL pattern uses the regex it does is provided in a comment directly above the pattern. Here’s the full pattern with the comment:

# Activation keys get matched by \w+ instead of the more specific
# [a-fA-F0-9]{40} because a bad activation key should still get to the view;
# that way it can return a sensible "invalid key" message instead of a
# confusing 404.
url(r'^activate/(?P<activation_key>\w+)/$',
    activate,
    name='registration_activate'),

Distinguishing between these questions and the places where they should generally be answered — “what” and “how” in docstrings, and “why” in comments — is a big thing that I look for when reading code. Also, failure to distinguish between docstrings and comments, or recognizing only one or the other, is something that’ll get you marked down in my book; I’ve often griped, for example, about Ohloh, which treats documentation as a metric for judging code quality but refuses to recognize docstrings in Python code as documentation.

Being obsessive about documentation

Before I go any further, let me just get this out of the way: I’m really, really, really anal about documentation. Those of you who know me or who’ve read through some of my code can attest to the fact that I try quite hard to practice what I preach, sometimes taking it to a ridiculous level. I don’t expect everyone to take documentation to the extremes I do, but I do expect good code to include good documentation. Python has a style guide for docstrings which includes some good, sensible recommendations, and following that can take you quite a long way. Let me add a few tips:

Yes, PEP 8 says to “[w]rite docstrings for all public modules, functions, classes, and methods.” But no, you really shouldn’t do that, because you’ll end up with a lot of really dumb docstrings.
Instead, document things which don’t have a clearly-established meaning, and document anything which does have a standard meaning but is deviating from that standard. For example, a Django model class typically defines a method named get_absolute_url(), and the purpose of this method is well-understood; it’s a convention of the Django framework and is documented as such, so constantly repeating a docstring which says “Return the public URL for viewing this object” is pointless. On the other hand, if you’re writing a model and you override the save() method to provide custom behavior, you should give that a docstring to explain what it’s doing that varies from the standard behavior.
Building on the style guide’s recommendations, don’t put something in a docstring if it can be obtained by introspection; for example, a module docstring doesn’t need to list the module’s contents, because that can be worked out programmatically (and most documentation tools that work with Python do so automatically).

And if you want to go the hardcore route, you can try something I find myself doing fairly often and which, in a slightly drunken chat with Michael, Eric and others at PyCon, I dubbed “Documentation-Driven Development”. You may already be familiar with the idea of “Test-Driven Development” or “TDD”, where you write a unit test for a new piece of code before you write the code itself, in which case you can guess what I’m getting at: I typically write the docstring first, and then write the code to match it.

Of course, it’s not always that easy, and typically I start with just a short outline and flesh it out as I go, but I’ve found this to be an incredible aid to producing usefully-documented code. As an added bonus, it meshes neatly with a couple more aphorisms from the Zen of Python:

If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.

If I can’t write a simple, short docstring describing what a piece of code will do, it’s almost always a bad idea to try to write it without first putting some more thought into the design. And since it’s much easier (from a psychological perspective) to scrap a fledgling docstring and start over than it is to throw out a piece of code, this has saved me quite a lot of time and effort; instead of realizing after the fact that I’d written some code badly, or written some code I shouldn’t have, I’ve found myself realizing ahead of time that I’m describing code I shouldn’t write at all, and stopping.

I don’t think this should be the only way to write code (I don’t always write this way myself), and I don’t think anyone should be forced to do it, but I have found it to be quite useful; if you’re feeling adventurous, give it a try sometime and see if it works well for you, too. If nothing else, evidence of obsessive documentation practices will win you major brownie points if I ever have to review your code.

Functional documentation

There’s one more thing that really stands out, even at a glance, in well-documented Python code: functional examples. Just including an example or two of how to use a piece of code goes a long way, but Python can help you get even more mileage out of that documentation. The secret lies in Python’s “doctest” module, which can treat examples in your docstrings as unit tests for your code.

I don’t use doctest nearly as often as I ought to, but here’s a fairly complete example from a little library I wrote for working with HTML and CSS color codes (also: this module, which has about 240 lines of actual Python code in a file over 800 lines long, is a good example of what I meant when I mentioned I occasionally take documentation to “ridiculous” levels):

def rgb_to_hex(rgb_triplet):
    """
    Convert a 3-tuple of integers, suitable for use in an ``rgb()``
    color triplet, to a normalized hexadecimal value for that color.
    
    Examples:
    
    >>> rgb_to_hex((255, 255, 255))
    '#ffffff'
    >>> rgb_to_hex((0, 0, 128))
    '#000080'
    >>> rgb_to_hex((33, 56, 192))
    '#2138c0' 
    
    """
    return '#%02x%02x%02x' % rgb_triplet

At first glance this looks like a perfectly normal docstring: it explains what the function does, and provides some examples to illustrate its use. But lurking down at the bottom of the module is this little piece of code:

if __name__ == '__main__':
    import doctest
    doctest.testmod()

When executed “standalone” as a Python script, this module will invoke doctest which will, in turn, scan the docstrings looking for bits that resemble Python interpreter sessions, like the examples in the docstring above. It will then execute the lines of code in those examples and verify that the results match what’s in the docstrings. In other words, the documentation also serves as a verifiable functional specification for the code, because the embedded documentation can be executed as a unit-test suite.

If you’re using Django, keep in mind that the Django test framework can locate and use doctest-based tests in your code. Quite a bit of Django’s own test suite is written this way, and doctest is used to some advantage to produce files which serve as both API examples and unit tests; for example, this file in Django’s test suite is located by the test system and executed as a set of unit tests, but also can be parsed by the documentation builder to produce this example in the online docs.

Go forth and document

While the obsessive levels to which I take documentation may well be unique to me, looking at documentation as an indicator of code quality (and, hence, developer quality) almost certainly isn’t. So if you aren’t taking advantage of the tools Python gives you to produce well-documented code, start now. And if you’re using Django, you really should start documenting your code, because Django provides some useful helpers for writing and accessing documentation.