Another take on content negotiation

November 29, 2008 Django, Python

Today my co-worker Daniel posted a class which performs content negotiation for Django views, allowing you to write a single view which returns any of several types of responses (e.g., HTML, JSON, XML) according to the incoming HTTP Accept header. While it’s certainly cool, he notes a couple of issues with it (including the redundancy it introduces in URL configuration).

So let’s see if we can’t come up with a way to work around this.

Holiday decorations

It seems that the best way to approach this is to avoid the need to do too much rewriting in the view and URL code; it’d be nice to simply say, after the fact, what types of responses the view should be able to return without changing the view too much or tweaking the URL configuration over and over. And the natural way to do this in Python is with a decorator, so the goal here should be to write a class which generates decorators which handle the content negotiation. And while we’re at it, let’s cut down on some of the repetition by allowing the class to take the set of content types and template names only once, and then reuse them. Ideally, we’ll end up with something which would work like this:

xml_or_html = MultiResponse({ 'text/html': 'hello.html',
                              'application/xml': 'hello.xml' })

@xml_or_html
def hello(request):
    return { 'message': 'Hello, world!' }

The hello view would then, when called, use the hello.html template and return a Content-Type of text/html if the Accept header states a preference for it, or use the hello.xml and return a Content-Type of application/xml if the client’s Accept header prefers that. The only real difference in the way the view is written is that it doesn’t directly return an HttpResponse; instead it returns a dictionary, and our decorator will deal with the problem of turning that dictionary into a template context, rendering the template and returning the response. So let’s get started on the code.

A decorator generator

We’re going to end up defining a class named MultiResponse, whose instances will be able to act as decorators. The first part of this is easy; we’ll just define the class and give it an __init__ method which takes a dictionary mapping content types to template names:

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

Now we need to make sure instances of MultiResponse can be used as decorators. A Python decorator is normally described as a function which takes another function as an argument, and returns yet another function (whose behavior is slightly different), but in reality the “functions” involved can be any callable Python objects, and it’s easy to make practically anything in Python callable. Generally speaking, in Python some_function(some_argument) is syntactic sugar for some_function.__call__(some_argument), so defining a method named __call__ on our class will make it callable. Let’s start with just that:

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self):
        pass

Now, what we actually want to do is pass some other callable — a view function — to __call__() in order to decorate that view, so the argument signature actually should be:

def __call__(self, view_func):
    pass

And then to actually decorate the view, we need to define a new function which calls the view (which in this case will return a dictionary instead of an HttpResponse), then chooses a template, renders a response of the appropriate Content-Type using that template and finally returns the response. We can start on that by defining the new function, setting it up with an appropriate argument signature (it needs to accept an HttpRequest and then any number or positional or keyword arguments, since we don’t know what additional arguments the view is expecting), and call the view with those arguments to get the dictionary which will become the context:

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)

(remember that in Python, the * and ** syntax allow a function to receive, and then pass on, any combination of positional and/or keyword arguments)

Now we need to figure out which template and Content-Type to use, based on the request’s Accept header. That header will be in request.META[‘HTTP_ACCEPT’] and will consist of one or more content types (and optional “quality” parameters to state a preference for one type or another) separated by commas. Daniel’s original MultiResponse did some simple parsing to figure this out, but whenever I have to do content negotiation in Python I invariably use Joe Gregorio’s mimeparse.py, which implements the HTTP specification’s rules for determining which content type is “preferred” out of an Accept header. The function we want from mimeparse is best_match(), which takes a list of supported content types and an Accept header, and returns the best supported content type out of the available choices, so we can get the content type from that:

import mimeparse

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])

This simply uses the keys in the template_mapping dictionary (which, remember, will map content types to template names) as the supported content types, and gets the best match out of them. From there, we can use render_to_response to get an HttpResponse object by looking up the template name for the content type we’re using and passing in the context dictionary obtained from the view:

import mimeparse

from django.shortcuts import render_to_response

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])
            response = render_to_response(self.template_mapping[content_type],
                                          context_dictionary)

Finally, we can set the outgoing Content-Type header and return the response:

import mimeparse

from django.conf import settings
from django.shortcuts import render_to_response

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])
            response = render_to_response(self.template_mapping[content_type],
                                          context_dictionary)
            response['Content-Type'] = "%s; charset=%s" % (content_type, settings.DEFAULT_CHARSET)
            return response

Note that the Content-Type header involves two parts: the actual content type, and the character set being used. While you can omit the charset parameter, it’s generally a bad idea to do so (especially if you’re working with XML — see Mark Pilgrim’s “XML on the Web has Failed” for an excellent writeup of some of the issues). The DEFAULT_CHARSET setting in Django provides the character set used by default for HTTP responses, so we just use that as the charset value.

Now we have the fully-defined function our decorator will need to produce, so __call__() can simply return it and we’ll be done:

import mimeparse

from django.conf import settings
from django.shortcuts import render_to_response

class MultiResponse(object):
    def __init__(self, template_mapping):
        self.template_mapping = template_mapping

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])
            response = render_to_response(self.template_mapping[content_type],
                                          context_dictionary)
            response['Content-Type'] = "%s; charset=%s" % (content_type, settings.DEFAULT_CHARSET)
            return response
        return wrapper

Improving `MultiResponse`

At this point we’ve met our original goal; we could write view code like the example above, and it would work as expected:

xml_or_html = MultiResponse({ 'text/html': 'hello.html',
                              'application/xml': 'hello.xml' })

@xml_or_html
def hello(request):
    return { 'message': 'Hello, world!' }

Or for one-off uses we could write it like so:

@MultiResponse({ 'text/html': 'hello.html', 'application/xml': 'hello.xml' })
def hello(request):
    return { 'message': 'Hello, world!' }

The hello view would, depending on the incoming Accept header, change change its template and the Content-Type of its response. But with a couple of changes, our MultiResponse class can become a bit more useful:

It’d be nice to give it the ability to use RequestContext to automatically populate common variables we always want to have.
Currently, the eventual decorated view function will lose its docstring, and Django’s built-in admin documentation system won’t be able to show any useful information about it, because we’ve thrown away all the original information about the view we’re decorating.

The first improvement is easy enough; we can simply use the context_instance argument of render_to_response, and have MultiResponse take an argument telling it whether to use RequestContext:

import mimeparse

from django.conf import settings
from django.shortcuts import render_to_response
from django.template import Context
from django.template import RequestContext


class MultiResponse(object):
    def __init__(self, template_mapping, request_context=True):
        self.template_mapping = template_mapping
        self.request_context = request_context

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            context_instance = self.request_context and RequestContext(request) or Context()
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])
            response = render_to_response(self.template_mapping[content_type],
                                          context_dictionary,
                                          context_instance=context_instance)
            response['Content-Type'] = "%s; charset=%s" % (content_type, settings.DEFAULT_CHARSET)
            return response
        return wrapper

The request_context argument to __init__() defaults to True, but allows this to be overridden when needed, and __call__() simply looks at that to determine what type of context to use.

Preserving information about the original view function, so that things like automatic documentation will continue to work, is a bit trickier because it involves fiddling with attributes of the decorated function that don’t normally get fiddled with. Fortunately, Python 2.5 introduced the functools module into the standard library, which contains (among other things) a function called update_wrapper; given a function being returned in a decorator, and the original function being decorated, update_wrapper copies over things like the docstring to the new function. And since this is generally useful (and needed for things like Django’s admin documentation), Django bundles a version of this function in django.utils.functional, backported to work with Python 2.4 and Python 2.3. So we can try to import update_wrapper from the standard library and fall back to Django’s copy if it’s unavailable, then use it to copy over the original view function’s attributes:

try:
    from functools import update_wrapper
except ImportError:
    from django.utils.functional import update_wrapper

import mimeparse

from django.conf import settings
from django.shortcuts import render_to_response
from django.template import Context
from django.template import RequestContext


class MultiResponse(object):
    def __init__(self, template_mapping, request_context=True):
        self.template_mapping = template_mapping
        self.request_context = request_context

    def __call__(self, view_func):
        def wrapper(request, *args, **kwargs):
            context_dictionary = view_func(request, *args, **kwargs)
            context_instance = self.request_context and RequestContext(request) or Context()
            content_type = mimeparse.best_match(self.template_mapping.keys(),
                                                request.META['HTTP_ACCEPT'])
            response = render_to_response(self.template_mapping[content_type],
                                          context_dictionary,
                                          context_instance=context_instance)
            response['Content-Type'] = "%s; charset=%s" % (content_type, settings.DEFAULT_CHARSET)
            return response
        update_wrapper(wrapper, view_func)
        return wrapper

And we’re done.

This sort of system is extremely useful for writing, for example, web-based APIs which often need to tailor their response types to a client’s needs, and for doing so based on nothing more than standard information you can already get from HTTP (so that you don’t need to muddy your URL configuration or require GET arguments to determine the response type), as well as a few other tasks. Since this is an area where different people will probably need very different behaviors, however, I don’t think it necessarily needs to become part of Django; given writeups like Daniel’s and this one as starting points, it ought to be easy to produce solutions tailored to specific needs.

Another take on content negotiation

Holiday decorations

A decorator generator

Improving MultiResponse

Improving `MultiResponse`