So you want a dynamic form

November 9, 2008 Django, Programming, Python

So I havent really been doing much writing lately. That’s mostly a consequence of the fact that:

Django 1.0 was released, and meeting the schedule for that took up an enormous amount of time.
After that, there was DjangoCon.
Oh, and there are all sorts of things in my life, including a book, a gigantic codebase and dozens of sites, which all need to be updated. Our entire team here in Lawrence has been running short on sleep for a good long while now.

Anyway, I’m going to force myself to break out of that every once in a while and write something useful and fun, because if I don’t I’ll probably go insane. Fortunately, there’s a whole community full of people talking about Django these days, so I’m not short on topics.

Most recently, I’ve been quietly watching some discussion in the Django ticket tracker and on IRC about how to build “dynamic forms” in Django, where a “dynamic” form is one whose behavior or fields change depending on, say, the current user or some other bit of current information. This isn’t something that’s really documented, but personally I think that’s because it falls into a sort of gray area; like many useful things you can do with Django, it’s not actually Django-specific. It just requires a little bit of knowledge of Python and of some standard programming practices. In a nutshell, there are two ways you’ll want to go about this:

Write a form class which knows how to adapt itself, say by doing some shuffling around in __init__() (and possibly taking some custom arguments to tell it what sort of shuffling to do).
Write a factory function which knows how to build and return the type of form you want.

So let’s dive in.

Constructing in your constructor

The easiest and most common way to do a little bit of dynamic customization in any Python class is by overriding its __init__() method and adding your own special-case logic there. For sake of an example, let’s consider a contact form: you want a form that lets people get in touch and send you messages. But since there are a lot of evil spammers out there, you also want to avoid automated submissions. Suppose you put together a CAPTCHA system and implement it as a Django form field, maybe called CaptchaField; then you’d just add it to the definition of your form, like any other field.

But let’s add a twist to this: let’s say that we only want to have this field when the form is being filled out by an anonymous visitor, and not when it’s coming from an authenticated user of the site. How could we do that?

One way would be to write two form classes — one with the CaptchaField and one without — and have the view select one after checking request.user.is_authenticated(). But that’s messy and leaves duplicated code lying around (even if you have one form subclass the other), and Django’s all about helping you avoid that sort of thing. So how about a form which automatically sprouts a CaptchaField as needed? We can do that pretty easily with a custom __init__() method and a single extra argument: we’ll have the view pass in the value of request.user, and work out what to do based on that.

Here’s how it might look:

class ContactForm(forms.Form):
    def __init__(self, user, *args, **kwargs):
        super(ContactForm, self).__init__(*args, **kwargs)
        if not user.is_authenticated():
            self.fields['captcha'] = CaptchaField()

That’s not so hard. All it requires is a little bit of knowledge of Python (namely, overriding __init__() to change the way instances of the class are set up, and using super() to get the parent class’ behavior as a starting point), and a tiny bit of knowledge about how a Django form works: it has a dictionary attached to it, named fields, whose values are the names of the form fields and whose values are the fields themselves. Once we have that, it’s easy to just stick things in self.fields as needed.

Similarly, you can use this to modify the behavior of a field in the form; if you wanted to change the max_length of a CharField, for example, you could simply pull it out of self.fields and make the changes you wanted. You could also overwrite things there with completely new fields according to what your form will need. And you can even compare the fields as they currently exist with the fields as they were originally defined on the form class: the base set of fields defined for the class lives in a dictionary called base_fields and isn’t overwritten when you change fields, so comparing the contents of self.fields to the contents of self.base_fields lets you see quickly what’s changed (and unless you really know what you’re doing, don’t mess around in base_fields; you’ll thank me for that advice later).

These sorts of techniques can take you a long way; I’ve written some fairly interesting and complex forms using nothing but __init__() tricks (features at work like this one are constructed using some extremely-dynamic form systems, for example), and generally it’s pretty easy and pretty useful.

Form, function and factory

Of course, there are limits to what’s practical to do in an overridden __init__(); sooner or later the amount of code required (and the gymnastics you’ll hve to do for things like custom validation methods) simply gets to be too much and you find yourself needing something a little more powerful. Typically, the solution for those cases is a factory function (or factory method, if you prefer) which knows how to build form classes.

One good example is in django-profiles, which needs to generate forms based on the user-profile model you’re using, and also needs to ensure that certain fields never show up in them (since, for example, we don’t want you being able to change the User instance associated with your own profile). This is solved by two functions in django-profiles: one knows how to retrieve the currently-active user-profile model, and the other knows how to build a form for it, minus the User field. The second function is the one we’re interested in, and the code for it is short and sweet:

def get_profile_form():
    """
    Return a form class (a subclass of the default ``ModelForm``)
    suitable for creating/editing instances of the site-specific user
    profile model, as defined by the ``AUTH_PROFILE_MODULE``
    setting. If that setting is missing, raise
    ``django.contrib.auth.models.SiteProfileNotAvailable``.
    
    """
    profile_mod = get_profile_model()
    class _ProfileForm(forms.ModelForm):
        class Meta:
            model = profile_mod
            exclude = ('user',) # User will be filled in by the view.
    return _ProfileForm

You’ll notice that the docstring for this function is almost as long as the actual function itself; all it does is fetch the user-profile model (that’s what get_profile_model() does), then define a ModelForm subclass based on that model and exclude the user field (because of the way user profiles work in Django, the field must be named user; if that requirement didn’t exist, introspection of the profile model could just as easily determine which field to exclude). Then it returns that newly-defined class.

Then, whenever a form for editing user profiles is needed, the view which will display the form can simply do something like this (the get_profile_form() method lives in a module in django-profiles named utils):

form_class = utils.get_profile_form()
if request.method == 'POST':
        form = form_class(data=request.POST, files=request.FILES)
        if form.is_valid():
            # Standard form-handling behavior goes here...

This works because Python supports a few features (most notably closures, a topic far too big to tackle here) which make it easy to define and work with new classes while your code is running. And in general, this is an extremely powerful feature to have: it means that instead of writing huge, complex classes which try to anticipate everything you might need to do, you can usually just write one or two simple functions which build and return brand-new tailor-made classes that do exactly what you want.

The example above was fairly easy because it just defined a ModelForm subclass, and those are typically short and easy to write, but it works just as well with more complex forms; you can define a full form, with fields, custom validation and whatever else you like, inside a function and then return it to be used immediately. There’s also a useful shortcut you can use if you know you’re just going to slap together a collection of fields and don’t want to bother with writing them all out: Python’s built-in type() function, which puts together a class on the fly. The type() function takes three arguments:

The name you want to give to your class.
A tuple of one or more classes it will inherit from, in order.
A dictionary whose keys and values will end up as the basic attributes of the class.

Let’s return to our original example: a contact form which sprouts a CAPTCHA field if the user isn’t authenticated. We could write a function which takes a User as an argument, and then returns either of two possible form classes, like so:

def make_contact_form(user):
    # The basic form
    class _ContactForm(forms.Form):
        name = forms.CharField(max_length=50)
        email = forms.EmailField()
        message = forms.CharField(widget=forms.Textarea)
    if user.is_authenticated():
        return _ContactForm
    class _CaptchaContactForm(_ContactForm):
        captcha = CaptchaField()
    return _CaptchaContactForm

This defines the form class on the fly, and if the user isn’t authenticated immediately defines a subclass which includes the CAPTCHA. But this is pretty ugly; let’s see how it works using type():

def make_contact_form(user):
    fields = { 'name': forms.CharField(max_length=50),
               'email': forms.EmailField(),
               'message': forms.CharField(widget=forms.Textarea) }
    if not user.is_authenticated():
        fields['captcha'] = CaptchaField()
    return type('ContactForm', (forms.BaseForm,), { 'base_fields': fields })

That’s much nicer; type just wants to know the name of our new class (ContactForm), what classes, if any, it should inherit from (when doing this, use django.forms.BaseForm instead of django.forms.Form; the reasons are a bit obscure and technical, but any time you’re not declaring the fields directly on the class in the normal fashion you should use BaseForm) and a dictionary of attributes. Recall from our earlier look at fiddling with a form’s fields that base_fields is the attribute which holds the baseline set of fields for the form class, so our attribute dictionary simply supplies base_fields with a collection of form fields, and adds the CAPTCHA only when needed.

This is a bit more advanced in terms of Python programming, but is an extremelyuseful technique; Django uses it all over the place to build classes on the fly, and you can too.

Where to go from here

There is one further way to dynamically control the way a class is set up in Python, and it gives you absolute control over every step of the process; writing a “metaclass”. But, in the words of Python guru Tim Peters: “Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t.” If you ever do get to the point where you’re convinced you do need them, there are plenty of tutorials on Python metaclasses floating around the Web. Django uses them in a few places (in fact, in more places than it strictly needs to), but that doesn’t mean you have to; doing work in __init__() or writing a factory function which knows how to build the class you want will easily cover 99% or more of all the cases you’re like to run into in the real world.

And though here we’ve been talking about Django forms specifically, I’d like to point out once again that this isn’t really Django-specific; these approaches work well with lots of different types of probems in Python and even in programming in other languages (although not every language makes it as easy to, for example, define new classes on the fly). Reading up on Python and getting to know some standard idioms and programming patterns will drastically improve your ability to reason out solutions like the ones I’ve given above, so if you’re interested in further reading, curl up with either the Python documentation or some material on programming practice; they’ll do you good.