Skip to content

Django tips: scaling an application

Published on: July 3, 2006    Categories: Django

In today’s ripped-from-the-mailing-list Django tip, we’ll be looking at a common scaling pattern: an application which starts out with one user, then has to gain separate “instances” for each of multiple users. And for bonus goodness, we’ll scale it even further to work on multiple sites simultaneously.

Let’s build a blog

For purposes of this example, let’s say that you’re building a blog with Django; that was the example in the mailing-list thread, so I’ll run with it, but there are lots of types of applications that could go through this process.

When you first start your weblog it’s just you writing the entries, so you really don’t need a whole lot. The following model will do quite nicely: 

class Entry(models.Model):
    title = models.CharField(maxlength=250)
    slug = models.SlugField(prepopulate_from=('title',))
    pub_date = models.DateTimeField(default=models.LazyDate())
    summary = models.TextField(null=True, blank=True)
    body = models.TextField()
    
    class Admin:
        pass

A title, a slug for the URL, a date, an optional summary and a body, with admin goodness thrown in; looks like we’ve got all the bases covered. Wire it up to generic views, fill out some templates and you’d literally have a fully functional blog in just a few minutes.

Blogs for everybody!

But after a while you decide you want to offer blogs to all your friends, too. To accomodate that, let’s set up a new model: 

class Weblog(models.Model):
    title = models.CharField(maxlength=250)
    slug = models.SlugField(prepopulate_from=(\'title\',))
    author = models.ForeignKey(User)
    
    class Admin:
        pass

And remember to add from django.contrib.auth.models import User at the top of your model file. So now we can set up multiple Weblogs, each with their own author. How do we tie entries to them, though? By making a small change to the Entry model. Add this: 

weblog = models.ForeignKey(Weblog)

Now you’ll need to do a little bit of database surgery to make this work; first go in and create a Weblog for yourself; it’s the first one you created, so it’ll get an id of 1. Then you can execute the appropriate SQL to change the Entry table; for example, the easiest way to do this with PostgreSQL would probably be to do it in three steps:

  1. Add the new column for the foreign key, and give it a default value of 1 (this will insert a value of 1 for all existing entries, which is correct since they all belong to your weblog).
  2. Add the foreign-key constraint.
  3. Drop the default value.

The SQL might look like this: 

ALTER TABLE weblog_entry ADD COLUMN weblog_id int DEFAULT 1 NOT NULL;
ALTER TABLE weblog_entry ADD FOREIGN KEY (weblog_id) REFERENCES (weblog_weblog);
ALTER TABLE weblog_entry ALTER COLUMN weblog_id DROP DEFAULT;

This assumes that your models live in an application called “weblog”; weblog_entry and weblog_weblog are the names Django will give to the tables in that case.

What about the views? Your entry URLs will now contain two slugs: one for the entry itself, and one for the weblog it belongs to. This means you can’t use generic views directly, but you could very easily wrap the generic views with your own function which looks up the weblog from the slug and changes the queryset parameter you passed in to return only entries belonging to that weblog (by appending filter to the QuerySet).

To handle the security implications of multiple users blogging, you could wrap around the create, update and delete generic views with a little bit of code which makes sure that the person requesting access is logged in and is the author of that particular weblog; assuming those tests pass, you’d just call the generic view as normal (and, of course, if they fail you can redirect to a login page or show an error).

You’d probably also want to have some way to show a list of all the weblogs you’re running on the site, and for that the object_list generic view would still work out-of-the-box.

And in just a few minutes you’ve gone from having one weblog for yourself to having one weblog for for you and one for each of your friends.

No, really, blogs for everybody!

Suppose that along the way you meet a few people who own their own domain names, but don’t have any hosting and want to have blogs. Nice guy that you are, you think to yourself, “well, I’ll host their domains and set up blogs for them!” But how do you do that in Django?

The answer is Django’s built-in “sites” framework, which allows you to tie objects to a particular site. To accomodate your domain-owning friends, you’d start out by installing django.contrib.sites (assuming you haven’t installed it already; the admin app used to require it), and then you’d make a change to your Weblog model by adding this line: 

site = models.ForeignKey(Site)

And, of course, up at the top of the file you’d add from django.contrib.sites.models import Site. Then you’d need to do a bit more database surgery, very similar to what you did in the last set of changes: 

ALTER TABLE weblog_weblog ADD COLUMN site_id int DEFAULT 1 NOT NULL;
ALTER TABLE weblog_weblog ADD FOREIGN KEY (site_id) references (django_site);
ALTER TABLE weblog_weblog ALTER COLUMN site_id DROP DEFAULT;

This will relate all existing weblogs to the first site in the database (which, of course, will be your own domain and has all of the weblogs created so far).

Next you’ll want to create separate settings files for each domain you’re adding; each one will need its own MEDIA_URL and other settings. You’ll also want to do two things to make sure everything works out properly for administering the different sites:

  1. Create a new Site object in your admin for each domain, and put the id of that Site into its settings file as SITE_ID so Django knows which site in the database corresponds to this settings file.
  2. In the settings file for your original site (the one with id 1), add the other sites’ settings files to the ADMIN_FOR setting, to let Django know that this one instance of the admin application will handle all of the sites.

Now you can use the single Django admin running on your domain to create weblogs which will show up on your friends’ domains.

Then it’s time to look at the views. Initially we were just using generic views to display the entries, but then we needed to write wrappers around them to make sure they filtered according to the correct weblog. It might be tempting to use the same idea somehow, by looking up the domain name in the incoming HTTP HOST header, but Django provides an easier way: the CurrentSiteManager in the “sites” application. You’ll need to make one last set of changes to your model file; first, add from django.contrib.sites.managers import CurrentSiteManager at the top, and then on the Weblog model add these two lines: 

objects = models.Manager()
on_site = CurrentSiteManager()

You won’t have to change your database for this, because it didn’t add or change any fields. All it did was tell Django to add the CurrentSiteManager as a manager for the Weblog model, and to keep the default manager (called objects) as-is (this is because adding custom-defined managers to a model removes the default objects manager unless you say otherwise, and because Django uses the first defined manager as the default).

Now you can use Weblog.on_site anywhere that you’d previously used Weblog.objects; for example, in the object_list view — which I mentioned might be a good choice for listing all the weblogs — you could pass Weblog.on_site.all() as the queryset parameter, and you’d get back only the weblogs that are part of the current site. In much the same way, you could use Weblog.on_site.get() instead of Weblog.objects.get() in the “wrapper” functions which deal with entries — that way you could look up whether the weblog slug in the URL corresponds to a weblog on the current site, and raise a 404 if it doesn’t before passing on to the generic view to get that weblog’s entries.

Wasn’t that easy?

So now we’ve gone from a simple weblog application just for yourself, to multiple weblogs for you and all your friends, to multiple weblogs spread out over multiple domains. And through it all, the code changes have been fairly light; they probably don’t represent more than a few hours of work, and you’re likely to spend much more time on writing HTML and CSS than on application code.

It really is that easy.