Living without projects
The official Django tutorial walks you through the process of developing a Django application inside a Django project; in this view of the world, the “project” is seemingly the major focus, and the “application” just happens to be something you stuff inside it. For the purposes of learning and exploring Django — especially if you’re new to Python and unfamiliar with things like how to create your own modules and set up your Python path — this is a pretty good way to work, because it means that django-admin.py and manage.py can do most of the heavy lifting for you.
But in successful deployments I’ve seen and been a part of, this is pretty much never the way things are done; the concept of the “project” as a thing that’s worth dealing with and paying attention to, aside from its incidental duty of providing a settings file and a root URLConf, is pretty rare, and the focus is, instead, on applications. Malcolm touched on this today with an excellent write-up of application-centric development practices which you really should read, and I’d like to follow up today with some more thoughts on the same topic.
What do we mean by “project”?
The first thing to deal with is exactly what’s meant by this word, “project”, because it seems to have multiple interpretations depending on who you talk to. To many people, it seems to mean “the place where all of my site’s code goes”, which is something that’s sort-of true when you’re walking through the tutorial (if you ignore the use of the admin application and generic views, neither of which live inside your project). To others, it seems to mean more of a wrapper, a convenient place where you drop in some configuration while all the actual code — applications, templates, etc. — lives elsewhere (in the case of Python code, typically in application modules directly on the Python path). To yet another group, there’s no such thing as a “project” in most cases.
I tend to work almost exclusively in the latter style, and most real-world Django deployments I’ve seen or worked with have done the same. Of course, that may be a bit of a jump if you’re accustomed to thinking of projects as the place where everything has to go (and if you’ve worked with some other frameworks which don’t have a distinction similar to the one Django draws between projects and applications, it can be a natural thing to do), so let’s look at how an “application-centric” development style helps you out in the long run.
The pitfalls of project-centered development
As Malcolm points out, and as most people find out once they start developing moderately-complex applications, the biggest drawback of trying to put everything inside a project is that everything ends up coupled to the project; you either have to maintain that directory structure everywhere you use one of the applications (so that the imports, which will depend on the existence of the project directory, will work) or you have to do some gymnastics with your Python path to ensure that the individual applications are importable independently of the project. At which point the alternative — developing the applications directly on the Python path, independently of any particular project — really becomes attractive.
It’s also generally much easier to produce distributable applications when you’re developing them independently of a project; the two most common Python distribution mechanisms — distutils and setuptools — are both at their best when you have some flexibility to place distribution scripts (such as setup.py and a package manifest) in a directory above the actual application module; project-centric development impairs this severely.
So what are projects good for?
Aside from providing a place to keep configuration (e.g., the settings file and the root URLConf), there’s really only one use case I know of for projects containing actual code, and that’s when there’s some bit of code which is so coupled to a single deployment that it doesn’t logically make sense anywhere else. One common example is a site-specific “home page” view, which is logically part of the project it’s used in.
I don’t really use this myself, though; I typically have enough site-specific code that it makes sense to put it in its own module and treat it like another application. For example, I’ve written about the system of redirects I use on this site to handle the transition from an older URL scheme, and that code — along with a few other things specific to this site — lives in an application module named b_list, which is directly on my server’s Python path.
There is no spoon, and no project either
From this point it’s a fairly easy jump to just not bothering with “projects” at all, at least not as first-class modules in their own right. At work we manage a lot of Django-powered sites which use Ellington, our news-oriented CMS. Ellington is, ultimately, just a large collection of Django applications which provide different components of a news site, and so all of those applications live inside an encompassing module named ellington which sits directly on a server’s Python path. Meanwhile, the settings files for all of our sites live in a different directory (called, appropriately enough, settings), and root URL configurations for the sites live in a urls directory (though I’m simplifying a bit here for sake of example; there are some particular aspects of our setup that aren’t really relevant to this discussion).
This means that there’s really not anything, anywhere on any of our servers, which corresponds to a “project”, and it’s actually an incredibly flexible way to work; individual sites’ configurations can be edited easily, and new sites can be rolled out with a minimum of boilerplate (since all that’s needed on the Python side of things is a new settings file and a new URLConf).
How I work
For one more real-world example, I’ll go over how I manage my own personal development work, because I have a setup that’s pretty simple to work with, but which makes writing and maintaining my applications fairly easy.
First of all, I have a directory, ~/dev/personal/, on my laptop which holds the full setup of every application, including its parent directory for packaging purposes; for example, django-registration lives in ~/dev/personal/django-registration/. This directory, and the package directories inside it, are not on my Python path.
Meanwhile, a second directory — ~/dev/python-local/ — is on my Python path, and it holds all of the third-party Python software I use. Each of the package directories in ~/dev/personal/ contains a Python module which holds the actual application code, and I’ve just got symlinks from ~/dev/python-local/ into those directories, which means that the application modules are right on my Python path, with the packaging information conveniently out of the way. I do something similar for my checkout of Django, because I’ve got a copy of the full tree — branches, release tags, trunk and all — and there’s a symlink in ~/dev/python-local/ which points at the trunk/django/ directory of that checkout (and which can easily be pointed somewhere else if I need to work against a specific release version of Django).
From there it’s pretty easy to use various flexible solutions for settings files and URL configuration; I’ve got two hierarchies of settings and configuration data — one for my personal testing setups and one for the sites I deal with at work — and just point to whatever’s appropriate for a given situation.
And, again, there’s absolutely no need to have “projects” in this setup. Like Malcolm, I still do use a project module every once in a while as a quick way to get something up and running, but I inevitably factor everything out of it before I get close to deploying or distributing. The result is a simple — but still flexible and powerful — development platform which translates well into production deployment or packaged distribution, depending on what I’m working on.