On “conceptual integrity”
Mark Ramm has a nice post this morning talking about Adam Gomaa’s discussion of “conceptual integrity” in Python web frameworks; I think it’s worth a read.
The key point Mark touches on is good documentation: ensuring that people see and understand the overarching structure of the framework is, ultimately, the key to helping people understand how it can help them get their work done. In the case of Django there is an official “overview” document, but it’s really not as good as it could be; it’s basically a whirlwind tour of components, and really has to be coupled with at least one other document (the design philosophies explanation) to provide the beginnings of a complete picture. I think Django could do a better job with that.
I think Pylons and TG face the same issue, though — as Mark explains, far more clearly and accurately than I ever could — from a different angle, since there is no one set of components to focus on (at least not in Pylons; TG takes a stand on defaults, which allows a little more room but also complicates explanations of swapping things out).
In other words, the thing that’s really missing (as far as “conceptual integrity” is concerned) in the new wave of frameworks is a good explanation of not just the ideology but also the practicality behind their philosophies: Django’s philosophy document explains what some of the ideas are, but doesn’t really get into solid examples of how they benefit Django applications, for example. Ever since Jonathan posted his explanation of why he uses TurboGears, I’ve been trying to put together a similar post on why I, personally, prefer Django. One of the questions I asked him in the comments there — about a different level of “pluggability” enabled by the ability to mostly assume the default stack — is the key, but I haven’t yet been able to put it into words.
Of APIs and applications
What I’d like to point at is something like the sidebar on this page, which is pulling objects from multiple applications and displaying them together; I can do that because I’ve written a set of generic content-retrieval tags which work with any Django template and with any model based on the Django ORM, and they’re useful to a lot of people because those people also use the Django template system and the Django ORM.
As I’m so very fond of saying, there’s a trade-off involved. If you take a little bit less flexibility (or, rather, perceived flexibility; I have a much longer set of thoughts on the accuracy of that perception) in “swappability” of framework components, you can get a useful benefit in return: a set of common APIs that you can rely on, in exactly the way that my generic content tags rely on Django’s model-loading API, or that things like my user-registration and user-profile applications rely on the APIs and near-universality of django.contrib.auth.
But we (the entire Django community, really) don’t do a very good job (yet) of bringing that to the forefront; for example, unless you read the right blogs or go digging into the Django source code, you’ll be blissfully unaware of the fact that get_model() exists. And even though they’re documented, a lot of people still don’t realize how easy it is for one Django application to expose functionality to others through things like context processors.
And as far as I know there really aren’t any good examples of what, to me, is the greatest strength of Django: the notion of truly “pluggable” applications. This site — just this site, that you see here — uses over a dozen different applications, each of which encapsulates one particular piece of functionality in a reusable way. If I need comment moderation on something that isn’t a blog, I can just reuse the same application that’s doing it here. If I need tagging for something that isn’t a collection of links posted to del.icio.us, I can just reuse the same application that’s doing it here. Everybody realizes that you can use and reuse the admin interface or the auth system, but because those come bundled with Django I think it takes a lot of people (myself included) longer to realize that any Django application can be made generic and reusable like that.
We’ve been doing that for years at World Online, where our entire platform is based around plugging applications together like this, but unless you pony up for an Ellington license you might never know that’s how it works. And that’s a shame, because the level of pluggability a well-designed Django application offers is really what makes Django such a great tool, at least to me.
Maybe the best example is by way of comparison to a non-Python framework: when Django first appeared I was playing with Rails, and the available options for this sort of reusable functionality in Rails all, frankly, sucked: engines and plugins and what-have-you, and none of it was really elegant. When I realized the level of reusability Django’s architecture enabled, it was literally a revelation, but I had to come to that realization on my own: it’s not really explained anywhere in the official documentation. When Adrian mentioned it at the “Snakes and Rubies” talk, it got Rails people to sit up and pay attention. But to this day the official tutorial develops one application, and then hands you over to the reference docs.
How and why
I think that if we’re going to talk about “conceptual integrity”, whatever that actually means, these sorts of discussions need to come to the forefront: given the architectural choices the framework has made, what does the big picture look like and how does it make web development easier or faster? Given the trade-offs made in the overall design, what sorts of patterns of development emerge and how are they helpful?
Or, succinctly, why this set of philosophies and trade-offs, and not that set? Why did the framework’s developers think these were useful choices to make, and how do they affect me as I’m developing applications?
Ultimately, I think that’s where any sense of conceptual integrity has to come from, and I don’t know that any of the new-ish Python frameworks — Django, TurboGears or Pylons — are doing a great job of explaining it just yet.