On Django’s longevity
“Django is about to be a teenager.”
I don’t remember exactly who said that, but it came up in a discussion of potential followup events to 2015’s “Django Birthday”, held in Lawrence, Kansas, to celebrate the tenth anniversary of Django’s initial public release. There might still be some parties to throw — maybe a Django sweet sixteen — but regardless of what the future holds, the summer of 2018 will mark thirteen years since the first public release, and Django will officially be a teenager.
Most pieces of software don’t make it to 13 years. Many don’t make it to 10, or even 5. But Django is still here, still pumping out releases, still going strong. I saw an article recently which surveyed Stack Overflow questions year to year, and found interest in Django is remarkably stable.
So how did that happen?
Before I try to answer that question, a reminder: this is a blog. My blog. It’s where I post my opinions, and opinions are not objective truth; if I ever need to speak ex cathedra (assuming I find some topic I could even do that with), I’ll be sure to mark it clearly, So you may well disagree with what I have to say below, or feel that I’m biased. I certainly am, and so is everyone else, and I’m not presenting this as anything other than my own opinion.
So. Here goes.
There’s an app for that
Yes, the ecosystem of third-party more-or-less-pluggable Django applications is a huge advantage. But how did we get there?
For (literally) a decade now, I’ve been telling anyone who will listen that Django’s concept of an application is one of its secret weapons. At first, a lot of people asked why that mattered — after all, Python uses WSGI and WSGI has an application concept, and WSGI applications are composable! Why would Django’s variation on that matter?
If you’re not familiar with WSGI — the gateway protocol for Python web applications — it’s a CGI-like programming model (and I’ve criticized it in the past, on grounds that we really ought to be able to do better than CGI‘s model). A WSGI application is a Python callable with a specific signature:
def application(environ, start_response): # Actual application code goes here.
Here, environ is a Python dict corresponding to the environment variables a CGI web application would access, and start_response is a callable which can be invoked with an HTTP status code and headers (and optionally information about exceptions raised). Then the application callable returns an iterable of the response body.
WSGI applications are composable, then, but they are composable in the sense that, say, HTTP proxies are. A WSGI application never (officially) knows, and often does not care, whether it has been invoked directly by the server, or by some other WSGI application proxying a request to it. This allows WSGI applications to call each other and act as “middlewares”, and provide additional functionality (for example, a WSGI application might proxy to a second application, and only modify things by compressing the other application’s response when appropriate).
But there’s no explicit knowledge of what other applications are present or what functionality they might provide. Like HTTP proxies, which communicate by setting and reading particular standard or custom header values, WSGI applications communicate by setting and reading particular standard or custom keys in the environ dict which gets passed around, and interoperability comes solely from knowing which keys to look for and what their values indicate.
Deployment of Django takes place as a WSGI application; Django provides a callable implementing the WSGI signature, and then hands off to its own machinery before returning a WSGI-appropriate response.
But a deployment of Django almost always consists of multiple Django applications, which gives us a hint that a Django application is something different. A succinct description would be that a Django application is an encapsulated piece of functionality, including all the code necessary to provide that functionality and also to expose it for use by Django or by other Django applications.
Django itself provides just enough consistency and standardization and API support to make this work. Django applications are expected to know about each other, and to import and call and subclass and use each other’s code as needed.
To take an example: this site, that you’re looking at right now, uses over a dozen Django applications. Some come with the framework itself and are bundled into django.contrib, like the auth system and the session framework. Others I wrote myself specifically for this site, like the app which provides the models, views and URL routing for my blog posts. Still others were written by me, or other people, for generic re-use, like the contact form or the app that generates my Content Security Policy header.
You don’t get the gigantic ecosystem of Django apps without the framework providing the tools to make that happen. And although people have periodically suggested that Django could do more — such as more enforced structure, or options for applications to supply configuration automatically — I think it’s pretty clear that the Django application, as a flexible abstraction for writing reusable functionality, has been a gigantic success.
It’s also worth noting, since one of the biggest debates in the early days of Django was whether it would lose out to frameworks which were agnostic about component choices (i.e., bring your own preferred ORM, template language, etc. and the framework is mostly glue code to plug them together), that Django shipping its own set of components and being relatively tightly coupled to them — some of the nice integrations still work when you swap out some components, but not all, and especially not the ORM — has been one of the biggest enablers of the Django application boom. A Django application can make a lot of assumptions about available components, and doesn’t have to go through a bunch of abstracted/indirect APIs to try to make all components look the same to the framework. This does mean you lose some flexibility — you can’t, for example, just decide to drop the Django ORM and use SQLAlchemy and still have everything, or even most things, work — but the payoff from that is, well, the entire ecosystem of Django applications.
Django is boring
I love boring software, and apparently a lot of other people do, too. And Django is a great example of boring software.
One way Django is boring is its pace of change: you can go look up Django applications written years ago, or documentation for ancient versions of Django, and a huge amount of what’s presented will still work, or require only very minor changes. There was a big rewrite of the ORM between 0.91 and 0.95, which required models and query code to be updated in basically every app, but that’s the only large-scale backwards-incompatible break Django has ever had, and it happened way back in 2006.
There have, of course, been backwards-incompatible changes since then, but Django uses a combination of slow deprecation cycles and long-term support releases to minimize the impact. If you stick to an LTS, you get three years of upstream support, and in the post-Django-2.0 world, there are strong guarantees being made that if your code runs on an LTS release without raising any deprecation warnings, you can jump to the next LTS release with no code changes.
And in spite of changes over time, code written with Django today looks remarkably like code written a decade ago. Most of the changes have been in favor of making already-existing things easier and more consistent, rather than change for change’s sake, and several massive code changes have been pulled off with few or no backwards incompatibilities in APIs provided for developers. For example, the ORM has actually been rewritten multiple times over Django’s history; you just wouldn’t notice all of them unless you were using a bunch of undocumented internals, or you were otherwise digging around in the source.
The persistence, not just of particular APIs but of the cohesive feel of “Django-ness”, makes it easy to learn Django once and then come back to it years later without facing a massive learning curve.
Consistency over time isn’t the only way Django is boring, though. Django is also reliable: you know you can deploy something built with Django and, if your pager goes off at 2AM on a weekend, it’s almost certainly not going to be due to something wrong with Django. And if there is a serious bug, you know it’s going to be fixed soon; there’s a regular cadence for bugfix releases (they come out monthly), and show-stoppers can get releases on a faster schedule if they’re bad enough, though I don’t recall the last time there was a bug bad enough to force an emergency release (there was one security issue that memorably resulted in me rolling a release at around 2AM in a hotel room in Denver, but that had more to do with it being initially reported publicly).
Of course, like any open-source project that survives long enough, Django does have its share of ancient tickets sitting open in the tracker, but most of them are feature requests that never got a design fleshed out, or major changes that probably will never happen all these years later (did we ever finally close the super-old one that asked to have the ORM completely redesigned? I should check).
Finally, in a highly subjective sense, Django is software that’s used in boring ways. I don’t meant to say nobody’s ever done something hip and trendy with Django — plenty of people have, or have tried — just that it feels like Django doesn’t have a reputation for that. Instead, Django — remember the motto, “The framework for perfectionists with deadlines” — mostly seems to have a reputation for being software that people just use to get things done, without a ton of fanfare for the technology involved.
I have a lot of strong thoughts on that and why it’s a good thing, but all I’ll say for now is that the longer I work in tech, the more convinced I am that lots of conversations about or focus on technology choices at a company is a sign of other things being badly wrong.
Mostly, though, Django-powered sites and services seem to be pretty low-key about their tech stacks, to such a degree that I often find I’ve seen or used something a bunch of times, and only much later discover it was built with Django. That may not be someone else’s idea of boring, or of the good kind of boring-ness, but it is mine.
Django isn’t perfect
In the early days, a common criticism of Django was that none of the components it shipped with were the best available in Python. People would say SQLObject (remember SQLObject?) was clearly the best Python ORM, or put forward Cheetah or Genshi or Mako or plenty of other candidates as the best Python template language, FormEncode as the best forms library, WebOb/Paste as the best HTTP request/response abstraction, and use these as arguments against Django. People still do this today, to an extent, by pointing out that they’d rather use SQLAlchemy than Django’s ORM (though Django’s ORM is now a lot more powerful than people expect or remember from the old days, but that’s a story for another time)
And — from the perspective of someone who wants to choose the best available library for each role and glue them together — they were right. Django’s ORM has never been the best ORM available in Python. Django’s template library has never been the best template library available in Python. None of the components Django ships with are the best version of that component you could get.
And that’s OK. Remember that tagline? Django didn’t advertise itself as the perfect framework. It advertised itself as “the framework for perfectionists with deadlines”. That means it’s OK not to have the absolute best possible ORM, for example, or the absolute best possible template language or forms library or the best possible anything. Django’s job is to deliver good, not best or perfect, and I think Django has been very successful at doing that. For a working programmer, perfect is the enemy of good, and shipping something good quickly is better than maybe being able to ship something perfect much later.
There are people who would strongly disagree with me on this, and who prefer frameworks that are built to let them mix and match components to their heart’s content, or frameworks which try to pick the best of each type of component at a given moment, and focus on making them work with each other. And that’s OK, too. This is why Flask and Pyramid and other frameworks exist, and people use them. My personal preference, though, is for Django’s approach; I think it’s paid off in (subjectively) higher practicality and ease of use, and that those are a big part of why Django has been so successful.
The same thing is true in a lot of other aspects of Django. For example, I hear sometimes that Django has a reputation for security. But we’re very far from perfect on that (though to be fair, nothing is or can be perfect when it comes to security). In fact, last year at PyCon I gave a tutorial on the topic which opened by mentioning that in the (at the time) twelve years since Django’s initial release, we’d averaged one disclosed security issue every 66 days. Still, we’re apparently doing something right, or right enough, to have a reputation.
Django can’t protect you from everything, of course, or even from most things. But it does its best to protect you from or mitigate common security issues. Django also tries to make it easy to do the right thing for cases where a global default behavior isn’t sufficient; for example, several basic best practices around running over SSL are quick toggles in settings. And at the level of the framework itself, there’s a security policy which, while not perfect (and improving a few sections of it is on my to-do list), has at least led to a productive, mostly cooperative relationship with people who find issues in Django.
I could go on about this for a while, but I think the point is clear enough: Django isn’t perfect, but Django is and consistently has been good, and that goes a long way.
Django is made of people
Finally, Django is more than just the framework, or the app ecosystem. It started out as an internal tool at a single company, but then was open-sourced, and then a nonprofit foundation was created to be the steward of the copyrights and trademarks (necessary disclaimer: since January 2016 I’ve served on the Board of Directors of that foundation; this blog post is still my own personal opinions and does not represent an official position of the DSF or its Board). It started out with two “BDFLs” — Adrian and Jacob — who then stepped down from that role and were replaced with a rough consensus model and a rotating technical board to act as the ultimate tie-breaker/decision-maker when all else failed (necessary disclaimer number two, I suppose: I served on the technical board for the Django 1.10, 1.11, and 2.0 release cycles; this blog post is still just my personal opinions).
There’s a trend here: openness and bringing in more people, while keeping central “ownership” or authority as limited and little-used as possible.
So while the DSF owns and protects the copyrights and trademarks, and collects and distributes donations to support and promote Django, and the technical board serves as a backstop, “Django” isn’t and never can be owned by any one person or entity. It’s the result of a worldwide community of people. And it’s not just the code of the core framework.
There are people who use Django to teach programming and web technology. There are people who put on conferences and social events. There are people who write about Django, or try to improve it, or try to make learning and using it easier. They’re respectful and friendly and welcoming and thoughtful and helpful and patient and many other virtues besides. Prior to Django, I’d never been part of, or heard of, a community around a software project that was as good as what coalesced around Django.
I think part of the reason for this was touched on above, in some of the ways Django is “boring”. Part of it was Adrian and Jacob leading by example in the early days, which attracted like-minded people who carried on the good work. Part of it was that Django, led by two liberal-arts majors turned programmers working at a newspaper, launched with very good (by the standards of open-source projects in 2005) documentation, written in a friendly and accessible way, and has continued to emphasize good documentation ever since. Part of it was probably just plain luck; some truly incredible people have, for whatever reason, decided to join the Django community and start doing wonderful things. I feel lucky to have gotten to meet and work with them, and — through roles like serving in the DSF — to support their efforts.
And I believe the community — however and whyever it formed and has kept going — is absolutely one of the strongest selling points of Django. And it shows no sign of fading any time soon.
Ite, missa est
I could probably ramble on for a good while longer, but I’ve hit the main points that, in my opinion, form the core of why Django has been successful over a (for an open-source web framework) long period. You may have noticed that none of them were things like “blazing fast performance” or “massive scalability at the push of a button” or “written by rockstar guru ninja wizard 1000x programmers”. There’s a reason for that, and if you’re unsure what it is, re-read the sections above.
Meanwhile, here’s to another thirteen years of Django, and a continuation of all the good things I’ve listed above.