Core no more
If you’re not the sort of person who closely follows the internals of Django’s development, you might not know there’s a draft proposal to drastically change the project’s governance. It’s been getting discussion on GitHub and mailing lists, but I want to take some time today to walk through and explain what this proposal does and what problems it’s trying to solve. So. Let’s dive in.
What’s wrong with Django?
Django the web framework is doing pretty well. It’s a stable, popular piece of software. Django the open-source project is OK right now, but there are some things that need to be solved. The biggest issue is one that most large open-source projects run into sooner or later: difficulty in recruiting and retaining enough active contributors.
Some of the biggest open-source projects dodge this by having, effectively, corporate sponsorship of contributions: companies which rely on the software also have employees whose jobs, at least in part, involve helping to maintain it. I suppose this was true for Django in its very earliest days, since it was open-sourced by the company where it was developed and several key contributors were on staff there for some time afterward, but it hasn’t really worked out as a long-term plan.
Which means Django is dependent on the goodwill of people who mostly don’t get paid to work on it. Those are in short supply to begin with, and it’s been even more of a problem in recent years as Django has become sort of a victim of its own success: the types of easy bugfixes and small features that often are the path to growing new committers have mostly been done already in Django.
This doesn’t mean Django is in terrible shape right now, but it does mean Django’s at risk of being badly off in the future; for some time now, the project has not managed to bring in new committers at a sufficient rate to replace those who’ve become less active or even entirely inactive, and that’s not sustainable for much longer.
On top of that, Django has not done a great job of drawing contributors from across its entire user base. Django itself is popular around the world and, thanks to various free workshops and curricula, is a common first introduction to software development for budding coders in the developing world, as well as a frequent introduction to programming for many women who decide to start careers in tech. Yet the current roster of active committers is not at all reflective of that. Only a handful of women have ever been significant technical contributors to Django, let alone committers, and none of them are presently active while the current roster of active committers appears to be entirely drawn from Europe, North America and Australia.
That’s a strong sign the project is not doing a good job of turning developers who work with Django into developers who work on Django. Drawing contributions from the entire population of Django users is a necessity for the project’s long-term survival, and also vital for avoiding the sorts of problems to which a monoculture team is susceptible.
Something needs to change, which is why this proposal exists, and why reforming/reorganizing the Django core team has been a frequent topic for the last couple years on various Django-related mailing lists; the proposal I’m describing here is an attempt to move that forward from discussion into real action. It’s not a panacea, and it’s not going to solve all of Django’s problems overnight. But I believe it is a necessary first step toward being able to solve them.
A bit of background
To understand what’s being proposed, though, it’s useful to understand some of the history of roles and governance in Django as it’s been run up to now.
In the beginning, Adrian, Jacob, Simon and Wilson developed some tooling for their employer (a newspaper, the Lawrence Journal-World of Lawrence, Kansas) to make it easier to build web applications quickly. They got permission to release it open-source under the BSD license, named it “Django”, and unveiled it to the public in mid-2005. As contributions started rolling in from the community, they granted commit rights to a few more people, and the first governance structure coalesced: Adrian and Jacob were “BDFLs” (Benevolent Dictators for Life, borrowing Guido van Rossum’s title from the Python language), but they mostly served as tie-breakers for cases when a decision couldn’t be made any other way. There were also committers who could add or modify code in Django at will, and who helped triage, review and merge contributions from the broader community.
In 2014, Adrian and Jacob announced their intention to retire from the BDFL roles. This didn’t significantly change Django’s governance, since they had rarely been called on to exercise their decision-making powers, but as a deadlock-avoidance measure they were replaced by a new group: the Django technical board. The technical board is made up of committers, elected by the committers, once for each release cycle. The technical board fills the role the BDFLs used to, serves as a final tie-breaker and decision-maker, and has veto power (which it has never exercised) over adding new committers to Django.
The Journal-World originally owned Django, in the sense of being the holder of the trademarks and the copyright to the first publicly-released code (contributors to Django own their own contributions, and agree to license those contributions for use in Django; the Journal-World only owned the original code). But being owned by a for-profit company is an awkward situation for an open-source project, so in 2008 the Journal-World helped to form a 501(c)(3) non-profit — the Django Software Foundation — which could act as the steward and owner of the Django intellectual-property assets. The DSF also engages in fundraising and uses those funds to sponsor Django-related events around the world, as well as to advance the state of Django itself.
Finally, in 2014 the DSF established the position of “Django Fellow”. The Django Fellows are contractors who are paid to help with the day-to-day maintenance and running of the Django open-source project. They triage tickets, review pull requests, make and publish the release packages, answer questions on mailing lists, and do many other vital things within the project.
And for sake of disclosure and background, I know all this, and am making the proposal to change it, because I wear and have worn quite a lot of hats within the Django project and the broader community. Among those: I’ve had a commit bit since 2007; I’ve been on the technical board for five releases; and I’ve been on the Board of Directors of the Django Software Foundation for the past three years.
The proposal
And now I can finally get to explaining what’s being proposed. Keep in mind this is a high-level summary, and omits many details. If you want the entire proposal with all the minutiae, read the draft DEP.
In theory, the current governance model of Django the open-source project is that anyone can propose a change or submit a patch, but all decisions about what goes into Django are made by the committers (“Django core”), with the technical board as a backstop. Committers can make basically any changes they like to Django, and are accountable only to each other and to the technical board.
In reality, very few committers actually use their commit bits. Decisions are made by consensus, involving input from committers and non-committers, on the django-developers list, and most actual commits to the main Django repository are made by the Fellows. Even people who have commit bits of their own, and therefore the right to just push their changes straight into Django, typically use pull requests and solicit discussion, and many committers (there are around 40) are inactive.
At the same time, though, being a Django committer (“core developer”, “Django core”, or whatever you choose to call it) is still seen as a high-prestige title, and committers are given a lot of deference by the wider community. Given that the actual day-to-day governance rarely relies on the committers having any sort of special powers or prestige, this is a problem, especially because it seems to create an impression among potential contributors that they’re not “good enough” to match up to those awe-inspiring titanic beings (please note sarcasm: I am not anything close to awe-inspiring).
So what I’ve proposed is the dissolution of “Django core”, and the revocation of almost all commit bits. They’re mostly unused anyway!
In place of the committers, two roles would be formalized — Mergers and Releasers — who would, respectively, merge pull requests into Django, and package/publish releases. But rather than being all-powerful decision-makers, these would be bureaucratic roles; no single person would have the authority to write up code and push it into Django on their own. The initial set of Mergers would be the current set of Fellows, and something similar will happen for Releasers.
In place of the (in theory) committers making decisions, governance would take place entirely in public, on the django-developers mailing list, with anyone who wants to participate in good faith free to do so, including voting in favor of or against any controversial change. This already basically is the way things work.
As a final tie-breaker, the technical board would be retained, and would have some extra decision-making power it doesn’t currently possess (mostly around participating in the selection of the Merger/Releaser roles as necessary, and confirming that new versions of Django are ready for release). The board would be elected less often than it currently is (probably once per major release series rather than once per minor release), but given that the technical board already has low turnover that doesn’t seem like it’d be a problem. Voting for the technical board would, as with other aspects of Django’s governance, be open to people who engage in good faith on the django-developers list, with the DSF acting as a neutral administrator of the technical board elections (and having the power to reject voters who are acting in bad faith; members of the DSF‘s Board of Directors who stand for technical board elections would, of course, be required to recuse from matters having to do with those elections).
Goals and rationales
Removing the distinction between godlike “committers” and plebeian ordinary contributors will, I hope, help to make the project feel more open to contributions from anyone, especially by making the act of committing code to Django into a bureaucratic task, and making all voices equal on the django-developers mailing list. By itself this won’t solve the issue of recruiting enough new contributors, or the lack of diversity among them, but I think it will put us on much better footing to tackle those problems.
Preserving the technical board as a backstop for resolving deadlocked decisions is probably a necessity. Giving it a little bit more authority than it currently possesses — such as issuing the final go-ahead on releases — is not ideal, but I think will help assuage the fears of people whose immediate reaction to this proposal will be that Django is going to descend into some sort of chaotic mob rule. Eligibility for the technical board currently requires being a Django committer; going forward, it would still require a documented history of contribution to the framework.
Involving the DSF in an oversight role for technical-board elections is a practical matter. Someone needs to do that, and the DSF is the best fit for a neutral entity that nonetheless cares about Django’s long-term survival (in fact, that’s legally one of the purposes of the DSF‘s existence, as specified in its articles of incorporation). And giving it a broad power to monitor the voter rolls serves as a guard against trolls, vote-stuffers and other malcontents (though I still worry about them).
Finally, I think this will bring the formal description of Django’s governance much more in line with the reality of how the project actually works and has worked for the past several years, which is always nice.
Next steps
The full details, which are still being worked out, are in the draft proposal in the DEP repository (a DEP is the Django equivalent of Python’s PEP). I’m still collecting feedback and making adjustments to the proposal in response to that feedback, but I’d like to get it into reasonable shape for voting within the next month.
From there, a lot of bureaucracy has to occur to make this happen:
- The Django core team has to vote on the proposal. Changes to the organization and governance of Django the open-source project require a four-fifths majority to pass.
- The Django technical board — which has absolute veto power over changes to organization and governance — has to consider the proposal and not veto it.
- The Django Software Foundation has to consider the proposal, and its Board of Directors has to vote to accept the DSF‘s role in the new governance structure and to provide any necessary resources.
- Finally, the proposal would have to be implemented, which would involve closing the current django-core mailing list and #django-core IRC channel; revoking a bunch of commit bits and other permissions; and setting up the new governance processes.
All of which will, of course, take time. But my hope is that by the middle of next year, Django can be operating under this governance structure and better able to tackle the remaining issues.
If you’ve got feedback, feel free to leave comments on the pull request in the DEP repository, or get in touch with me privately.