Skip to content

PyCon ORM panel liveblog

Published on: March 28, 2009    Categories: Django, Frameworks, Python

I’m sitting about five rows back in the ballroom at the Hyatt, waiting for the ORM panel to begin.

Panel’s starting. Moderator Alex Gaynor introduces himself and panelists:

First question: brief history of your ORM

SQLALchemy: wanted to do a CMS, but never ended up writing it. Had idea for a modular library to talk to databases, different bits you could use or not use — connection pooling, SQL expressions, ORM — as you liked. Background was from previous work doing stats and content management for major-league baseball, which involved lots of complex SQL.

Django: began life at a small family-owned newspaper in Kansas, as internal content-management system. Jacob says it’s really been through three versions. First version was inspired by SQLObject. Second version was rewritten by magic-removal effort. Third, current version came from major refactor by Malcolm Tredinnick. Jacob says he’s “far from qualified” to speak about ORM.

SQLObject: Ian started working on it in 2001, first release 2002. It was something he found handy as an abstraction on top of the Python DB-API. Realized later he was doing Active Record pattern. Originally did code generation, which offended his sensibilities, so he started using metaprogramming instead. Eventually passed maintenance on to another developer.

web2py: In 2007 started using Python in classes at DePaul University. Wanted to have class on web frameworks, looked into using existing frameworks but decided to build a new one the students could study — existing frameworks were still changing too quickly. Ended up gaining drivers for many databases. ORM layer was meant to express any possible SQL statement, and express syntax with operators instead of keyword arguments.

App Engine: Guido’s not sure why he’s on this panel, he’s never written or used an ORM. App Engine isn’t relational, so “the R is not relevant”. Originally had low-level API built around dictionaries, but that was cumbersome, so a layer was written on top of that which introduced App Engine’s model class. Supports a very minimal subset of SQL to do some types of queries, but most queries are done by writing method calls.

Second round

What should the relationship be between ORM and non-relational DBs?

SQLAlchemy: People should be able to use whatever they want, but SQLAlchemy has chosen to focus on SQL and will never implement a backend for a non-relational store. Easily switching an existing project from, say, an RDBMS to CouchDB doesn’t seem possible. And at any rate, a goal of SA is to keep things explicit and out where you can see them (non-relational support might not be able to do this in the same library).

Django: We’ve never viewed what we have as an “ORM”, we’ve viewed Django’s model layer as a model layer: objects and operations on sets of objects, happen to be backed by an RDBMS. So the model layer should work well on a non-relational DB; it’s theoretically possible, but nobody’s done the work to find out where the problems would be. Would take someone doing the work to find where bugs are. Jacob agrees that writing an application first in a relational frame of mind, then deploying non-relational probably won’t work — need to choose one up-front when writing an application.

web2py: Already ported DB abstraction layer to work on App Engine. Committed to supporting more functionality — sees commonalities rather than differences. Not every query expression will work on App Engine, but that’s OK; things that don’t work on a given backend (e.g., App Engine doesn’t have a concept of transactions) just don’t work.

App Engine: Give users what they want. Hopes someone will get Django models working on App Engine.

Chatter (Jacob, Massimo, Guido) about different mindsets involved in relational and non-relational application design. Guido agrees that porting from one to the other with the same app is a non-starter, but choosing a framework shouldn’t tie you into one option.

Mike brings up the distinction between a model and a representation of a query. Even if you could, say, map a SQLAlchemy class on to a non-relational DB, you can’t still do the same types of queries, can you? Massimo says no, he’s interested in the types of things you can do on both.

Round three

Should tools for working with databases map exclusively to what exists in the database, or should it be more full-featured?

Guido starts to say something, there’s an interruption about ZODB. Guido likes explicitly stating schema definition (as opposed to ZODB mapping Python objects into data store). Massimo agrees. Jacob takes a “very slightly” different viewpoint: there’s value to declaring models in Python because you can have metadata and constraints the DB can’t support. Mike says models should do more than just talk to DB — they’re supposed to have business logic.

Audience questions

Question: Cite a feature or concept from someone else’s ORM that you think is well-done and you’d like to adopt?

Guido: Django’s admin. Jacob says the fact that it’s tied to models is a bug.

Ian: SQLAlchemy’s separation of concepts.

Jacob: “Expando” feature from App Engine. But relational DBs can’t do it.

Massimo: SQLAlchemy’s support for existing database schemas.

Question: This panel’s about design decisions. What benefits/drawbacks do you see from design choices like Active Record or Data Mapper patterns, lowest-common-denominator support, etc., especially with DBs adding new features all the time?

Jacob: Django doesn’t have, say, an IPV6 field, but you can define one if your DB supports that data type. That’s important.

Mike: SQLAlchemy has a lowest-common-denominator way to use, but that’s not what they aim for. If you want to tie yourself to advanced Postgres features, SQLAlchemy will let you and will support them.

(Massimo gave a short answer I didn’t quite hear)

Question: Running on alternate Python VMs?

Mike: Targeting Python 3.0, Jython and IronPython. Broke up DB dialect support from code that knew DB API quirks. Important to do this since on alternate VMs you use different adapter modules.

Jacob: Django runs (or is meant to run) on all the major VMs. Support for VM-specific DB adapters is up to community, though. Talking about similar breaking up of information to avoid duplicating code between backends which use different driver models on same DB.

Massimo: Built-in web server in web2py and built-in DB drivers don’t work on alternate VMs. Anyone an expert on Jython and JDBC?

Guido: No plans for App Engine to run on alternate VMs.

Question: Any frameworks addressing things like OLAP/data warehousing? Should frameworks do this?

Mike: says he doesn’t really have an answer, he doesn’t work with that stuff. Jacob agrees, saying he’s unqualified. Ian passes, as does the rest of the panel.

Question: What are your plans for migrating to Python 3?

Mike: SA has 2-to-3 converter.

Jacob: Django supports older Pythons and has to manage transition steadily/gradually. Excited about proposed “3to2” project.

Ian: Doesn’t maintain SQLObject anymore, doesn’t know its roadmap.

Massimo: Python 3 poses both technical and social problems. Technical problems are minor. Social problem is the main thing: web2py has a backwards-compatibility commitment for applications, need to be careful about breaking it.

Guido: Google tends to lag a bit on versions because it’s hard to switch. App Engine isn’t completely tied down, but porting will be a very large project, akin to supporting another whole language (and they’re already working on other languages, but Python 3.0 isn’t one of them).

And with that Alex says we’re out of time.