How to break Python

November 28, 2016 Django, Python

Edited March 2019: updated Python 3.8 section.

Edited December 2018: added sections for Python 3.7 and upcoming 3.8, updated Python 3.6 section since Python 3.6 has been released, and updated Python 3.3 section since 3.3 has reached end-of-life.

Don’t worry, this isn’t another piece about Python 3. I’m fully in favor of Python 3, and on record as to why. And if you’re still not convinced, I suggest this thoroughly comprehensive article on the topic, which goes over not just the bits people get angry about but also the frankly massive amount of cool stuff that only works in Python 3, and that you’re missing out on if you still only use Python 2.

No, this is about how you as a developer can break Python, and break it thoroughly, whenever you need to.

Why break Python?

If you maintain applications, libraries or other code written in Python, and you ever intend to distribute any of that code to third parties, you need to make some decisions. You need to decide how to license your code, you need to decide how to distribute it (if you’re open source, the Python Package Index is the place to do it), etc.

But you also need to — and this is an important and oft-overlooked step — decide which versions of Python you’ll support. And here I don’t just mean Python 2 vs. Python 3. If you support Python 2, which specific Python 2 releases do you want to support? On Python 3, which specific Python 3 releases?

Supporting lots of versions of Python can significantly increase your workload in writing and maintaining code. Python improves with each release of the language, but — aside from some 2-to-3 compatibility features present in Python 2.6/2.7, and some security stuff that got backported to 2.7, those improvements don’t migrate backward into older versions of the language. Which means that every version of Python you support adds another set of restrictions on what you can write, and can leave you having to invent workarounds for convenient features that unfortunately don’t exist in a version you’re still supporting. This is a recipe for burning out as a maintainer, so you really need to pick a set of Python versions and stick to that in order to put some limits on how much work you’ll have to do.

Also, once you’ve decided on a set of versions to support, you really want to be sure about it. Many people nowadays are good about having some kind of continuous integration running against a matrix of supported versions, but that only handles half the problem: you know your code works on the versions you support, but are you sure it doesn’t work on unsupported Python versions?

This is more important than it sounds at first. But one of the nightmare scenarios as a maintainer is to suddenly have someone who’s very angry because they didn’t change anything in their setup and suddenly it stopped working. And that’s exactly what happens with code that accidentally “works” — or at least mostly works — on an unsupported Python version. Everything is fine until one day the fateful code path gets reached for the first time, or you push a new release that turns your declaration of incompatibility into actual incompatibility.

So if at all possible, you should take steps to ensure that when you declare a set of supported Python versions, you’re actually enforcing them.

Which versions should you support?

This is entirely up to you, and will depend on how much work you feel like doing and how badly you want features that only exist in newer versions of Python. My own personal policy, though, is:

When I’m writing a generic Python library, I support any version of Python receiving upstream support from the Python core team.
When I’m writing something for use with Django, I look at the current supported versions of Django, and support any version of Python those will run on, minus any which have reached end-of-life (for example, Django 1.8 — still supported by the Django team — supported Python 3.2 at release, but Python 3.2 is now past end of life, so I don’t support Python 3.2 in my personal Django apps).

There are plenty of other sensible options available. You could support only the latest stable 2.x and latest stable 3.x, or only the latest stable 3.x, or only supported 3.x releases, and so on. The important thing is not the specific set of versions you decide on, but that you decide and take action on that decision.

You should also make sure — for anything which will go on the Python Package Index or a similar service — to use trove classifiers in your setup.py to let people know which versions you’re supporting. The full list of trove classifiers is quite large, and lets you also indicate topical categories and things like framework version compatibility, but at the very least you should use the Python-version classifiers so people know what to expect (you can also browse PyPI by classifiers, in order to find, say, game-related code which supports Python 3.5 and is known to work on macOS, or any other arbitrary set of classifiers you care to combine).

How not to break Python

There is a very easy brute-force way to enforce use of specific versions of Python: you can just check the version someone’s running, and crash if it’s not one you want to support. For example, if you wanted to only allow Python 3.3 and later, you could do something like:

import sys

if sys.version_info < (3, 3):
    sys.exit("You must use Python 3.3, or newer.")

This works because sys.version_info is a tuple, and Python supports ordered comparisons on tuples (and on versions of Python which include namedtuple in the collections library, sys.version_info is a namedtuple with useful names on its fields, letting you inspect it semantically instead of having to memorize the indices of the different version components).

But it’s not exactly elegant, and it litters a bunch of these brute-force version checks throughout your code. If you’re OK with that, then it does of course get the job done.

Personally, I prefer an approach which involves simply writing natural Pythonic code to do whatever it is my library or application does, but in a way that also ensures compatibility with only the specific set of Python versions I’ve chosen to support. Which isn’t terribly hard once you know a few things, and those things are what the rest of this article will cover.

This will not, however, just be a list of what was new in each version of Python. Instead, the focus will be on generally-useful things which were added to Python and which achieve one of the following (in decreasing order of how preferable they are to use):

Causing an import-time SyntaxError. This is the ideal, because it ensures absolutely that your code will never work on an unsupported version of Python, with no chance of accidentally being able to sort-of work for long enough to mislead someone.
Causing an import-time ImportError, NameError or AttributeError. This is almost as good as a syntax error, but not quite: these exceptions will get the job done, just without clearly communicating that your code is written for a different version of Python.
Causing an exception fairly quickly and predictably, but not automatically on import. This is the least desirable option, because it runs the risk of someone thinking it’s a bug in your code instead of a deliberate way of breaking compatibility with an unsupported version.

Now let’s get started.

Python 2.6

As I mentioned above, I support any version of Python that still has upstream support, which means that for now I still support Python 2 even if I don’t recommend or use it personally. The only 2.x release still receiving upstream support is Python 2.7, but thanks to long-term third-party support contracts there are people still running 2.6. You really should try to stop supporting or using Python 2.6 as soon as possible, but if you must support it — and I highly recommend you make sure you’re paid well for doing so — here’s how to ensure your code works on 2.6 but not on 2.5:

Use a with statement, and do not include the future import. Python 2.5 only supported with when preceded by a top-level from __future__ import with_statement, so omitting that will cause an import-time SyntaxError on Python 2.5.
Use as in an except block to assign the exception to a variable. Python 2.6 was the first version to support the except ExceptionClass as varname syntax (previously you had to write except ExceptionClass, varname, which was problematic if you had a tuple of exceptions to catch), so the as form is an import-time SyntaxError in 2.5.
Use the b prefix on byte strings. This prefix does not exist prior to 2.6, so again is an import-time SyntaxError in 2.5.
Perform string formatting with the format() method instead of the % operator. The format() method was new as of 2.6, so will cause an AttributeError the first time string formatting attempts to execute in 2.5.

Note: do not attempt to use the print/print() distinction to break Python 2.5 support. Python 2.6 introduced the from __future__ import print_function flag for emulating Python 3’s behavior, but print followed by parentheses is syntactically valid in earlier versions of Python. It just reads as “print this tuple”.

Python 2.7

This is ideally the minimum Python version anyone should support now, which means you’ll want to break Python 2.6 compatibility. For that:

You can use a set literal or set comprehension. Those were new in 2.7 and are a SyntaxError in 2.6.
Similarly, dict comprehensions are new in 2.7 and are a SyntaxError in 2.6.
Use multiple context managers in the same with statement. This is a SyntaxError in Python 2.6.
Use the format() method to do string formatting, but omit positional identifiers in the placeholders (for example, do '{}'.format('something') instead of '{0}'.format('something')). Being able to omit those was a new feature in 2.7, and will raise a ValueError on Python 2.6 when the format() call executes.

Python 3

If you only want to support Python 3, and cut off Python 2 altogether, it’s somewhat tricky precisely because the compatibility tools for supporting Python 2 and 3 in a single codebase were so good. This makes it hard to trigger a generic “this doesn’t work on Python 2” SyntaxError, since Python 3 held off on adding new syntax for a few releases in order to give people a chance to start their porting and shake out issues in the early 3.x releases. Even worse, several syntactic changes got backported for compatibility into 2.7, taking away the option to use them.

The easiest and most obvious thing you can do, syntactically, is to use function annotations. These were never backported into a 2.x release, and are generally useful on a Python 3 codebase. Unfortunately, they’re most useful when combined with the typing type-hint library which was first shipped in Python 3.5, so unless you’re 3.5+ you’ll need to install the generic backported version for earlier Python versions.

If you can’t or don’t want to use annotations, you can:

Declare a metaclass using the metaclass= syntax. This is a SyntaxError on Python 2.7 and earlier.
Use the raise … from syntax when raising a new exception in an except block. This is a SyntaxError on Python 2.7 and earlier.
Use a keyword-only argument in a function definition. This is a SyntaxError on Python 2.7 and earlier.
Use extended iterable unpacking. This is a SyntaxError on Python 2.7 and earlier.

Python 3.3

Python 3.3 was the oldest currently-supported Python 3.x release when this article was originally written, but reached end-of-life in 2017. If you still need to support Python 3.3 but not older 3.x releases:

Use the u prefix on strings. This was legal in Python 2 to indicate a Unicode string instead of a byte string, but not in 3.0 through 3.2. It was re-added in 3.3 (where it’s a no-op since strings are all Unicode no matter what you do) to ease the porting process, and is extremely handy as a way to write code that works on Python 2.7 and 3.3+, but is a SyntaxError on 3.0, 3.1 and 3.2.
Use a yield from statement to delegate to a sub-generator. This is a SyntaxError on Python 3.2, and also in Python 2.

Python 3.4

This one’s tricky, because Python 3.4 did not add any new syntax to the language. So there’s no way to get the ideal situation of code that works on 3.4 and is a SyntaxError in 3.3. However, you can still force some errors in 3.4:

Use the asyncio, enum, or pathlib libraries, all of which are new as of Python 3.4 and so are automatic ImportError in 3.3 (and in Python 2).
Use hashlib.pbkdf2_hmac(). This function is new as of 3.4, and is another handy 2/3 compatibility shim: it doesn’t exist in 3.0, 3.1 or 3.2, but does exist in later 2.7-series releases.
Use pickle with protocol version 4, which was new as of Python 3.4.

Python 3.5

Python 3.5 has new syntax we can take advantage of:

Use the async or await keywords when writing asynchronous code. These are a SyntaxError in Python 3.4 (and in Python 2).
Use the % operator for formatting of bytes objects. This is another handy Python 2 compatibility trick: % formatting worked on byte strings in Python 2, was not implemented on bytes in Python 3.0 through 3.4, and was added to bytes in 3.5.
If you write numeric/scientific code, and have some types which implement it, use the new matrix-multiplication operator: @. This is a SyntaxError on Python 3.4 and Python 2.
Use the new new iterable-unpacking features, which are SyntaxError in Python 3.4 and Python 2.

Note: Python 3.5 added the typing module to the standard library, but as mentioned above it’s available as a separate download from PyPI for earlier Python versions. Which means importing from typing is not guaranteed to break compatibility with older Pythons; someone might have installed it separately, or might go and install it separately to make your code work.

Python 3.6

Python 3.6 made multiple additions to Python’s syntax:

Use f-prefixed strings for formatting operations. These are new as of 3.6, are a SyntaxError in Python 3.5 and Python 2, and save you some typing when you’d just be dumping local variables into a format() call.
Use annotations to give type hints to variables as well as to functions. This is a SyntaxError in Python 3.5 and Python 2.
Use underscores in numeric values. This improves readability (i.e., you can write one million as 1_000_000 instead of 1000000), and is a SyntaxError in Python 3.5 and Python 2.
Use async and await in comprehensions, or in a generator (previously, yield and await could not both occur in the same function body). Once again these get you SyntaxError in 3.5 and in Python 2.

Python 3.7

As of the last edit to this article, Python 3.7 was the most recent released version of Python. It provided a new way to break things immediately:

Enable deferred evaluation of annotations with the from __future__ import annotations statement. This is an import-time SyntaxError in Python 3.6 and earlier (future imports are implemented directly in the interpreter, rather than as actual imports), and you can place the future import in a module even if it doesn’t use annotations.

Python 3.7 also made async and await into reserved keywords, but that one goes the other way round: rather than stopping new code from running on an old Python, it stops old code from running on a new Python (using async or await as identifiers is a SyntaxError in Python 3.7)

The new built-in breakpoint() function is a NameError in Python 3.6, but since it’s a debugging tool you probably don’t want to leave it in production code.

Three new modules were added to the standard library, providing opportunities to immediately cause ImportError in older Python versions: contextvars, dataclasses, and importlib.resources. Of these, dataclasses is the one most likely to be generally useful.

Finally, it’s possible to write code using the typing module which works in Python 3.7 but raises TypeError at import time in older versions. Previously, the generics in typing had the metaclass typing.GenericMeta, which would in turn cause TypeError (due to metaclass conflict) whenever a class was a subclass of a generic and any parent that had a different metaclass. PEP 560 resolved this problem for Python 3.7.

Python 3.8

As of the last edit to this article, Python 3.8 was not yet released, but already contained some changes which could be used to break older Pythons:

The := assignment expression operator.
A continue statement is now legal inside a finally clause.
Iterable unpacking in a yield or return statement no longer requires parentheses.
The built-in int type now has an as_integer_ratio() method. This is not generally useful for int on its own, but does harmonize the interfaces of int, float and Decimal and do away with the need to check the type or existence of the method before trying to call it on a numeric value of unknown type.

And that’s it… for now

I’ve already implemented some of the above in a piece of code I’m shipping (webcolors 1.7 uses u prefixes to break Python 3.0-3.2, and a dict comprehension to break 2.6), and plan to do the same with my other projects over the course of their future release cycles. And hopefully the above list will help someone else manage the same in their projects, and perhaps prevent some maintenance headaches and bad days for developers and users.