Monday, October 15, 2012

Python Web Reverse URLs Benchmark

How fast python web frameworks reverse urls? While routing is a mapping of incoming request to a handler, url reverse function is designed to build urls for those handlers. A web page may have a number of urls from few dozen to hundreds... all related to your web site (e.g. links between related pages, tag cloud, most viewed posts, etc). A typical web application usually has deal with the following reverse url use cases:
  • Static: the URL path is fixed and never changes, e.g. https://bitbucket.org/explore.
  • Merge: the URL path is constructed dynamically, some information is taken from URL. A `user page` shows repositories. In this case a list of user repositories will be constructed as merge of information that came from URL (user name) and repository name.
  • Route: the URL path is constructed dynamically, all information can be taken from URL. A `repository page` displays a number of features: downloads, source, etc. Those links include current route information (user name and repository name).
We will examine all reverse url use cases mentioned above with... a trivial 'Hello World!' application that builds 20 urls for each case. The benchmark is executed in isolated environment using CPython 2.7. Latest available versions as of this writing:
  1. django 1.4.1
  2. flask 0.9
  3. pyramid 1.4a2
  4. tornado 2.4
  5. wheezy.web 0.1.307
Let setup few prerequisites to be able run this in a clean debian testing installation.
apt-get install make python-dev python-virtualenv \
    mercurial unzip
The source code is hosted on bitbucket, let clone it into some directory and setup virtual environment (this will download all necessary package dependencies per framework listed above).
hg clone https://bitbucket.org/akorn/helloworld
cd helloworld/03-urls && make env
Once environment is ready we can run benchmarks:
env/bin/python benchmarks.py
Here are raw numbers:
static        msec    rps  tcalls  funcs
django        9339   1071    1100     93
flask        19499    513    3033    138
pyramid       4224   2367     350     59
tornado       2941   3400     265     64
wheezy.web    1042   9598      89     30

merge         msec    rps  tcalls  funcs
django       24417    410    1292     92
flask        59415    168    4360    143
pyramid       7046   1419     415     62
tornado       6520   1534     727     78
wheezy.web    1766   5661      94     35

route         msec    rps  tcalls  funcs
django       23277    430    1203     91
flask        62790    159    4639    143
pyramid       7600   1316     440     62
tornado       7211   1387     786     78
wheezy.web    1756   5694     136     37
msec - a total time taken in milliseconds, rps - requests processed per second, tcalls - total number of call made by corresponding web framework, funcs - a number of unique functions used.
Note, you can run this benchmark using any version of python, including pypy. Here are make targets to use:
make env VERSION=3.3
make pypy
Environment Specification:
  • Intel Xeon CPU X3430 @ 2.40GHz x 4, Kernel 3.2.0-3-amd64
  • Debian Testing, Python 2.7.3
Python has a number of web frameworks. A trivial reverse url use case gives you an idea where particular web framework stands in terms of performance and internal effectivity. Looking for other benchmarks? Take a look at python benchmarks for web frameworks, routing and templates.

2 comments:

  1. I don't think this benchmark as it is implemented now is very meaningful. Assuming reverse mapping 20 URLs is rather cheap, what you're benchmarking here is just the general framework speed and not neccessarily the speed of the reverse URL mapping.

    Now it's probably not really feasible to get the URL mapping working in isolation, instead you could measure 2 times, a) the speed of the "naked" framework (i.e. with a handler that does nothing) and b) with a handler that generates 1000 reverse URLs (or more, depending of how significant the time difference is), and then subtract a from b.

    ReplyDelete
    Replies
    1. What is an average number of URLs per page? 1000 is somewhat unrealistic.

      Delete