Monday, October 15, 2012

Python Web Reverse URLs Benchmark

How fast python web frameworks reverse urls? While routing is a mapping of incoming request to a handler, url reverse function is designed to build urls for those handlers. A web page may have a number of urls from few dozen to hundreds... all related to your web site (e.g. links between related pages, tag cloud, most viewed posts, etc). A typical web application usually has deal with the following reverse url use cases:
  • Static: the URL path is fixed and never changes, e.g. https://bitbucket.org/explore.
  • Merge: the URL path is constructed dynamically, some information is taken from URL. A `user page` shows repositories. In this case a list of user repositories will be constructed as merge of information that came from URL (user name) and repository name.
  • Route: the URL path is constructed dynamically, all information can be taken from URL. A `repository page` displays a number of features: downloads, source, etc. Those links include current route information (user name and repository name).
We will examine all reverse url use cases mentioned above with... a trivial 'Hello World!' application that builds 20 urls for each case. The benchmark is executed in isolated environment using CPython 2.7. Latest available versions (November 17, 2013):
  1. django 1.6
  2. flask 0.10.1
  3. pylons 1.0.1
  4. pyramid 1.5a2
  5. tornado 3.1.1
  6. wheezy.web 0.1.373
Let setup few prerequisites to be able run this in a clean debian testing installation.
apt-get install make python-dev python-virtualenv \
    mercurial unzip
The source code is hosted on bitbucket, let clone it into some directory and setup virtual environment (this will download all necessary package dependencies per framework listed above).
hg clone https://bitbucket.org/akorn/helloworld
cd helloworld/03-urls && make env
Once environment is ready we can run benchmarks:
env/bin/python benchmarks.py
Here are raw numbers:
static        msec    rps  tcalls  funcs
django       21353    468    1786    120
flask        30576    327    2713    149
pylons       11370    880     794     91
pyramid       2849   3510     345     57
tornado       2604   3840     287     71
wheezy.web     713  14032      89     30

merge         msec    rps  tcalls  funcs
django       34974    286    3113    119
flask        46414    215    3440    153
pylons       12028    831     797     91
pyramid       5494   1820     410     60
tornado       5645   1771     709     84
wheezy.web    1323   7560      94     35

route         msec    rps  tcalls  funcs
django       34463    290    3039    119
flask        52045    192    3839    153
pylons       12140    824     803     91
pyramid       5988   1670     435     60
tornado       6069   1648     768     84
wheezy.web    1300   7691     136     37
msec - a total time taken in milliseconds, rps - requests processed per second, tcalls - total number of call made by corresponding web framework, funcs - a number of unique functions used.
Note, you can run this benchmark using any version of python, including pypy. Here are make targets to use:
make env VERSION=3.3
make pypy
Environment Specification:
  • Intel Core 2 Duo @ 2.4 GHz x 2
  • OS X 10.9, Python 2.7.6
Python has a number of web frameworks. A trivial reverse url use case gives you an idea where particular web framework stands in terms of performance and internal effectivity. Looking for other benchmarks? Take a look at python benchmarks for web frameworks, routing and templates.

4 comments :

  1. I don't think this benchmark as it is implemented now is very meaningful. Assuming reverse mapping 20 URLs is rather cheap, what you're benchmarking here is just the general framework speed and not neccessarily the speed of the reverse URL mapping.

    Now it's probably not really feasible to get the URL mapping working in isolation, instead you could measure 2 times, a) the speed of the "naked" framework (i.e. with a handler that does nothing) and b) with a handler that generates 1000 reverse URLs (or more, depending of how significant the time difference is), and then subtract a from b.

    ReplyDelete
    Replies
    1. What is an average number of URLs per page? 1000 is somewhat unrealistic.

      Delete
    2. Not at all ! Imagine a page rendering a table with 10 columns and 100 rows ... each cell having a link. Plausible right ?

      Delete
    3. This seems to be easy: if the django performance for 20 static reverse urls is almost 500 rps and you want to know the performance for 1000, that will be 10 rps.

      I think the 20 urls is just sort of reasonable number to compare oranges and apples. Don't you think?

      Delete