Asynchronous Python in Web Applications

Asynchronous or non-blocking IO allows worker to process other requests before current request has finished. That is possible in case a processing of current web request is related to an operation that is IO bound.

Prerequisites

What is important to understand about applicability of async pattern for IO bound operations, they must:

1. Utilize the same event loop, in this case worker's event loop can switch to next in the queue (otherwise it is blocked).
2. Have long latency (otherwise you are wasting event loop queue for short requests).

If operation is not IO bound it is considered blocking.

Solution

If we look at a web application there are the following major IO bound operations:

1. Serve file from disk
2. Call persistence (e.g. SQL)
3. Fetch some data from a web resource
4. Comet
5. Idle browser connections

Let's go through each of these cases:

1. Serve File from Disk

A trivial case would serve files from some directory and web servers like nginx can do that efficiently.

There are cases when a decision to serve a file might depend on application code (e.g. authorize access). At first that seems to be a good candidate... in reality a web server supports a technic known as XSendfile so you only have to set a path to your file in a response header.

So in both of these cases the job is delegated to web server.

2. Call Persistence

A request served by persistence (e.g. cache, SQL database, etc) has low latency. There is no advantage to use async calls (vs classical sync threaded / multi-process approach) and surprisingly you might experience decrease in performance. The async pattern can be interesting if you have a lot of idle connections, but that is no-sense waste of server resources... which leads you to utilize pooling, e.g. pgpool. Serving thousands of web connections needs just few persistent connections (especially if you use caching thus offload request from database).

If persistence is not able to respond quickly (to achieve low latency), due to high workload, very likely you end up with queuing requests that never get processed due to timeout or in worst case causing thundering herd problem.

3. Fetch Some Data from a Web Resource

Fetching data over internet has long latency. You can consider design your application in a way to use:

1. A message queue for processing so you delegate work to background process.
2. Dedicated application server instance with gevent event loop.
3. Commet in combination with message queue.

4. Comet

A long-held HTTP request which allows a web server to push data to a browser, without the browser explicitly requesting it. Chat application is a good example. A technic here is to utilize X-Accel-Redirect response header to perform nginx internal redirect with further processing by push-stream-module.

5. Idle Browser Connections

It is very common to have a lot of idle browser connections. nginx is very efficient to keep them open while proxying work to application server (e.g. uwsgi).

Summary

Recommendations above are not specific to any web framework, instead an attempt is made to suggest the most efficient approaches to resolve typical tasks that benefit from non-blocking IO.