PrerequisitesWhat is important to understand about applicability of async pattern for IO bound operations, they must:
- Utilize the same event loop, in this case worker's event loop can switch to next in the queue (otherwise it is blocked).
- Have long latency (otherwise you are wasting event loop queue for short requests).
SolutionIf we look at a web application there are the following major IO bound operations:
- Serve file from disk
- Call persistence (e.g. SQL)
- Fetch some data from a web resource
- Idle browser connections
1. Serve File from DiskA trivial case would serve files from some directory and web servers like nginx can do that efficiently.
There are cases when a decision to serve a file might depend on application code (e.g. authorize access). At first that seems to be a good candidate... in reality a web server supports a technic known as XSendfile so you only have to set a path to your file in a response header.
So in both of these cases the job is delegated to web server.
2. Call PersistenceA request served by persistence (e.g. cache, SQL database, etc) has low latency. There is no advantage to use async calls (vs classical sync threaded / multi-process approach) and surprisingly you might experience decrease in performance. The async pattern can be interesting if you have a lot of idle connections, but that is no-sense waste of server resources... which leads you to utilize pooling, e.g. pgpool. Serving thousands of web connections needs just few persistent connections (especially if you use caching thus offload request from database).
If persistence is not able to respond quickly (to achieve low latency), due to high workload, very likely you end up with queuing requests that never get processed due to timeout or in worst case causing thundering herd problem.
3. Fetch Some Data from a Web ResourceFetching data over internet has long latency. You can consider design your application in a way to use:
- A message queue for processing so you delegate work to background process.
- Dedicated application server instance with gevent event loop.
- Commet in combination with message queue.