Sunday, October 13, 2013

Python HTTP Client API

The modern web services expose public API to the world and JSON is de-facto standard in this communication. Here is a simple use case that integrates with buildbot public API.

Lets setup a virtual environment and install wheezy.core package:
virtualenv env
env/bin/easy_install wheezy.core
Launch python from virtual environment (env/bin/python) and try this:
>>> from wheezy.core.httpclient import HTTPClient
>>> c = HTTPClient('http://buildbot.buildbot.net/json/')
>>> c.get('project')
200
>>> project = c.json
>>>> str(project.title)
Buildbot
Here is another example that demonstarates etag handling (the second time we request events the server responds with HTTP status code 304, not modified):
>>> base = 'https://api.github.com/repos/python/cpython/'
>>> c = HTTPClient(base)
>>> c.get('events')
200
>>> c.headers['content-encoding']
['gzip']
>>> c.get('events')
304
The HTTPClient supports HTTP(S) GET/HEAD/POST verbs, follows redirects, handles cookies and etags between requests, gzip content decoding.

10 comments:

  1. Your project seems very interesting, looking to use it in future.

    Suggestion: describe better what it does, purposes, etc.

    Does you agree that "wheezy.core is a python package written in pure Python code. It provides core features. It is optimized for performance, well tested and documented." is very uninformative?

    Thanks


    ReplyDelete
  2. Is this better than https://pypi.python.org/pypi/requests

    ReplyDelete
    Replies
    1. HTTP in the real world is NOT simple. urllib2 failed not because it wasn't simple to use (btw, I don't agree with people who say it has bad API) but because it lacks many features needed in day-to-day work which requests/urllib3 provide out of the box. “Everything should be made as simple as possible, but no simpler.” - Albert Einstein

      Delete
    2. Just wanted to enumerate the features you are referencing to: gzip decoding, file posting, ssl verification, connection reuse... am I missing something? - Kšyštof

      Delete
    3. So what was so complicated about Requests and urllib3 that you had to write your own http client library?

      Delete
    4. 16102 lines of python code in requests vs 3835 in urllib3 vs 255.

      Delete
  3. How about timeout?

    ReplyDelete
    Replies
    1. Supported by python 2.6+ (in seconds):
      c = HTTPClient(...)
      c.connection.timeout = 10

      Delete