PostgreSQL Streaming Replication Hot Standby

Streaming replication allows a standby server to stay up-to-date with primary. The standby connects to the primary, which streams WAL records to the standby as they're generated.

Hot standby is the term used to describe the ability to connect to the server and run read-only queries.

This method of replication is completely transparent to the client, it doesn't require any changes to database, allows query information from standby server and requires minimum administrative effort.

Python HTTP Client API

The modern web services expose public API to the world and JSON is de-facto standard in this communication. Here is a simple use case that integrates with buildbot public API.

Lets setup a virtual environment and install wheezy.core package:

virtualenv env
env/bin/easy_install wheezy.core

Launch python from virtual environment (env/bin/python) and try this:

>>> from wheezy.core.httpclient import HTTPClient
>>> c = HTTPClient('http://buildbot.buildbot.net/json/')
>>> c.get('project')
200
>>> project = c.json
>>>> str(project.title)
Buildbot

Here is another example that demonstarates etag handling (the second time we request events the server responds with HTTP status code 304, not modified):

>>> base = 'https://api.github.com/repos/python/cpython/'
>>> c = HTTPClient(base)
>>> c.get('events')
200
>>> c.headers['content-encoding']
['gzip']
>>> c.get('events')
304

The HTTPClient supports HTTP(S) GET/HEAD/POST verbs, follows redirects, handles cookies and etags between requests, gzip content decoding.

Keep FreeBSD up to date with subversion

The freebsd ports tree is quite big and sometimes you need just few packages. You can do that with subversion sparse checkouts.

pkg_add -r subversion

Checkout only immediate top level. The list of svn mirrors is here.

svn checkout --depth=immediates \
    http://svn0.eu.freebsd.org/ports/head /usr/ports

Update tree on as needed basis:

cd /usr/ports
# common
svn up --set-depth=infinity Mk Templates Tools/scripts
# portmaster
svn up ports-mgmt/portmaster ports-mgmt/dialog4ports 
# python
svn up lang/python lang/python2 lang/python27 \
    devel/gettext converters/libiconv
# mercurial
svn up devel/mercurial
# python development
svn up devel/py-distribute py-setuptools \
    devel/py-virtualenv
# vim
svn up editors/vim
# subversion
svn up devel/subversion devel/libtool devel/apr1 \
    databases/db42 databases/gdbm databases/sqlite3 \
    textproc/expat2 www/serf

Consider configure /etc/make.conf as advised here and install packages:

make install clean -sC /usr/ports/ports-mgmt/portmaster
portmaster lang/python27 devel/mercurial \
    devel/py-virtualenv editors/vim \
    devel/subversion

The above ports require only 12Mb in /usr/ports and next time ports update takes few seconds.

Convenient Remote Access with SSH Config

If you are working with a lot of remote ssh hosts it becomes hard to remember all that host specific information: username, ip address, identity file, non-standard port or local/remote port forwarding. ssh_config to rescue. Here is a sample to give you an idea (file ~/.ssh/config):

Compression yes
IdentityFile ~/.ssh/id_rsa
LogLevel ERROR
Port 22

Host h1
    HostName 192.168.91.57
    User master
    IdentityFile ~/.ssh/h1.pem

Host db1
    HostName usca45d1.example.com
    User pg
    LocalForward 5432 127.0.0.1:5432

The above configuration let me access those hosts simply by name, e.g.:

ssh h1
scp schema.sql db1:~/

How to manage Git or Mercurial repositories

Managing version control repositories can be a challenge in multi-user environment especially when simplification of user collaboration is your goal. There are usually two primary concerns while considering enterprise deployment for version control repositories: access control and safety of your data. Both are not directly addressed by version control itself, thus a sort of security facade is necessary.