Building Python EGGs in Batch

While working with different versions of python you deal with libraries that comes with source code and doesn't provide a compiled python egg file.

Method1: Offline

Here is a script that develops eggs per python version you have installed for all packages found in lib directory (you must create that directory at the same place where the script below and drop downloaded source distribution archives in).

#!/bin/sh

rm -rf eggs
mkdir eggs
for p in /usr/local/bin/python?.?
do
    echo -n $p
    rm -rf env/
    virtualenv --no-site-packages --python=$p -q env > /dev/null 2>/dev/null
    echo -n .
    env/bin/easy_install -i lib -U -O2 -z distribute > /dev/null 2>/dev/null
    for f in lib/*  # packages to build
    do
        echo -n .
        env/bin/easy_install -i lib -O2 -z $f > /dev/null 2>/dev/null
    done
    cp env/lib/python*.*/site-packages/*.egg eggs/ 2>/dev/null
    echo done
done
rm -rf env/
rm -f eggs/setuptools* eggs/distribute*

Run script with:

./sh make_eggs.sh

Python eggs are now in eggs directory.

Method2: Online

In this case the script relies on internet connection and downloads sources right from pypi.

#!/bin/sh

rm -rf eggs
mkdir eggs
for p in /usr/local/bin/python?.?
do
    echo -n $p
    rm -rf env/
    virtualenv --no-site-packages --python=$p -q env > /dev/null 2>/dev/null
    echo -n .
    env/bin/easy_install -U -O2 -z distribute > /dev/null 2>/dev/null
    for f in $@
    do
        echo -n .
        env/bin/easy_install -O2 -z $f > /dev/null 2>/dev/null
    done
    cp env/lib/python*.*/site-packages/*.egg eggs/ 2>/dev/null
    echo done
done
rm -rf env/
rm -f eggs/setuptools* eggs/distribute*

Run script by passing packages you want to build:

./sh make_eggs.sh pycrypto lxml

Python eggs are now in eggs directory.

Using Buildbot with Multiple Projects

The BuildBot is a system to automate the compile/test cycle required to validate code changes. Here we are going setup buildbot master and slaves with the following requirements:

1. We have serveral projects: project1, project2, project3
2. These projects live under Mercurial (e.g. hosted at bitbucket.org) with base url https://scm.dev.local/hg/ followed by project name (e.g. https://scm.dev.local/hg/project1)
3. We should be able run at least 2 builds in parallel (this is where we can use buildbot slaves: linux-slave1 and linux-slave2)
4. The build should trigger automatically if the changes detected every 30 minutes during working day.
5. The builder should execute make all that ultimatelly does everything we need to verify integrity.
6. The builder should be checked agains certain python versions: 2.4, 2.5, 2.6, 2.7 and 3.2