Python duplicate code detection with clonedigger

The python tool clonedigger can be used to examine your source code for duplication. It can be installed with easy_install clonedigger. Here is how to run it:

user1@deby:~/devenv/trunk$ ../bin/clonedigger src/
Parsing  src/greatings/helloworld.py ... done
Parsing  src/greatings/__init__.py ... done
Parsing  src/greatings/tests/__init__.py ... done
Parsing  src/greatings/tests/test_helloworld.py ... done
3 sequences
average sequence length: 2.666667
maximum sequence length: 3
Number of statements:  8
Calculating size for each statement... done
Building statement hash... done
Number of different hash values:  5
Building patterns... 6 patterns were discovered
Choosing pattern for each statement... done
Finding similar sequences of statements... 0  sequences were found
Refining candidates... 0 clones were found
Removing dominated clones... 0 clones were removed

Read more about clonedigger here.

Python code metrics with pymetrics

The python tool pymetrics can be used to measure your source code complexity. You can install it with easy_install pymetrics. Here is an example (running in virtual environment devenv; source code is located in src):

user1@deby:~/devenv/trunk$ ../bin/pymetrics src/greatings\
/helloworld.py

=== File: src/greatings/helloworld.py ===
Module src/greatings/helloworld.py is missing a module doc string.
Detected at line 1

Basic Metrics for module src/greatings/helloworld.py
----------------------------------------------------
          1    maxBlockDepth
          4    numBlocks
        189    numCharacters
          1    numDocStrings
          1    numFcnDocStrings
          2    numFunctions
          7    numKeywords
         16    numLines
         73    numTokens

         50.00 %FunctionsHavingDocStrings

Functions DocString present(+) or missing(-)
--------------------------------------------
- main
+ say

McCabe Complexity Metric for file src/greatings/helloworld.py
--------------------------------------------------------------
          2    __main__
          1    main
          1    say

COCOMO 2's SLOC Metric for src/greatings/helloworld.py
-------------------------------------------------------
          8    src/greatings/helloworld.py

*** Processed 1 module in run ***

Here is a script that let you generate report for all your files in src directory:

#!/bin/sh

working_dir=/tmp/$USER/pymetrics
mkdir -p $working_dir

find src/ -name \*.py > $working_dir/files.txt
../bin/pymetrics --nosql --nocsv -f $working_dir/files.txt

Here is a guideline for understanding cyclomatic complexity number:

• 1 - 15: simple code, minimum risk, can be easily covered by tests
• 15 - 30: complicated code, consider refactoring before writing tests
• 30 - 50+: complex code, refactor now, almost impossible to write a good tests

Read more about software complexity here and here.

Python static code analysis with pyflakes

Python tool pyflakes is focused on identifying common errors quickly without executing Python code. It can be installed with easy_install pyflakes. Since the tool doesn't import your code, you just specify a folder with source code:

user1@deby:~/devenv/trunk$ ../bin/pyflakes src/

Using pyflakes with IDE like PyDev make good sense since you see errors while you are typing.

Python static code analysis with pylint

Pylint is a python tool that checks if a module satisfies a coding standard. You can install pylint with easy_install (pylint has dependencies on logilab_common and logilab_astng, so consider download them as well in case of offline install).

easy_install pylint

Running checks is pretty easy:

user1@deby:~/devenv/trunk/src$ ../../bin/pylint greatings
No config file found, using default configuration
************* Module greatings
C:  1: Missing docstring
************* Module greatings.helloworld
C:  1: Missing docstring
C: 10:main: Missing docstring
************* Module greatings.tests
C:  1: Missing docstring
W:  4: Relative import 'test_helloworld', should be 
'greatings.tests.test_helloworld'
W:  7:suite: Redefining name 'suite' from outer scope (line 6)
C:  6:suite: Missing docstring
************* Module greatings.tests.test_helloworld
C:  1: Missing docstring
C:  4:HelloworldTestCase: Missing docstring
C:  6:HelloworldTestCase.test_say: Missing docstring
R:  6:HelloworldTestCase.test_say: Method could be a function
W: 12:suite: Redefining name 'suite' from outer scope (line 10)
C: 10:suite: Missing docstring
  ...
Global evaluation
-----------------
Your code has been rated at 5.94/10

The tool also produces a report that include:

• Statistics by type
• External dependencies
• Duplication
• Raw metrics
• Messages by category
• % errors / warnings by module
• Messages

If you want to change the default behaviour, you can define options in pylintrc file (use --rcfile option to specify a location). Here is an example.

user1@deby:~/devenv/trunk$ wget -P tools/ http://www.\
logilab.org/cgi-bin/hgwebdir.cgi/pylint/raw-file\
/df8f34aa3dd2/examples/pylintrc
user1@deby:~/devenv/trunk$ cd src/
user1@deby:~/devenv/trunk/src$ ../../bin/pylint \
--rcfile=../tools/pylintrc greatings

Read more about pylint here.

Python static code analysis with pychecker

Python tool PyChecker is used to find typical programming errors in your source code. In order to install it you need download it and install manually (if you try install it using easy_install it will not work, unfortunately). We will assume the following directory structure (note, devenv is virtual environment, you can create it issuing virtualenv devenv).

~/devenv/
`-- trunk/
    |-- src/
    |   `-- greatings/
    |       |-- __init__.py
    |       |-- helloworld.py
    |       `-- tests/
    |           |-- __init__.py
    |           `-- test_helloworld.py
    `-- tools/

Let download pychecker into tools directory and proceed with installation:

user1@deby:~/devenv/trunk$ wget -P tools/ http://downloads.\
sourceforge.net/project/pychecker/pychecker/0.8.18/\
pychecker-0.8.18.tar.gz
...
user1@deby:~/devenv/trunk$  cd tools && tar zxf \
pychecker-0.8.18.tar.gz \
 && cd  pychecker-0.8.18
user1@deby:~/devenv/trunk/tools/pychecker-0.8.18$ \
../../../bin/python setup.py install && cd ../.. && \
rm -rf tools/pychecker-0.8.18
...

The next step you need to "fix" a bit a pychecker file installed into your virtual environment bin directory so the content look like this.

#!/bin/sh

site_packages_dir=$(dirname $0)/../lib/python2.6/site-packages
python $site_packages_dir/pychecker/checker.py "$@"

Since pychecker import all the files it is going to check, you need to start the tool in correct directory, in our case src:

user1@deby:~/devenv/trunk$ cd src/
user1@deby:~/devenv/trunk/src$ ../../bin/pychecker greatings/*.py
Processing module helloworld (greatings/helloworld.py)...
Processing module __init__ (greatings/__init__.py)...

Warnings...

None

Please note that pychecker doesn't drill into sub-packages, so in order to analyze tests issue the following command:

../../bin/pychecker greatings/tests/*.py

If you want to change the default behaviour, you can define options in pycheckrc file (use -F option to specify a location). Here is an example.

user1@deby:~/devenv/trunk$ wget -P tools/ http://pychecker.\
cvs.sourceforge.net/viewvc/pychecker/pychecker/pycheckrc
user1@deby:~/devenv/trunk$ cd src/
user1@deby:~/devenv/trunk/src$ ../../bin/pychecker -F \
../tools/pycheckrc greatings/*.py

Read more about pychecker here and here.

Python mock testing with pymock module

Python module pymock is based on EasyMock. Install pymock module:

easy_install pymock

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

Pymock uses a recording and replay model. Here is our test (file pymockexample.py):

from datetime import datetime
from atm import Atm
import unittest
from pymock import Controller, Any

class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mocker = Controller()
        self._mock_account = self._mocker.mock()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()
        self._mocker.verify()

    def test_withdraw(self):
        # Arrange
        self._mock_account.deposit(150)
        self._mock_account.withdraw(100)
        self._mock_account.comission(0.5)

        # WARNING: This doesn't work
        # self._mock_account.balance(Any())
        self._mock_account.balance(datetime.now())
        self._mocker.returns(49.5)
        self._mocker.replay()

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5

    def test_withdraw_insufficient_funds(self):
        # Arrange
        self._mock_account.deposit(50)
        self._mock_account.withdraw(100)
        self._mocker.raises(ValueError('Insufficient funds'))
        self._mock_account.comission(0.1)
        self._mock_account.balance(datetime.now())
        self._mocker.returns(49.9)
        self._mocker.replay()

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9

if __name__ == '__main__':
    unittest.main()

Run tests:

python pymockexample.py

Read more about pymock here.

Python mock testing with mox module

Python module mox is based on EasyMock. Install mox module:

easy_install mox

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

When you create a mock object, it is in record mode. You record the behavior by calling the expected methods. Once you are done, switch to replay mode. Here is our test (file moxexample.py):

from datetime import datetime
from atm import Atm
import unittest
from mox import Mox, IgnoreArg, Func

class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mox = Mox()
        self._mock_account = self._mox.CreateMockAnything()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()
        self._mox.VerifyAll()

    def test_withdraw(self):
        # Arrange
        self._mock_account.deposit(150)
        self._mock_account.withdraw(100)
        self._mock_account.comission(0.5)
        self._mock_account.balance(\
                Func(lambda d: d <= datetime.now()))\
                .AndReturn(49.5)
        self._mox.ReplayAll()

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5

    def test_withdraw_insufficient_funds(self):
        # Arrange
        self._mock_account.deposit(50)
        self._mock_account.withdraw(100)\
                .AndRaise(ValueError('Insufficient funds'))
        self._mock_account.comission(0.1)
        self._mock_account.balance(IgnoreArg())\
                .AndReturn(49.9)
        self._mox.ReplayAll()

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9

if __name__ == '__main__':
    unittest.main()

Run tests:

python moxexample.py

Read more about mox here.

Python mock testing with mock module

Install mock module:

easy_install mock

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

After performing an action on Mock instance, you can make assertions about which methods / attributes were used and arguments they were called with. Here is our test (file mockexample.py):

from datetime import datetime
from atm import Atm
import unittest
from mock import Mock

class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mock_account = Mock()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()

    def test_withdraw(self):
        # Arrange
        def balance(d):
            self.assertTrue(d >= datetime.now()) 
            return 49.5
        self._mock_account.balance.side_effect = balance

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5
        self._mock_account.deposit.assert_called_with(150)
        self._mock_account.withdraw.assert_called_with(100)
        self._mock_account.comission.assert_called_with(0.5)

    def test_withdraw_insufficient_funds(self):
        # Arrange
        self._mock_account.withdraw\
                .side_effect = ValueError('Insufficient funds')
        self._mock_account.balance.return_value = 49.9

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9
        self._mock_account.deposit.assert_called_with(50)
        self._mock_account.withdraw.assert_called_with(100)
        self._mock_account.comission.assert_called_with(0.1)

if __name__ == '__main__':
    unittest.main()

Run tests:

python mockexample.py

Read more about mock here.

Python mock testing with mocker module

First of all install mocker:

easy_install mocker

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

A Mocker instance is used for expectations record/replay. Here is our test (file mockerexample.py):

import unittest
from datetime import datetime

from mocker import Mocker, ANY, expect
from atm import Atm


class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mocker = Mocker()
        self._mock_account = self._mocker.mock()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()
        self._mocker.restore()
        self._mocker.verify()

    def test_withdraw(self):
        # Arrange
        self._mock_account.deposit(150)
        self._mock_account.withdraw(100)
        self._mock_account.comission(0.5)
        expect(self._mock_account.balance(ANY))\
                .result(49.5)\
                .call(lambda d: self.assertTrue(d <= datetime.now()))
        self._mocker.replay()

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5

    def test_withdraw_insufficient_funds(self):
        # Arrange
        self._mock_account.deposit(50)
        expect(self._mock_account.withdraw(100))\
                .throw(ValueError('Insufficient funds'))
        self._mock_account.comission(0.1)
        expect(self._mock_account.balance(ANY))\
                .result(49.9)
        self._mocker.replay()

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9

if __name__ == '__main__':
    unittest.main()

Run tests:

python mockerexample.py

Read more about mocker here.

Python mock testing with mockito module

This module is a port of the Mockito mocking framework to Python. Install mockito with easy_install:

easy_install mockito

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

Here is our test (file mockitoexample.py):

from datetime import datetime
from atm import Atm
import unittest
from mockito import Mock, when, verify, any, verifyNoMoreInteractions

class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mock_account = Mock()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()
        verifyNoMoreInteractions(self._mock_account)

    def test_withdraw(self):
        # Arrange
        when(self._mock_account).balance(any(datetime))\
                .thenReturn(49.5)

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5
        verify(self._mock_account).deposit(150)
        verify(self._mock_account).withdraw(100)
        verify(self._mock_account).comission(0.5)
        verify(self._mock_account).balance(any(datetime))

    def test_withdraw_insufficient_funds(self):
        # Arrange
        when(self._mock_account).withdraw(100)\
                .thenRaise(ValueError('Insufficient funds'))
        when(self._mock_account).balance(any(datetime))\
                .thenReturn(49.9)

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9
        verify(self._mock_account).deposit(50)
        verify(self._mock_account).withdraw(100)
        verify(self._mock_account).comission(0.1)
        verify(self._mock_account).balance(any(datetime))

if __name__ == '__main__':
    unittest.main()

Run tests:

python mockitoexample.py

Read more about mockito here.

Python mock testing with ludibrio module

In a unit test, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. You can install ludibrio mock framework module issuing the following command:

easy_install ludibrio

This framework comes with:

• Mock - recorded expectations, strict behaviour
• Dummy - never validated, ignore behaviour
• Stub - canned answers, loose behaviour, give me as much as you can or Dummy
• Spy - proxy, forwarder behaviour, give what you setup, everything else from wrapped type

Suppose you need to test withdraw operation in ATM (file atm.py):

from datetime import datetime

class Atm:
    def signin(self, account):
        self._account = account

    def signout(self):
        self._account = None

    def withdraw(self, amount):
        try:
            self._account.withdraw(amount)
            self._account.comission(amount * 0.005)
        except ValueError:
            self._account.comission(amount * 0.001)
        return self._account.balance(datetime.now())

Here is our test (file ludibrioexample.py):

from atm import Atm
import unittest
from ludibrio import Mock, Dummy

class TestAtm(unittest.TestCase):

    def setUp(self):
        self._mock_account = Mock()
        self._atm = Atm()
        self._atm.signin(self._mock_account)

    def tearDown(self):
        self._atm.signout()
        self._mock_account.validate()

    def test_withdraw(self):
        # Arrange
        with self._mock_account as a:
            a.deposit(150) >> None
            a.withdraw(100) >> None
            a.comission(0.5) >> None
            a.balance(Dummy()) >> 49.5

        # Act
        self._mock_account.deposit(150)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.5

    def test_withdraw_insufficient_funds(self):
        # Arrange
        with self._mock_account as a:
            a.deposit(50) >> None
            a.withdraw(100) >> ValueError('Insufficient funds')
            a.comission(0.1) >> None
            a.balance(Dummy()) >> 49.9

        # Act
        self._mock_account.deposit(50)
        remaining_balance = self._atm.withdraw(100)

        # Assert
        assert remaining_balance == 49.9

if __name__ == '__main__':
    unittest.main()

Run tests:

python ludibrioexample.py

Read more about ludibrio here.

Python unit testing with nose module

You can install nose issuing the following command:

easy_install nose

nose collects tests automatically, as long as you follow some simple guidelines for organizing test code.

from nose import SkipTest
from nose.tools import raises

class Counter:
    def __init__(self, value = 0):
        self.value = value

    def add(self, x):
        if not x:
            raise ValueError
        self.value += x
        return self.value

class TestCounter:
    def setup(self):
        """Automatically called by nose before and
           for each test method invoked
        """
        self._counter = Counter()

    def teardown(self):
        """Automatically called by nose after and
           for each test method invoked
        """
        self._counter = None

    def test_initial_value(self):
        assert self._counter.value == 0

    def test_add(self):
        assert 5 == self._counter.add(5)
        assert 5 == self._counter.value

    @raises(ValueError)
    def test_add_zero_raises_error(self):
        self._counter.add(0)

    def test_skip_me(self):
        raise SkipTest("Not ready yet")
        assert False

if __name__ == '__main__':
    import nose
    nose.runmodule()

In order to execute all tests just run the following:

test1@deby:~$ python noseexample.py
...S
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK (SKIP=1)

You can run doctests with nose. The easiest way to do so is to add the following to your setup.cfg file:

[nosetests]
verbosity=1
with-doctest=1

And use command-line:

nosetest noseexample.py

Read more about nose testing tools here.

Python unit testing with py.test module

You can install py.test issuing the following command:

easy_install py

py.test offers automatic test discovery:

from py.test import raises
from py.test import skip

class Counter:
    def __init__(self, value = 0):
        self.value = value

    def add(self, x):
        if not x:
            raise ValueError
        self.value += x
        return self.value

class TestCounter:
    def setup_method(self, state):
        """Automatically called by py.test before and
           for each test method invoked
        """
        self._counter = Counter()

    def teardown_method(self, state):
        """Automatically called by py.test after and
           for each test method invoked
        """
        self._counter = None

    def test_initial_value(self):
        assert self._counter.value == 0

    def test_add(self):
        assert 5 == self._counter.add(5)
        assert 5 == self._counter.value

    def test_add_zero_raises_error(self):
        raises(ValueError, self._counter.add, 0)

    def test_skip_me(self):
        skip('Not ready yet')
        assert False

if __name__ == '__main__':
    pass

In order to execute all tests just run the following:

test1@deby:~$ py.test pytestexample.py
============================= test session starts ==================
test object 1: pytestexample.py

pytestexample.py ...s

===================== 3 passed, 1 skipped in 0.08 seconds ==========

Read more about py.test features here.

Python unit testing with doctest module

The doctest module is part of the Python standard library. There is no need to write separate test functions/methods in doctest, you simply copy the expected results and paste them in the docstring that corresponds to the tested function.

class Counter:
    def __init__(self, value = 0):
        """
        >>> Counter().value
        0
        >>> Counter(100).value
        100
        """
        self.value = value
    
    def add(self, x):
        """
        >>> c = Counter()
        >>> c.add(5)
        5
        >>> c.value
        5
        >>> c.add(0)
        Traceback (most recent call last):
        ...
        ValueError
        >>> c.skip_me # doctest: +SKIP
        """
        if not x:
            raise ValueError
        self.value += x
        return self.value

if __name__ == '__main__':
    import doctest
    doctest.testmod()

In order to execute all tests just run the following (on success no output is given):

python doctestexample.py

Alternatively you can run doctest with unittest:

if __name__ == '__main__':
    import doctest, unittest
    suite = unittest.TestSuite()
    suite.addTest(doctest.DocTestSuite(__name__))
    runner = unittest.TextTestRunner()
    runner.run(suite)

Read more about doctest here.

Python unit testing with unittest module

The unittest module (also known as pyunit) is a unit test framework included in Python standard library. Let see how you can test the following:

class Counter:
    def __init__(self, value = 0):
        self.value = value
    
    def add(self, x):
        if not x:
            raise ValueError
        self.value += x
        return self.value

All test methods should start with word test by convention. Here is a unit test (file unittestexample.py):

import unittest

class CounterTestCase(unittest.TestCase):
    def setUp(self):
        """Automatically called by TestCase before and
           for each test method invoked
        """
        self._counter = Counter()

    def tearDown(self):
        """Automatically called by TestCase after and
           for each test method invoked
        """
        self._counter = None

    def test_initial_value(self):
        self.assertFalse(self._counter.value)
        
    def test_add(self):        
        self.assertEqual(5, self._counter.add(5))
        self.assertEqual(5, self._counter.value)

    def test_add_zero_raises_error(self):
        self.assertRaises(ValueError, lambda: self._counter.add(0))

    def skip_test_skip_me(self):
        assert False

if __name__ == '__main__':
    unittest.main()

In order to execute all tests just run the following (each dot corresponds to a test run):

test1@deby:~$ python unittestexample.py
...
---------------------------------------
Ran 3 tests in 0.000s

Verbose output:

python unittestexample.py -v

A selected test run:

python unittestexample.py CounterTestCase.test_add

Alternatively you can combine several test cases into test suites.

 ...

def suite():
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    suite.addTest(loader.loadTestsFromTestCase(CounterTestCase))
    return suite

if __name__ == '__main__':
    unittest.TextTestRunner(verbosity=2).run(suite())

Here is file __init__.py

import unittest

import unittestexample

def suite():
    suite = unittest.TestSuite()
    suite.addTests(unittestexample.suite())
    return suite

if __name__ == '__main__':
    unittest.TextTestRunner(verbosity=2).run(suite())

You can read more about unittest module here and here.