Continuous integration for Python

September 02, 2015

Let's explore how to set up continuous integration for a Python project.

We'll use the following tools:

Starting from scratch

Let's build up a simple Python project, beginning with the tests.

tests/test_lists.py:

from nose.tools import assert_equals
from lists import fmap
from lists import flatMap

def test_map():
  assert_equals(fmap(lambda x: x * 2, [1,2,3]), [2,4,6])

def test_flatMap():
  assert_equals(flatMap(lambda x: [x, x + 1], [1,3,5]), [1,2,3,4,5,6])

While we're at it, we add an empty __init__.py to the tests/ directory.

Now let's write an implementation for lists that will make the tests pass.

lib/lists.py:

def fmap(f, xs):
  return list(map(f, xs))

def flatten(xss):
  return reduce(lambda a, b: a + b, xss)

def flatMap(f, xs):
  return flatten(fmap(f, xs))

We depend on nose2 and nose-cov for test and coverage support, respectively, so let's put them in requirements.txt.

requirements.txt:

nose2
nose-cov

Now we can install our dependencies using pip:

$ pip install -r requirements.txt

Run tests locally

We now have enough to run our tests locally and peruse a report of the code coverage. Let's run the tests:

$ pip install -r requirements.txt
$ nose2 --with-cov \
        --coverage-report term \
        --coverage-report html \
        --coverage lib
..
----------------------------------------------------------------------
Ran 2 tests in 0.022s

OK
---------- coverage: platform linux2, python 2.7.6-final-0 -----------
Name        Stmts   Miss  Cover
-------------------------------
lib/lists       6      3    50%
Coverage HTML written to dir htmlcov

The tests passed, with half of the code covered. Let's read the coverage report:

$ firefox htmlcov/index.html

Loading the coverage report in our browser, it looks something like this:

Run tests via Travis CI

Let's use Travis CI to automatically do our testing (and reporting back to GitHub) remotely.

.travis.yml:

language: python
install:
  - travis_retry pip install -r requirements.txt
  - travis_retry pip install coveralls
script:
  - coverage run --source=lib -m nose2.__main__ -v
after_success:
  - coveralls

Run tests via CircleCI

Instead of (or in addition to) Travis CI, we can use CircleCI to do the same work.

circle.yml:

dependencies:
  override:
    - pip install -r requirements.txt
    - pip install coveralls

test:
  override:
    - coverage run --source=lib -m nose2.__main__ -v
  post:
    - coveralls

For CircleCI, we make sure to set our Coveralls repo_token in project settings :: Environment variables:

Review coverage

Finally we're ready for some sweet continuous integration goodness.

Make some changes, and submit a pull request. Travis/Circle kicks off a build, and sends coverage data to Coveralls.

Both Travis/Circle and Coveralls leave feedback on the pull request: