This lesson is being piloted (Beta version)

Continuous Integration

Overview

Teaching: 10 min
Exercises: 10 min
Questions
  • How do you ensure code keeps passing

Objectives
  • Use a CI service to run your tests

Developers often need to run some tasks every time they update code. This might include running tests, or checking that the formatting conforms to a style guide.

Continuous Integration (CI) allows the developer to automate running these kinds of tasks each time various “trigger” events occur on your repository. For example, you can use CI to run a test suite on every pull request.

In this episode we will set up CI using GitHub Actions:

GitHub Actions workflows directory

GitHub Actions is made up of workflows which consist of actions. Workflows are files in the .github/workflows folder ending in .yml.

Triggers

Workflows start with triggers, which define when things run. Here are three triggers:

on:
  pull_request:
  push:
    branches:
      - main

This will run on all pull requests and pushes to main. You can also specify specific branches for pull requests instead of running on all PRs (will run on PRs targeting those branches only).

Running unit tests

Let’s set up a basic test. We will define a jobs dict, with a single job named “tests”. For all jobs, you need to select an image to run on - there are images for Linux, macOS, and Windows. We’ll use ubuntu-latest.

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.10"

      - name: Install package
        run: python -m pip install -e .[test]

      - name: Test package
        run: python -m pytest

This has five steps:

  1. Checkout the source (your repo).
  2. Prepare Python 3.10 (will use a preinstalled version if possible, otherwise will download a binary).
  3. Install your package with testing extras - this is just an image that will be removed at the end of the run, so “global” installs are fine. We also provide a nice name for the step.
  4. Run your package’s tests.

By default, if any step fails, the run immediately quits and fails.

Running in a matrix

You can parametrize values, such as Python version or operating system. Do do this, make a strategy: matrix: dict. Every key in that dict (except include: and exclude should be set with a list, and a job will be generated with every possible combination of values. You can access these values via the matrix variable; they do not “automatically” change anything.

For example:

example:
  strategy:
    matrix:
      onetwothree: [1, 2, 3]
  name: Job ${{ matrix.onetwothree }}

would produce three jobs, with names Job 1, Job 2, and Job 3. Elsewhere, if you refer to the example job, it will implicitly refer to all three.

This is commonly used to set Python and operating system versions:

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  tests:
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.8", "3.11"]
        runs-on: [ubuntu-latest, windows-latest, macos-latest]
    name: Check Python ${{ matrix.python-version }} on ${{ matrix.runs-on }}
    runs-on: ${{ matrix.runs-on }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install package
        run: python -m pip install -e .[test]

      - name: Test package
        run: python -m pytest

There are two special keys: include: will take a list of jobs to include one at a time. For example, you could add Python 3.9 on Linux (but not the others):

include:
  - python-version: 3.9
    runs-on: ubuntu-latest

include can also list more keys than were present in the original parametrization; this will add a key to an existing job.

The exclude: key does the opposite, and lets you remove jobs from the matrix.

Other actions

GitHub Actions has the concept of actions, which are just GitHub repositories of the form org/name@tag, and there are lots of useful actions to choose from (and you can write your own by composing other actions, or you can also create them with JavaScript or Dockerfiles). Here are a few:

There are some GitHub supplied ones:

And many other useful ones:

Exercise

Add a CI file for your package.

Key Points

  • Set up GitHub Actions on your project

  • Run your tests on multiple platforms and with multiple Python versions