Code coverage using pytest and codecov.io#

I got my asgi-csrf Python package up to 100% code coverage. Here’s the pull request.

I started by installing and using the pytest-cov pytest plugin.

1
pip install pytest-cov
2
pytest --cov=asgi_csrf

This shows the current code coverage percentage for the asgi_csrf module in the terminal output:

1
collected 18 items
2

3
test_asgi_csrf.py ..................                                                                                                                           [100%]
4

5
---------- coverage: platform darwin, python 3.7.3-final-0 -----------
6
Name           Stmts   Miss  Cover
7
----------------------------------
8
asgi_csrf.py     169     13    92%
9

10

11
========= 18 passed in 0.37s =========

To generate an HTML report showing which lines are not covered by tests:

1
pytest --cov=asgi_csrf --cov-report=html
2
open htmlcov/index.html

Here’s a hosted copy of that report: https://asgi-csrf-htmlcov-ewca4t9se.vercel.app/asgi_csrf_py.html

Failing the tests if coverage is below a certain threshold#

The --cov-fail-under=100 option does this:

1
pytest --cov-fail-under=100 --cov asgi_csrf
2
======= test session starts =======
3
platform darwin -- Python 3.7.3, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
4
rootdir: /Users/simon/Dropbox/Development/asgi-csrf
5
plugins: cov-2.10.1, asyncio-0.14.0
6
collected 18 items
7

8
test_asgi_csrf.py ..................                                                                                                                           [100%]
9

10
---------- coverage: platform darwin, python 3.7.3-final-0 -----------
11
Name           Stmts   Miss  Cover
12
----------------------------------
13
asgi_csrf.py     169     13    92%
14

15
FAIL Required test coverage of 100% not reached. Total coverage: 92.31%

I added this to my GitHub test action:

1
    - name: Run tests
2
      run: |
3
        pytest --cov-fail-under=100 --cov asgi_csrf

Pushing results to codecov.io#

https://codecov.io/ offers free coverage reporting for open source projects. I authorized it against my GitHub account, then enabled it for the asgi-csrf project by navigating to https://codecov.io/gh/simonw/asgi-csrf (hacking the URL saves you from having to paginate through all of your repos looking for the right one).

codecov.io gives you a token - set that as a GitHub repository secret as CODECOV_TOKEN - then add the following to the test action configuration:

1
    - name: Upload coverage to codecov.io
2
      run: bash <(curl -s https://codecov.io/bash)
3
      if: always()
4
      env:
5
        CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

This will upload your coverage report (no matter if the previous test step failed or succeeded). codecove.io then reports back to pull requests and maintains a dashboard for your project.

codecov.io doesn’t detect if you use a main or master branch so I had to switch the default branch in the settings at https://codecov.io/gh/simonw/asgi-csrf/settings

I also added their badge to my project README, using the following Markdown:

1
[![codecov](https://codecov.io/gh/simonw/asgi-csrf/branch/main/graph/badge.svg)](https://codecov.io/gh/simonw/asgi-csrf)

Newsletter

Code coverage using pytest and codecov.io#

Failing the tests if coverage is below a certain threshold#

Pushing results to codecov.io#