Opt-in integration tests with pytest —integration#

For both s3-credentials and datasette-publish-fly I have a need for real-world integration tests that actually interact with the underlying APIs (AWS or Fly) to create and destroy resources on those platforms.

Most of the time I want my tests to run without doing these. I want the option to run pytest --integration to opt-in to running those extra integration tests.

Here’s the pattern I’m using. First, in tests/conftest.py:

1
import pytest
2

3

4
def pytest_addoption(parser):
5
    parser.addoption(
6
        "--integration",
7
        action="store_true",
8
        default=False,
9
        help="run integration tests",
10
    )
11

12

13
def pytest_configure(config):
14
    config.addinivalue_line(
15
        "markers",
16
        "integration: mark test as integration test, only run with --integration",
17
    )
18

19

20
def pytest_collection_modifyitems(config, items):
21
    if config.getoption("--integration"):
22
        # Also run integration tests
23
        return
24
    skip_integration = pytest.mark.skip(reason="use --integration option to run")
25
    for item in items:
26
        if "integration" in item.keywords:
27
            item.add_marker(skip_integration)

This implements a @pytest.mark.integration marker which I can use to mark any test that should be considered part of the integration test suite. These will be skipped by default… but will not be skipped if the --integration option is passed to pytest.

Then in the tests I can either do this:

1
@pytest.mark.integration
2
def test_integration_s3():
3
    # ...

Or if I have a module that only contains integration tests - tests/test_integration.py - I can use the following line to apply that decorator to every test in the module:

1
import pytest
2

3
pytestmark = pytest.mark.integration
4

5
def test_integration_s3():
6
    # ...