Start a server in a subprocess during a pytest session#

I wanted to start an actual server process, run it for the duration of my pytest session and shut it down at the end.

Here’s the recipe I came up with. This fixture lives in conftest.py:

1
import pytest
2
import sqlite_utils
3
import subprocess
4

5
@pytest.fixture(scope="session")
6
def ds_server(tmp_path_factory):
7
    db_directory = tmp_path_factory.mktemp("dbs")
8
    db_path = db_directory / "test.db"
9
    db = sqlite_utils.Database(db_path)
10
    insert_test_data(db)
11
    ds_proc = subprocess.Popen(
12
        [
13
            "datasette",
14
            str(db_path),
15
            "-p",
16
            "8041"
17
        ],
18
        stdout=subprocess.PIPE,
19
        stderr=subprocess.STDOUT,
20
    )
21
    # Give the server time to start
22
    time.sleep(2)
23
    # Check it started successfully
24
    assert not ds_proc.poll(), ds_proc.stdout.read().decode("utf-8")
25
    yield ds_proc
26
    # Shut it down at the end of the pytest session
27
    ds_proc.terminate()

A test looks like this:

1
import httpx
2

3
def test_server_starts(ds_server):
4
    response = httpx.get("http://127.0.0.1:8041/")
5
    assert response.status_code == 200

Alternative recipe for serving static files#

While adding tests to Datasette Lite I found myself needing to run a localhost server that served static files directly.

I completely forgot about this TIL, and instead took inspiration from pytest-simplehttpserver - coming up with this pattern:

1
from subprocess import Popen, PIPE
2
import pathlib
3
import pytest
4
import time
5
from http.client import HTTPConnection
6

7
root = pathlib.Path(__file__).parent.parent.absolute()
8

9

10
@pytest.fixture(scope="module")
11
def static_server():
12
    process = Popen(
13
        ["python", "-m", "http.server", "8123", "--directory", root], stdout=PIPE
14
    )
15
    retries = 5
16
    while retries > 0:
17
        conn = HTTPConnection("localhost:8123")
18
        try:
19
            conn.request("HEAD", "/")
20
            response = conn.getresponse()
21
            if response is not None:
22
                yield process
23
                break
24
        except ConnectionRefusedError:
25
            time.sleep(1)
26
            retries -= 1
27

28
    if not retries:
29
        raise RuntimeError("Failed to start http server")
30
    else:
31
        process.terminate()
32
        process.wait()

Again, including static_server as a fixture is enough to ensure requests to http://localhost:8123/ will be served by that temporary server.

I like how this version polls for a successful HEAD request (a trick inspired by pytest-simplehttpserver) rather than just sleeping.