Newsletter
TechAnV Blog
Get updates on security engineering, Rust, eBPF, and DevSecOps. No spam, unsubscribe anytime.
Check your inbox and click the confirmation link to complete your subscription.
Using pytest-httpx to run intercepted requests through an in-memory Datasette instance#
I’ve been working on a tool called dclient which is a CLI client tool for talking to Datasette instances.
I wanted to write some tests for that tool which would simulate an entire Datasette instance for it to talk to, without starting a localhost server running Datasette itself.
I figured out a pattern for doing that using pytest-httpx to intercept outbound HTTP requests and send them through a Datasette ASGI application instead.
Here’s a simplified example of this pattern, with inline comments explaining how it works.
Dependencies are:
1pip install pytest pytest-httpx httpx datasetteI saved this as test_demo.py and ran it with pytest test_demo.py:
1import asyncio2from datasette.app import Datasette3import httpx4import pytest5
6
7@pytest.fixture8def non_mocked_hosts():9 # This ensures that httpx-mock will not affect things once a request10 # starts being processed within Datasette itself via datasette.client11 return ["localhost"]12
13
14# Here's an example function we will be testing. This uses httpx.get() to15# make an outbound HTTP request to a Datasette instance - we want to intercept16# that call and handle it ourselves using httpx_mock17def get_versions():18 response = httpx.get("https://datasette.example.com/-/versions.json")19 response.raise_for_status()20 return response.json()21
22
23# The httpx_mock fixtures comes from pytest-httpx24def test_get_version(httpx_mock):25 ds = Datasette()26
27 # This test is a regular function, but we need to be able to make some28 # await... calls later on - so we need the event loop to run them against29 loop = asyncio.get_event_loop()30
31 # This function will be called every time pytest-httpx intercepts an HTTP request32 def custom_response(request: httpx.Request):33 # Need to run this in async loop, because get_versions uses34 # sync HTTPX and not async HTTPX35 async def run():36 # Here we use ds.client.request() - an internal method within Datasette37 # which can be used to simulate passing an HTTP request through the ASGI app38 # This is why we needed to include localhost in non_mocked_hosts earlier39 response = await ds.client.request(40 request.method,41 request.url.path,42 content=request.read(),43 headers=request.headers,44 )45 # Create a fresh response to avoid an error where stream has been consumed46 response = httpx.Response(47 status_code=response.status_code,48 headers=response.headers,49 content=response.content,50 )51 return response52
53 return loop.run_until_complete(run())54
55 # We add custom_response as a callback function for any intercepted HTTP requests:56 httpx_mock.add_callback(custom_response)57
58 # And here's our actual test59 versions = get_versions()60 assert "asgi" in versions61 assert "datasette" in versions62 assert "python" in versionsYou can see a much more complex example of this pattern in action in this file: https://github.com/simonw/dclient/blob/0.2/tests/test_insert.py