git bisect#

I extracted and enhanced this TIL from my April 8th 2020 weeknotes to make it easier to find.

I fixed two bugs in Datasette using git bisect run - a tool which lets you run an automated binary search against a commit log to find the source of a bug.

Since I was figuring out a new tool, I fired up another GitHub issue self-conversation: in issue #716 I document my process of both learning to use git bisect run and using it to find a solution to that particular bug.

It worked great, so I used the same trick on issue #689 as well.

Watching git bisect run churn through 32 revisions in a few seconds and pinpoint the exact moment a bug was introduced is pretty delightful.

The first step is to tell it the range of commits that you want to search in, using git bisect start:

1
$ git bisect start main 0.34
2
Bisecting: 32 revisions left to test after this (roughly 5 steps)
3
[dc80e779a2e708b2685fc641df99e6aae9ad6f97] Handle scope path if it is a string

Then you provide a script that will return an error if the bug is present.

Usually you would use pytest or similar for this, but for the bug I was investigating here I wrote this custom script and saved it as check_templates_considered.py:

1
import asyncio
2
import pathlib
3
from datasette.app import Datasette
4
import httpx
5

6
async def run_check():
7
    ds = Datasette([])
8
    async with httpx.AsyncClient(app=ds.app()) as client:
9
        response = await client.get("http://localhost/")
10
        assert 200 == response.status_code
11
        assert "Templates considered" in response.text
12

13
if __name__ == "__main__":
14
    loop = asyncio.get_event_loop()
15
    loop.run_until_complete(run_check())

This script will fail with an assertion error if Templates considered is not included in the HTML for the homepage.

To run the bisection, use git bisect run <script goes here>:

1
$ git bisect run python check_templates_considered.py
2
running python check_templates_considered.py
3
Traceback (most recent call last):
4
...
5
AssertionError
6
Bisecting: 15 revisions left to test after this (roughly 4 steps)
7
[7c6a9c35299f251f9abfb03fd8e85143e4361709] Better tests for prepare_connection() plugin hook, refs #678
8
running python check_templates_considered.py
9
Traceback (most recent call last):
10
...
11
AssertionError
12
Bisecting: 7 revisions left to test after this (roughly 3 steps)
13
[0091dfe3e5a3db94af8881038d3f1b8312bb857d] More reliable tie-break ordering for facet results
14
running python check_templates_considered.py
15
Traceback (most recent call last):
16
...
17
AssertionError
18
Bisecting: 3 revisions left to test after this (roughly 2 steps)
19
[ce12244037b60ba0202c814871218c1dab38d729] Release notes for 0.35
20
running python check_templates_considered.py
21
Traceback (most recent call last):
22
...
23
AssertionError
24
Bisecting: 1 revision left to test after this (roughly 1 step)
25
[70b915fb4bc214f9d064179f87671f8a378aa127] Datasette.render_template() method, closes #577
26
running python check_templates_considered.py
27
Traceback (most recent call last):
28
...
29
AssertionError
30
Bisecting: 0 revisions left to test after this (roughly 0 steps)
31
[286ed286b68793532c2a38436a08343b45cfbc91] geojson-to-sqlite
32
running python check_templates_considered.py
33
70b915fb4bc214f9d064179f87671f8a378aa127 is the first bad commit
34
commit 70b915fb4bc214f9d064179f87671f8a378aa127
35
Author: Simon Willison
36
Date:   Tue Feb 4 12:26:17 2020 -0800
37

38
    Datasette.render_template() method, closes #577
39

40
    Pull request #664.
41

42
:040000 040000 def9e31252e056845609de36c66d4320dd0c47f8 da19b7f8c26d50a4c05e5a7f05220b968429725c M  datasette
43
bisect run success

The final output shows exactly which commit introduced the bug. In this case it was 70b915fb4bc214f9d064179f87671f8a378aa127 (the “first bad commit”).