Protocols in Python#

Datasette currently has a few API internals that return sqlite3.Row objects. I was thinking about how this might work in the future - if Datasette ever expands beyond SQLite (plugin-provided backends for PostgreSQL and DuckDB for example) I’d want a way to return data from other stores using objects that behave like sqlite3.Row but are not exactly that class.

I thought about implementing my own wrapper class for sqlite3.Row, but one of its benefits is that it’s written in C and hence should provide optimal memory usage and performance.

It looks like that’s what typing.Protocol() is for.

Here’s some code I put together (with initial assistance from both Claude and ChatGPT) to explore what that would look like:

1
from typing import Any, Dict, List, Protocol, Union
2
import sqlite3
3

4

5
class RowProtocol(Protocol):
6
    def keys(self) -> List[str]:
7
        ...
8

9
    def __getitem__(self, index: Union[int, str]) -> Any:
10
        ...
11

12

13
class MyRow:
14
    def __init__(self, data: Dict[str, Any]):
15
        self.data = data
16

17
    def keys(self) -> List[str]:
18
        return list(self.data.keys())
19

20
    def __getitem__(self, index: Union[int, str]) -> Any:
21
        if isinstance(index, int):
22
            key = self.keys()[index]
23
            return self.data.get(key)
24
        elif isinstance(index, str):
25
            return self.data.get(index)
26
        else:
27
            raise TypeError("Index must be either int or str.")
28

29

30
def get_rows() -> List[RowProtocol]:
31
    row1 = MyRow({"name": "Milo", "species": "cat"})
32

33
    conn = sqlite3.connect(":memory:")
34
    conn.row_factory = sqlite3.Row
35
    row2 = conn.execute("select 'Cleo' as name, 'dog' as species").fetchone()
36

37
    return [row1, row2]
38

39

40
if __name__ == "__main__":
41
    rows = get_rows()
42
    for row in rows:
43
        # Uncomment this when running mypy:
44
        # reveal_type(row)
45
        print(row.keys(), row["name"])

This passes a mypy check. Running it demonstrates that the MyRow and sqlite3.Row objects can be treated equivalently.

Uncommenting reveal_type(row) causes mypy to print out the RowProtocol type while it is running.

The thing that surprised me about this at first is that I had expected I would need to “register” the types with the protocol in some way - but it turns out protocols really are just a formalization of Python’s duck typing.

Effectively this code is saying “the objects returned by get_rows() should only be accessed via their .keys() and __getitem__() methods”.

Which looks like exactly what I would need to implement my own alternative to sqlite3.Row in the future in a way that works neatly with Python type checking tools.

1
from typing import TYPE_CHECKING
2

3
...
4

5
if TYPE_CHECKING:
6
    reveal_type(obj)