Snapshot testing with Syrupy#

I’m a big fan of snapshot testing - writing tests where you compare the output of some function to a previously saved version, and can re-generate that version from scratch any time something changes.

I usually do this by hand - I run pytest -x --pdb to stop at the first failing test and drop into a debugger, then copy out the representation of the generated value and copy it into the test. I wrote about how I use this pattern a few years ago in How to cheat at unit tests with pytest and Black.

Today I learned how to do the same thing with the Syrupy plugin for pytest. I think I’ll be using this for many of my future projects.

Some initial tests#

I created a tests/test_stuff.py file with the following contents:

1
def test_one(snapshot):
2
    assert "hello" == snapshot
3

4

5
def test_two(snapshot):
6
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}

Then I installed both pytest and syrupy:

1
pip install pytest syrupy

Now in my parent folder I can run this:

1
pytest

And the tests fail:

1
tests/test_stuff.py FF                                                              [100%]
2

3
======================================== FAILURES =========================================
4
________________________________________ test_one _________________________________________
5

6
snapshot = SnapshotAssertion(name='snapshot', num_executions=1)
7

8
    def test_one(snapshot):
9
>       assert "hello" == snapshot
10
E       AssertionError: assert [+ received] == [- snapshot]
11
E         Snapshot 'test_one' does not exist!
12
E         + 'hello'
13

14
tests/test_stuff.py:2: AssertionError
15
________________________________________ test_two _________________________________________
16

17
snapshot = SnapshotAssertion(name='snapshot', num_executions=1)
18

19
    def test_two(snapshot):
20
>       assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}
21
E       AssertionError: assert [- snapshot] == [+ received]
22
E         Snapshot 'test_two' does not exist!
23
E         + dict({
24
E         +   'bar':
25
E
26
E         ...Full output truncated (9 lines hidden), use '-vv' to show
27

28
tests/test_stuff.py:5: AssertionError
29
--------------------------------- snapshot report summary ---------------------------------
30
2 snapshots failed.
31
================================= short test summary info =================================
32
FAILED tests/test_stuff.py::test_one - AssertionError: assert [+ received] == [- snapshot]
33
FAILED tests/test_stuff.py::test_two - AssertionError: assert [- snapshot] == [+ received]
34
==================================== 2 failed in 0.05s ====================================

The snapshots don’t exist yet. But I can create them automatically by running this:

1
pytest --snapshot-update

Which outputs passing tests along with:

1
--------------------------------- snapshot report summary ---------------------------------
2
2 snapshots generated.
3
==================================== 2 passed in 0.01s ====================================

And sure enough, there’s now a new folder called tests/__snapshots__ with a file called test_stuff.ambr which contains this:

1
# serializer version: 1
2
# name: test_one
3
  'hello'
4
# ---
5
# name: test_two
6
  dict({
7
    'bar': dict({
8
      'baz': 'qux',
9
    }),
10
    'foo': list([
11
      1,
12
      2,
13
      3,
14
    ]),
15
  })
16
# ---

Running pytest again passes, because the snapshots exist and continue to match the test output.

The serialized snapshot format is designed to be checked into Git. It’s pleasantly readable - I can review that and see what it’s testing, and I could even update it by hand - though I’ll much more likely use the --snapshot-update flag and then eyeball the differences.

Adding a dataclass#

My snapshots so far are pretty simple - a string and a nested dictionary. I decided to add a dataclass to my code and see what that looks like:

1
import dataclasses
2

3

4
@dataclasses.dataclass
5
class Foo:
6
    bar: int
7
    baz: str
8

9

10
def test_one(snapshot):
11
    assert "hello" == snapshot
12

13

14
def test_two(snapshot):
15
    assert snapshot == {"foo": [1, 2, 3], "bar": {"baz": "qux"}}
16

17

18
def test_three(snapshot):
19
    assert Foo(1, "hello") == snapshot

Running pytest again failed. pytest --snapshot-update passed and updated my snapshot file, adding this to it:

1
# name: test_three
2
  Foo(bar=1, baz='hello')

OK, neat - it looks like it’s using the Dataclass’s __repr__ method to serialize the object.

I tried it with a custom non-dataclass object… and it worked too!

1
class WeirdClass:
2
    def __init__(self, foo, bar):
3
        self.foo = foo
4
        self.bar = bar
5

6
def test_four(snapshot):
7
    assert WeirdClass(1, 2) == snapshot

Serialized to:

1
# name: test_four
2
  WeirdClass(
3
    bar=2,
4
    foo=1,
5
  )

I wasn’t expecting this to work. The Syrupy documentation says:

The default serializer supports all python built-in types and provides a sensible default for custom objects.

It looks like there are a bunch of more advanced ways to customize objects to make them work well with Syrupy, but I haven’t dived into those yet.

First impressions are that this looks like exactly the snapshot tool I’ve been waiting for.