Packaging a Python app as a standalone binary with PyInstaller#

PyInstaller can take a Python script and bundle it up as a standalone executable for macOS, Linux and apparently Windows too (I’ve not tried it on Windows yet).

Today I managed to build Datasette executables for macOS and Linux using the following recipe:

1
export DATASETTE_BASE=$(python -c 'import os; print(os.path.dirname(__import__("datasette").__file__))') \
2
pyinstaller -F \
3
    --add-data "$DATASETTE_BASE/templates:datasette/templates" \
4
    --add-data "$DATASETTE_BASE/static:datasette/static" \
5
    --hidden-import datasette.publish \
6
    --hidden-import datasette.publish.heroku \
7
    --hidden-import datasette.publish.cloudrun \
8
    --hidden-import datasette.facets \
9
    --hidden-import datasette.sql_functions \
10
    --hidden-import datasette.actor_auth_cookie \
11
    --hidden-import datasette.default_permissions \
12
    --hidden-import datasette.default_magic_parameters \
13
    --hidden-import datasette.blob_renderer \
14
    --hidden-import datasette.default_menu_links \
15
    --hidden-import uvicorn \
16
    --hidden-import uvicorn.logging \
17
    --hidden-import uvicorn.loops \
18
    --hidden-import uvicorn.loops.auto \
19
    --hidden-import uvicorn.protocols \
20
    --hidden-import uvicorn.protocols.http \
21
    --hidden-import uvicorn.protocols.http.auto \
22
    --hidden-import uvicorn.protocols.websockets \
23
    --hidden-import uvicorn.protocols.websockets.auto \
24
    --hidden-import uvicorn.lifespan \
25
    --hidden-import uvicorn.lifespan.on \
26
    $(which datasette)

The --hidden-import lines are needed because PyInstaller attempts to follow the module import graph for a package, but is very easily confused. Datasette dynamically imports a list of default plugins so I had to explicitly list each of those. I don’t know what’s going on with uvicorn here - I kept on running the script and then running dist/datasette and getting errors like this one:

1
(pyinstaller-datasette) pyinstaller-datasette % dist/datasette ~/Dropbox/Development/datasette/fixtures.db
2
Traceback (most recent call last):
3
  File "datasette", line 8, in <module>
4
  File "click/core.py", line 829, in __call__
5
  File "click/core.py", line 782, in main
6
  File "click/core.py", line 1259, in invoke
7
  File "click/core.py", line 1066, in invoke
8
  File "click/core.py", line 610, in invoke
9
  File "datasette/cli.py", line 548, in serve
10
  File "uvicorn/main.py", line 386, in run
11
  File "uvicorn/server.py", line 48, in run
12
  File "asyncio/base_events.py", line 642, in run_until_complete
13
  File "uvicorn/server.py", line 55, in serve
14
  File "uvicorn/config.py", line 301, in load
15
  File "uvicorn/importer.py", line 23, in import_from_string
16
  File "uvicorn/importer.py", line 20, in import_from_string
17
  File "importlib/__init__.py", line 127, in import_module
18
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
19
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
20
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
21
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
22
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
23
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
24
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
25
ModuleNotFoundError: No module named 'uvicorn.protocols.websockets'
26
[32142] Failed to execute script datasette

I solved this by adding each ModuleNotFoundError module to --hidden-import until it worked.

I’ve tested this script (and the generated executables) on both macOS and Ubuntu Linux so far, and it’s worked perfectly in both cases. See issue 93 for more details.