Using the sqlite3 Python module in Pyodide - Python WebAssembly#

Pyodide provides “Python with the scientific stack, compiled to WebAssembly” - it’s an incredible project which lets you run a full working Jupyter notebook, complete with complex packages such as numpy and pandas, entirely in your browser without any server-side Python component running at all.

It turns out it also now includes a working version of the standard library sqlite3 module, by bundling a WebAssembly compiled version of SQLite!

Trying this in the REPL#

pyodide.org/en/stable/console.html provides an interactive REPL for trying eut Pyodide. You can run a one-liner to demonstrate the available SQLite version like this:

1
Welcome to the Pyodide terminal emulator 🐍
2
Python 3.9.5 (default, Sep 16 2021 11:22:45) on WebAssembly VM
3
Type "help", "copyright", "credits" or "license" for more information.
4
>>> import sqlite3
5
>>> sqlite3.connect(":memory:").execute("select sqlite_version()").fetchall()
6
[('3.27.2',)]

Querying an existing database file from JupyterLite#

JupyterLite is “a JupyterLab distribution that runs entirely in the web browser, backed by in-browser language kernels.”

Their online demo is at jupyterlite.github.io/demo/lab/index.html. I opened that demo and created a new Pyolite notebook there, then used the bridge to the JavaScript fetch() function to download the 11MB power plants database file from this URL:

https://global-power-plants.datasettes.com/global-power-plants.db

(Downloading this via fetch() works because Datasette includes CORS headers for these files.)

1
from js import fetch
2

3
res = await fetch("https://global-power-plants.datasettes.com/global-power-plants.db")
4
buffer = await res.arrayBuffer()
5

6
# Now write that to the in-memory simulated filesystem:
7
open("tmp/power.db", "wb").write(bytes(buffer.valueOf().to_py()))
8

9
# And run some queries against it:
10
import sqlite3
11
c = sqlite3.connect("tmp/power.db")
12
c.execute('select * from "global-power-plants" limit 10').fetchall()

This works!

Screenshot of JupyterLite running my example code