928 words
5 minutes
A few notes on Rye

A few notes on Rye#

Rye is Armin Ronacher’s new experimental Python packaging tool. I decided to take it for a test-run.

Installing Rye#

Rye is built in Rust, and currently needs to be installed using Cargo. I had Cargo from previous dalliances with Rust, so I could install Rye using:

cargo install --git https://github.com/mitsuhiko/rye rye

This resulted in a 6.4MB binary file in:

~/.cargo/bin/rye

I added the following line to my ~/.zshrc file to make it available on my path:

export PATH=$PATH:$HOME/.cargo/bin

Installing Python with Rye#

The Rye docs suggested running this:

rye pin cpython@3.11

As far as I can tell all this did was write 3.11.1 to a ~/.python-version file. I’m not sure if there are any tools other than Rye which pay attention to this file. (UPDATE: pyenv uses this file too)

Actually fetching versions of Python can be done using rye toolchain fetch VERSION, for example:

% rye toolchain fetch 3.9
Downloading cpython@3.9.16
success: Downloaded cpython@3.9.16

This placed a whole bunch of files in ~/.rye/py/cpython@3.9.16/ - find . | wc -l reported 4,085 in that directory.

The one that matters most is:

~/.rye/py/cpython@3.9.16/install/bin/python3

Which gives me a 3.9.16 Python interpreter.

This is the feature of Rye I’m most excited about: the Python bundles it installs come from Gregory Szorc’s indygreg/python-build-standalone project.

This should mean that they completely ignore the many other weird ways that Python can end up installed on a system. Admittedly, this is a new weird way to install Python - but at least it shouldn’t clash with anything else.

Normally though you wouldn’t use rye toolchain fetch at all. The Rye suggested workflow is to run rye init in a new directory to create a pyproject.toml file:

% cd /tmp
/tmp % mkdir my-project
/tmp % cd my-project
my-project % rye init
success: Initialized project in /private/tmp/my-project/.
my-project % ls
README.md pyproject.toml
my-project % cat pyproject.toml
[project]
name = "my-project"
version = "0.1.0"
description = "Add a short description here"
authors = [
{ name = "Simon Willison", email = "...@gmail.com" }
]
dependencies = []
readme = "README.md"
requires-python = ">= 3.8"
license = { text = "MIT" }
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.rye]
managed = true
my-project % cat README.md
# my-project
Describe your project here.
* License: MIT

To add dependencies to that project, use rye add:

rye add httpx

This added the following line to pyproject.toml:

dependencies = ["httpx~=0.24.0"]

Then to actually install the dependencies, run rye sync:

my-project % rye sync
Initializing new virtualenv in /private/tmp/my-project/.venv
Python version: cpython@3.11.1
Generating production lockfile: /private/tmp/my-project/requirements.lock
Generating dev lockfile: /private/tmp/my-project/requirements-dev.lock
Installing dependencies
...
Successfully built my-project
Installing collected packages: sniffio, idna, h11, certifi, anyio, httpcore, httpx, my-project
Successfully installed anyio-3.6.2 certifi-2022.12.7 h11-0.14.0 httpcore-0.17.0 httpx-0.24.0 idna-3.4 my-project-0.1.0 sniffio-1.3.0
Done!

This adds a requirements.lock file that looks like this:

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
-e file:.
anyio==3.6.2
certifi==2022.12.7
h11==0.14.0
httpcore==0.17.0
httpx==0.24.0
idna==3.4
sniffio==1.3.0

And a requirements-dev.lock file with identical contents. Presumably this starts to differ as you install dev requirements (another Rye feature).

Your folder will now have a .venv hidden folder in it. Inside that is a Python virtual environment containing your installed dependencies and a file called rye-venv.json which just contains:

{
"python": "cpython@3.11.1"
}

Installing global applications#

The rye add X command adds a dependency to your current virtual environment / Rye project.

rye install does something completely different: it installs new global packages, in a similar fashion to pipx in that they get their own isolated environments so their dependencies don’t clash with other installed applications.

% rye install cowsay
Collecting cowsay
Downloading cowsay-5.0.tar.gz (25 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cowsay
Building wheel for cowsay (pyproject.toml) ... done
Created wheel for cowsay: filename=cowsay-5.0-py2.py3-none-any.whl size=25707 sha256=de872e9ef328d25cd9ac58df693c7bfd913033f696282c60562854d0db38737e
Stored in directory: /Users/simon/Library/Caches/pip/wheels/d4/2d/c7/c018bd8e6f825d6b47ae38f28baabd4588b3857e0e7dbc8cd3
Successfully built cowsay
Installing collected packages: cowsay
Successfully installed cowsay-5.0
installed script cowsay

You can now run cowsay with ~/.rye/shims/cowsay:

% ~/.rye/shims/cowsay hello
_____
| hello |
=====
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||

You can add ~/.rye/shims to your $PATH to make these commands available everywhere.

Failing and then succeeding to install Datasette#

I tried to install Datasette using rye install datasette and got this error:

% ~/.rye/shims/datasette
Traceback (most recent call last):
File "/Users/simon/.rye/shims/datasette", line 5, in <module>
from datasette.cli import cli
File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/cli.py", line 17, in <module>
from .app import (
File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/app.py", line 14, in <module>
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Rye has strong opinions, including omitting pip and setuptools entirely from the environments that it creates.

[ UPDATE: I released Datasette 0.64.3 with a fix for this and now it installs correctly under Rye ]

It turns out Datasette includes code that imports pkg_resources, assuming that setuptools will be present because it’s usually there as a Python environment default!

I added setuptools to Datasette’s setup.py dependencies in an attempt to fix that, in issue #2065.

When I’m using pip I often install development copies of my projects by feeding them the URL to a .zip generated by GitHub, for example:

pip install https://github.com/simonw/datasette/archive/main.zip

I tried that with rye install and got an error:

% rye install https://github.com/simonw/datasette/archive/main.zip
Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:`
https://github.com/simonw/datasette/archive/main.zip
^

Evidently Rye doesn’t (yet?) support installing from URLs.

UPDATE: Armin showed me a way to do this:

rye install 'datasette @ https://github.com/simonw/datasette/archive/main.zip'

This worked - running ~/.rye/shims/datasette --version confirmed the installation.

Instead, I downloaded the main.zip file and ran Rye install against that directly:

% rye install main.zip
Processing ./main.zip
Installing build dependencies ... done
Getting requirements to build wheel ... done
...
Successfully built datasette
...

That seemed to work. Oddly it didn’t add datasette to the .rye/shims directory - but I did find a working installation here instead:

~/.rye/tools/main-zip/bin/datasette
A few notes on Rye
https://mranv.pages.dev/posts/a-few-notes-on-rye/
Author
Anubhav Gain
Published at
2024-05-16
License
CC BY-NC-SA 4.0