Publish releases to PyPI from GitHub Actions without a password or token#

Adding a new trusted publisher on PyPI#

This process differs slightly depending on if you are planning on publishing a brand new package or updating an existing one to use Trusted Publishers going forward.

When publishing a brand new package you can instead use a special mechanism called “pending publishers”.

This also lets you reserve a package name before you publish the first version. I like this - in the past I’ve attempted to publish a package only to discover that someone else had already reserved a too-similar name.

You can create a new pending publisher from this page: https://pypi.org/manage/account/publishing/

You need to provide the PyPI project name, the GitHub repository owner and name, the workflow name (the name of a YAML file) and an optional environment name.

Here’s how I filled out that form for my new datasette/datasette-build repository:

I used publish.yml as the name of my workflow file.

I also set the environment to release. I don’t fully understand GitHub Actions environments yet, but the PyPI documentation suggested this was a good idea and I think it gives me more flexibility for setting extra permissions in the future.

PyPI says:

While not required, a dedicated publishing environment is strongly encouraged, especially if your repository has maintainers with commit access who shouldn’t have PyPl publishing access.

Creating the GitHub Actions environment#

Since we specified the environment on PyPI we need to create that environment. That can be done in the settings area for the repository - in my case that page was here:

https://github.com/datasette/datasette-build/settings/environments/new

Environments just have a name. I called mine release.

Configuring the workflow#

This took me the most time to figure out. I already have a publish.yml workflow I use for my other projects, which uses twine and a PyPI token to upload packages, after first running the tests.

Here’s the workflow I eventually landed, in .github/workflows/publish.yml:

1
name: Publish Python Package
2

3
on:
4
  release:
5
    types: [created]
6

7
jobs:
8
  test:
9
    runs-on: ubuntu-latest
10
    strategy:
11
      matrix:
12
        python-version: ["3.8", "3.9", "3.10", "3.11"]
13
    steps:
14
    - uses: actions/checkout@v4
15
    - name: Set up Python ${{ matrix.python-version }}
16
      uses: actions/setup-python@v5
17
      with:
18
        python-version: ${{ matrix.python-version }}
19
        cache: pip
20
        cache-dependency-path: '**/pyproject.toml'
21
    - name: Install dependencies
22
      run: |
23
        pip install -e '.[test]'
24
    - name: Run tests
25
      run: |
26
        pytest
27
  deploy:
28
    runs-on: ubuntu-latest
29
    needs: [test]
30
    environment: release
31
    permissions:
32
      id-token: write
33
    steps:
34
    - uses: actions/checkout@v4
35
    - name: Set up Python
36
      uses: actions/setup-python@v5
37
      with:
38
        python-version: '3.11'
39
        cache: pip
40
        cache-dependency-path: '**/pyproject.toml'
41
    - name: Install dependencies
42
      run: |
43
        pip install setuptools wheel build
44
    - name: Build
45
      run: |
46
        python -m build
47
    - name: Publish
48
      uses: pypa/gh-action-pypi-publish@release/v1

The test job is pretty standard - it sets up a matrix to run the tests against multiple Python versions, then runs pytest.

It’s set to trigger by this block:

1
on:
2
  release:
3
    types: [created]

This ensures the workflow runs any time a new GitHub release is created for the repository.

Where things get interesting is the deploy job. It runs python -m build to build the .tar.gz and .whl files, then uses the pypa/gh-action-pypi-publish@release/v1 action to publish the package.

Breaking that down a bit:

1
  deploy:
2
    runs-on: ubuntu-latest
3
    needs: [test]
4
    environment: release
5
    permissions:
6
      id-token: write

The environment: release key is needed because we configured an environment in PyPI. I think that can be omitted entirely if the PyPI environment field was left blank.

The permissions block there is essential - it’s required for the OpenID Connect token authentication to work.

needs: [test] means that this job waits for the test job to pass before it runs.

1
    - name: Publish
2
      uses: pypa/gh-action-pypi-publish@release/v1

This is the key line. It uses the pypa/gh-action-pypi-publish action to publish the package.

Note that it doesn’t need any settings - it just works, provided the trusted publisher on PyPI has been configured.

Publishing a release#

With all this in place, all that’s needed to ship a package is to ensure the version is set correctly in the pyproject.toml file (or setup.py file if you’re using that instead), then create a new release on GitHub.

For my repo I create a release using this form: https://github.com/datasette/datasette-build/releases/new

Creating the release triggers the workflow, which runs the tests, builds the package and then publishes it to PyPI.

Here’s the resulting package: pypi.org/project/datasette-build/

Configuring a trusted publisher for an existing package#

Here are the instructions for upgrading an existing package.

I tried this myself for datasette-edit-templates. Here’s the diff. I used this process:

1. Upgrade the publish.yml deploy step to include environment: release, permissions: id-token: write, a python -m build step and the pypa/gh-action-pypi-publish@release/v1 action.
2. Navigate to that projkect on PyPI and add a new trusted publisher - for my project that page was https://pypi.org/manage/project/datasette-edit-templates/settings/publishing/
3. Create the environment called release on the GitHub settings page for that repository.
4. Update the version number and create a new release.

The PyPI form looked like this: