Building Mastodon bots with GitHub Actions and toot#

Twitter announced today that they’ll be ending free API access for bots.

My @covidsewage Twitter bot posts a screenshot of the latest Covid sewage data for parts of the San Francisco Bay Area every morning. I decided to port it to Mastodon.

It’s now up and running in its new home at https://fedi.simonwillison.net/@covidsewage - here’s how the new bot works.

toot - a Mastodon command line client#

The bot uses toot to send a message with an attached image to Mastodon.

Here’s the command that does that:

1
toot post "Latest Covid sewage charts for the SF Bay Area https://covid19.sccgov.org/dashboard-wastewater" \
2
  --media /tmp/covid.png \
3
  --description "Screenshot of the latest Covid charts"

You can pass the combination of --media and --description up to four times.

The alt text here is terrible - I’ve had an open issue to fix that for a while, but it’s not an easy thing to implement properly.

Authentication#

The toot post command only works if you authenticate first.

Toot authentication is really nice. All you have to do is run:

1
toot login

It will ask for your Mastodon instance (my private one is fedi.simonwillison.net) and spit out a link to click on.

Click that link and your Mastodon server will ask you if you want to authorize the tool.

If you say yes, it gives you an authorization code which you then paste back into the command.

Toot then creates a file in ~/.config/toot/config.json. My file (redacted) looks like this:

1
{
2
 "active_user": "covidsewage@fedi.simonwillison.net",
3
 "apps": {
4
  "fedi.simonwillison.net": {
5
   "base_url": "https://fedi.simonwillison.net",
6
   "client_id": "cTQfcJy9EhlIUSGPRx90PRnMx_RroBuLUw8WcMvguD0",
7
   "client_secret": "... redacted ...",
8
   "instance": "fedi.simonwillison.net"
9
  }
10
 },
11
 "users": {
12
  "covidsewage@fedi.simonwillison.net": {
13
   "access_token": "... redacted ...",
14
   "instance": "fedi.simonwillison.net",
15
   "username": "covidsewage"
16
  }
17
 }
18
}

Toot can support multiple authenticated users and let you switch between them, but for this bot we just need the one.

Taking the screenshot#

I’m using my shot-scraper tool to take the screenshot, like this:

1
shot-scraper https://covid19.sccgov.org/dashboard-wastewater \
2
  -s iframe \
3
  --wait 3000 \
4
  -b firefox \
5
  --retina \
6
  -o /tmp/covid.png

This loads the https://covid19.sccgov.org/dashboard-wastewater page in a headless (no visible window) Firefox instance. It waits 3 seconds, then takes a screenshot of JUST the first iframe on the page (-s means “selector”).

The screenshot is taken in retina mode (2x the pixel density) and saved to a file called /tmp/covid.png.

Read more about shot-scraper in shot-scraper: automated screenshots for documentation, built on Playwright.

Running this in GitHub Actions#

My entire bot is implemented as a GitHub Actions scheduled workflow.

The workflow runs once a day and does the following:

Installs its dependencies (shot-scraper and toot)
Takes the screenshot and writes it to a temporary file
Creates that ~/.config/toot/config.json file from a GitHub Actions secret
Uses toot post to post that screenshot to Mastodon
Writes this file back to the GitHub repository so I can see when it last ran.

I pasted the entire JSON authentication file into a new Actions secret for the repository called MASTODON_TOOT_CONFIG.

Here’s the workflow file:

1
name: Toot
2

3
on:
4
  # Run when I click "run workflow"
5
  # in the GitHub UI - for debugging
6
  workflow_dispatch:
7
  schedule:
8
  # Run at 14:13 UTC every day
9
  # which is 6:13am Pacific time
10
  - cron: '13 14 * * *'
11

12
jobs:
13
  scheduled:
14
    runs-on: ubuntu-latest
15
    steps:
16
    - name: Check out this repo
17
      uses: actions/checkout@v3
18
      with:
19
        fetch-depth: 0
20
    - name: Configure Python with pip cache
21
      uses: actions/setup-python@v3
22
      with:
23
        python-version: '3.10'
24
        cache: 'pip'
25
    # shot-scraper uses Playwright, which
26
    # needs to download a custom copy of
27
    # Firefox. We cache this here so it
28
    # isn't downloaded every time we run.
29
    - name: Cache Playwright browsers
30
      uses: actions/cache@v3
31
      with:
32
        path: ~/.cache/ms-playwright/
33
        key: ${{ runner.os }}-browsers
34
    - name: Install dependencies
35
      run: |
36
        pip install -r requirements.txt
37
    - name: Install Playwright browser
38
      run: |
39
        shot-scraper install -b firefox
40
    - name: Configure Git for commits
41
      run: |-
42
        git config user.name "Automated"
43
        git config user.email "actions@users.noreply.github.com"
44
    - name: Generate screenshot with shot-scraper
45
      # This screenshots the first iframe
46
      # on the page after a 3s wait
47
      run: |-
48
        shot-scraper https://covid19.sccgov.org/dashboard-wastewater \
49
          -s iframe --wait 3000 -b firefox --retina -o /tmp/covid.png
50
    - name: Toot the new image
51
      env:
52
        TOOT_CONFIG: ${{ secrets.MASTODON_TOOT_CONFIG }}
53
      # Write that JSON to the config file
54
      run: |-
55
        mkdir -p ~/.config/toot
56
        echo $TOOT_CONFIG > ~/.config/toot/config.json
57
        toot post "Latest Covid sewage charts for the SF Bay Area https://covid19.sccgov.org/dashboard-wastewater" \
58
          --media /tmp/covid.png --description "Screenshot of the latest Covid charts" > latest-toot.md
59
    # So we can see what it last did:
60
    - name: Commit latest-toot.md
61
      run: |-
62
        git add -A
63
        timestamp=$(date -u)
64
        git commit -m "${timestamp}" || exit 0
65
        git push

And the requirements.txt file:

1
shot-scraper
2
toot