Start, test, then stop a localhost web server in a Bash script#

I wanted to write a bash script that would start a Datasette server running, run a request against it using curl, then stop the server again.

It should then return an exit code indicating if the curl request was succesful or not.

Research notes in this issue - here’s the final script I came up with:

1
#!/bin/bash
2

3
# Generate certificates
4
python -m trustme
5
# This creates server.pem, server.key, client.pem
6

7
# Start the server in the background
8
datasette --memory \
9
    --ssl-keyfile=server.key \
10
    --ssl-certfile=server.pem \
11
    -p 8152 &
12

13
# Store the background process ID in a variable
14
server_pid=$!
15

16
# Wait for the server to start
17
sleep 2
18

19
# Make a test request using curl
20
curl -f --cacert client.pem 'https://localhost:8152/_memory.json'
21

22
# Save curl's exit code (-f option causes it to return one on HTTP errors)
23
curl_exit_code=$?
24

25
# Shut down the server
26
kill $server_pid
27
sleep 1
28

29
# Clean up the certificates
30
rm server.pem server.key client.pem
31

32
echo $curl_exit_code
33
exit $curl_exit_code

There are a few additional tricks in this - it’s running python -m trustme to create self-signed certificates in the current directory which are used for the test - but the key parts are these:

datasette ... & - starts Datasette running as a background process
server_pid=$! - records the PID of the server we just started so we can shut it down later
curl -f ... - makes a curl request, but returns an exit code that indicates if the request succeeded or was a 404 or 500 error or similar
curl_exit_code=$? - records that exit code for later
kill $server_id - causes the server to exit - I then did a sleep 1 to provide time for it to output its shutdown text to the terminal
exit $curl_exit_code - the exit code for the Bash script is now the same as the exit code returned by curl

I’m running this script in CI in GitHub Actions.

An improved version by Jan Lehnardt#

Jan Lehnardt submitted a pull request with this improved version:

1
#!/bin/bash
2

3
# Generate certificates
4
python -m trustme
5
# This creates server.pem, server.key, client.pem
6

7
cleanup () {
8
    rm server.pem server.key client.pem
9
}
10

11
# Start the server in the background
12
datasette --memory \
13
    --ssl-keyfile=server.key \
14
    --ssl-certfile=server.pem \
15
    -p 8152 &
16

17
# Store the background process ID in a variable
18
server_pid=$!
19

20
test_url='https://localhost:8152/_memory.json'
21

22
# Wait for the server to start
23

24
# h/t https://github.com/pouchdb/pouchdb/blob/25db22fb0ff025b8d2c698da30c6c409066baa0c/bin/run-test.sh#L102-L113
25
waiting=0
26
until $(curl --output /dev/null --silent --insecure --head --fail --max-time 2 $test_url); do
27
    if [ $waiting -eq 4 ]; then
28
        echo "$test_url can not be reached, server failed to start"
29
        cleanup
30
        exit 1
31
    fi
32
    let waiting=waiting+1
33
    sleep 1
34
done
35

36
# Make a test request using curl
37
curl -f --cacert client.pem $test_url
38

39
# Save curl's exit code (-f option causes it to return one on HTTP errors)
40
curl_exit_code=$?
41

42
# Shut down the server
43
kill $server_pid
44
waiting=0
45
#         show all pids
46
#         |       find just the $server_pid
47
#         |       |                  don’t match on the previous grep
48
#         |       |                  |            we don’t need the output
49
#         |       |                  |            |
50
until ( ! ps ax | grep $server_pid | grep -v grep > /dev/null ); do
51
    if [ $waiting -eq 4 ]; then
52
        echo "$server_pid does still exist, server failed to stop"
53
        cleanup
54
        exit 1
55
    fi
56
    let waiting=waiting+1
57
    sleep 1
58
done
59

60
# Clean up the certificates
61
cleanup
62

63
echo $curl_exit_code
64
exit $curl_exit_code

There’s not much extra commentary I can add to this - Jan’s inline comments are fantastic!

I really like the waiting=0 pattern here for retrying up to 4 times.

Worth breaking down the curl command here a bit:

1
curl --output /dev/null --silent --insecure --head --fail --max-time 2 $test_url

It’s avoiding any output at all with a combination of writing output to /dev/null and using --silent to turn off logging.

It uses --insecure because our server is running with a self-signed certificate, which will produce errors without this option - and here we just want to poll until the server is up and running.

--max-time ensures each poll waits a maximum of two seconds.

And as before, --fail causes curl to return an exit code that indicates if the request was successful or not.

Jan used this as the impetus to start a new library of shell utility functions.