A shell script for running Go one-liners#

bitfield/script is a really neat Go project: it tries to emulate shell scripting using Go chaining primitives, so you can run code like this:

1
script.Stdin().Column(1).Freq().First(10).Stdout()

To achieve the same thing as:

1
cat file.txt | cut -f1 | sort | uniq -c | sort -nr | head -10

A comment from jvictor118 on Hacker News:

If one were actually going to use something like this, I’d think it’d be worth implementing a little shebang script that can wrap a single-file script in the necessary boilerplate and call go run!

This is exactly the kind of thing I can’t quite be bothered to write myself, but I’m happy to coach GPT-4 through building.

The result: goscript.sh. You can use it like this:

1
cat file.txt | ./goscript.sh -c 'script.Stdin().Column(1).Freq().First(10).Stdout()'

Or you can create a script file like this one, saved as top10.goscript:

1
script.Stdin().Column(1).Freq().First(10).Stdout()

And run:

1
cat file.txt | ./goscript.sh top10.goscript

Finally, you can set the shebang line in a script file like this:

1
#!/tmp/goscript.sh
2
script.Stdin().Column(1).Freq().First(10).Stdout()

Then run this:

1
chmod 755 top10.goscript
2
cat file.txt | ./top10.goscript

The script#

Here’s the goscript.sh script that GPT-4 and I came up with:

1
#!/bin/bash
2

3
TMPDIR=$(mktemp -d /tmp/goscript.XXXXXX)
4
SUBDIR="$TMPDIR/goscript_inner"
5
mkdir -p $SUBDIR
6
trap "rm -rf $TMPDIR" EXIT
7

8
TMPFILE="$SUBDIR/script.go"
9

10
# Write boilerplate to tmpfile
11
cat > $TMPFILE <<EOF
12
package main
13

14
import (
15
    "github.com/bitfield/script"
16
)
17

18
func main() {
19
EOF
20

21
# Check for -c flag
22
if [ "$1" == "-c" ]; then
23
    # Add the literal string from argument
24
    echo "$2" >> $TMPFILE
25
else
26
    # Add user's code from file
27
    sed '/^#!/d' "$1" >> $TMPFILE
28
fi
29

30
# Close main function
31
echo "}" >> $TMPFILE
32

33
# Initialize a new module in subdir, fetch dependencies, and run
34
pushd $SUBDIR > /dev/null 2>&1
35
go mod init tmp > /dev/null 2>&1
36
go get github.com/bitfield/script > /dev/null 2>&1
37
go run script.go
38
popd > /dev/null 2>&1

And the full ChatGPT transcript that lead to the final script presented here.

(Missing from that transcript is the final step where we added the sed line to strip out the shebang.)

Here’s what I learned from the above code.

The program itself is wrapped in the following boilerplate, using >> to write to the temporary file:

1
package main
2

3
import (
4
    "github.com/bitfield/script"
5
)
6

7
func main() {
8
    // User's code goes here
9
}

With modern Go you need to use the following pattern to get something like this to work with a dependency:

1
go mod init tmp
2
go get github.com/bitfield/script
3
go run script.go

go get downloads the dependency, using a cache if it’s already been downloaded.

The shell script runs all of that in a temporary directory, created using:

1
TMPDIR=$(mktemp -d /tmp/goscript.XXXXXX)
2
SUBDIR="$TMPDIR/goscript_inner"
3
mkdir -p $SUBDIR

That mktemp -d /tmp/goscript.XXXXXX line uses the templating feature of mktemp, where a sequence of XXX is replaced by random characters.

The trap call is interesting - see also Running multiple servers in a single Bash script. Effectively it ensures the temporary directory is deleted when the script terminates, no matter why it terminates (success or error):

1
trap "rm -rf $TMPDIR" EXIT

I wanted to support two ways of calling the script:

1
./goscript.sh -c 'go code here'
2
./goscript.sh script.goscript

That’s handled by this conditional check:

1
if [ "$1" == "-c" ]; then
2
    # Add the literal string from argument
3
    echo "$2" >> $TMPFILE
4
else
5
    # Add user's code from file
6
    sed '/^#!/d' "$1" >> $TMPFILE
7
fi

The sed line is necessary because if you have a script that looks like this:

1
#!/tmp/goscript.sh
2
script.Stdin().Column(1).Freq().First(10).Stdout()

That first line will be copied into the Go code in a way that breaks syntax. Using sed here strips that line out before copying the rest of the file into the main() function in the boilerplate.

With this accounted for, running ./top10.goscript effectively runs the same as calling ./goscript.sh top10.goscript.

Several of the commands in the script output information to stdout or stderr - we fixed that with this pattern:

1
go mod init tmp > /dev/null 2>&1

(It feels weird to refer to the combination of myself and GPT-4 as “we”, but I think it’s an honest description of the we we collaborated to build this.)