Using awk to add a prefix#

I wanted to dynamically run the following command against all files in a directory:

1
pypi-to-sqlite content.db -f /tmp/pypi-datasette-packages/packages/airtable-export.json \
2
-f /tmp/pypi-datasette-packages/packages/csv-diff.json \
3
--prefix pypi_

I can’t use /tmp/pypi-datasette-packages/packages/*.json here because I need each file to be processed using the -f option.

I found a solution using awk. The awk program '{print "-f "$0}' adds a prefix to the input, for example:

1
% echo "blah" | awk '{print "-f "$0}'
2
-f blah

I wanted that trailing backslash too, so I used this:

1
{print "-f "$0 " \\"}

Piping to awk works, so I combined that with ls ../*.json like so:

1
% ls /tmp/pypi-datasette-packages/packages/*.json | awk '{print "-f "$0 " \\"}'
2
-f /tmp/pypi-datasette-packages/packages/airtable-export.json \
3
-f /tmp/pypi-datasette-packages/packages/csv-diff.json \
4
-f /tmp/pypi-datasette-packages/packages/csvs-to-sqlite.json \

Then I used eval to execute the command. The full recipe looks like this:

1
args=$(ls /tmp/pypi-datasette-packages/packages/*.json | awk '{print "-f "$0 " \\"}')
2
eval "pypi-to-sqlite content.db $args
3
--prefix pypi_"

Full details in datasette.io issue 98.