hexdump and hexdump -C#

While exploring null bytes in this issue I learned that the hexdump command on macOS (and presumably other Unix systems) has a confusing default output.

Consider the following:

1
$ echo -n 'abc\0' | hexdump
2
0000000 6261 0063
3
0000004

Compared to:

1
$ echo -n 'a' | hexdump
2
0000000 0061
3
0000001

I’m using echo -n here to avoid adding an extra newline, which encodes as 0a.

My shell hell is zsh - bash requires different treatment, see below.

How come abc\0 starts with 6261 where a starts with 0061?

It turns out hexdump default format is 16-bit words in little-endian format, which is really confusing.

hexdump -C#

Using the -C option fixes this:

1
$ echo -n 'a' | hexdump -C
2
00000000  61                                                |a|
3
00000001
4
$ echo -n 'abc\0' | hexdump -C
5
00000000  61 62 63 00                                       |abc.|
6
00000004

C here stands for “canonical”.

In addition to causing hexdump to output byte by byte, it also includes an ASCII representation on the right hand side.

Null bytes in Bash#

Karl Pettersson pointed out that these examples won’t work on Bash.

I ran bash on my Mac and found the following:

1
bash-3.2$ echo -n 'abc\0' | hexdump -C
2
00000000  61 62 63 5c 30                                    |abc\0|
3
00000005
4
bash-3.2$ echo -n $'abc\0' | hexdump -C
5
00000000  61 62 63                                          |abc|
6
00000003
7
bash-3.2$ printf 'abc\0' | hexdump -C
8
00000000  61 62 63 00                                       |abc.|
9
00000004
10
bash-3.2$ printf $'abc\0' | hexdump -C
11
00000000  61 62 63                                          |abc|
12
00000003

So it looks like using printf 'abc\0' is the best recipe for Bash on macOS. I’m not sure if Bash on other platforms differs.

Bill Mill suggested echo -ne for this:

1
bash-3.2$ echo -ne 'abc\0' | hexdump -C
2
00000000  61 62 63 00                                       |abc.|
3
00000004

The -e option enables the interpretation of backslash escapes.