Shell Quoting

The difference between the different types of quotes as used by bash (and similar sh like shells) seems to keep coming up while reviewing code. It seems like people mostly use the double quote (“) and just hope for the best, but this can be problematic, depending on the data that you’re working with. Let’s try some examples:

$ TEST="hi"
$ echo "$TEST"
hi
$

We can clearly see that the “$” is evaluated in this case, and the contents of “TEST” are printed. Cool, so this is the same as not having quotes at all, right?

$ echo $TEST
hi

It looks like yes so far, but let’s try a more interesting example. First, we need a script or tool that can show how arguments are getting processed.

#!/bin/bash

for i in `seq 1 $#`; do
        echo "$1"
        shift 1
done

This little script iterates over each of the arguments and prints them on a their own line. Let’s save it as “test_args.sh” and try it out

$ vim test_args.sh
$ chmod +x test_args.sh
$ ./test_args.sh a b c
a
b
c
$

Getting back to the original question, what if we use some quotes?

$ ./test_args.sh "a b c"
a b c
$

Now we can see that our string was processed as a single argument instead of three. The double quotes allow us to group something as a single argument, but still process certain special characters like “$” in order to dereference variables or do other processing.

What about the single quote (‘) ?

$ TEST="hi"
$ echo '$TEST'
$TEST
$

With the single quotes, our variable (“TEST”) wasn’t dereferenced and we get the exact string back.

What if we want to include the single quote (‘) in our data? We have a few options. Here are two:

$ echo "'"
'
$ echo $'\''
'

The $‘string’ syntax supports a handful of C like escape sequences. There’s a good list in the bash manpage and on the bash-hackers.org wiki

There’s also one other type of quote that we’ve already seen, but is worth pointing out: the backtick (`), which allows us to execute a string.

$ mkdir test
$ cd test
$ touch a
$ touch b
$ ls
a	b
$ TEST=`ls`
$ echo "$TEST"
a
b

Here we’ve assigned the variable “TEST” to the output of “ls”. Of course, we can also do something similar with different syntax. Reusing the example above:

$ unset TEST
$ TEST=$( ls )
$ echo "$TEST"
a
b

If you try to execute something this way that is derived from user input, be very careful to ensure that the input is properly sanitized and not a threat to your system. Code injection is one of the bigger categories of security vulnerability.

All straight-forward right? One more thing to try on your own before we call it a day:

echo {a,b,c}