Accurate Quoting

To quote, or not to quote? The topic of quoting is a command-line mystery that confounds many new users. When do you use quotes, and when do you not use quotes? For instance, what’s the difference, if any, between these commands?

rm *

and

rm ‘*’

And then there’s the always-vexing issue of what type of quoting to use — single quoting, double quoting, or back quoting? If these are the kind of questions you’ve been asking yourself, then you’ve come to the right place. This article aims at setting straight the various Linux quoting types so that your quotes will be dead-on accurate.

To be clear, the material in this article and other articles in this series applies to the BASH shell, which is the most popular Linux shell. There are many other shells available, but covering their specific uses and functionality is beyond the scope of this article. If you use a different shell, the material will apply only to the extent that the behavior of your chosen shell conforms to that of the BASH shell. To see which shell you’re using, issue the command echo$SHELL. (It’s probably BASH.)

To Quote or Not to Quote

Quoting becomes an issue primarily because of the command-line feature known as filename globbing. Filename globbing lets you specify command arguments that refer to sets of files without having to individually list each filename. The similar, but much less sophisticated, MS-DOS feature is known as filename wildcards.

For example, by using filename globbing, you can delete all of the files in the current working directory by typing the single command rm *. The metacharacter * is one of many that can be used to build “regular expressions.” If you need a refresher on regular expressions, please check out the January 2001 Newbies column on the Web at http://www.linux-mag.com/2001-01/newbies_01.html.

Filename globbing is generally, but not always, convenient. For instance, suppose you’ve created a file named $PATH. You might do so by inadvertently typing:

>’$PATH’

If you try to delete the unwanted file by tying the command:

rm $PATH

you’re in for a surprise. As it happens, $PATH is an environment variable, and the shell interprets your command as though you’d typed something like:

rm /usr/bin:/bin:/usr/bin:
   /usr/X11R6/bin:/usr/local
   /bin:/home/bmccarty/bin

This command certainly won’t delete the file named $PATH. Can you guess how we will get rid of this unwanted file?

Opacity: Single Quotes

The shell gives you the choice of three kinds of quotes: single quotes, double quotes, and back quotes. Let’s examine each kind of quote in turn, beginning with the simplest type, single quotes. Essentially, single quotes turn off filename globbing of the text they contain. In other words, they protect their contents from filename globbing. Another way of looking at the function of single quotes is that they provide opacity; they render their contents opaque, or invisible, to the shell.

For instance, single quotes enable you to delete errantly named files such as $PATH. You can do so by using a command such as:

rm ‘$PATH’

Because of the single quotes, the shell doesn’t recognize $PATH as the name of an environment variable. Instead, it takes the expression as the literal name of a file and deletes the file.

Semi-Opacity: Double Quotes

Double quotes are a weaker form of quoting than single quotes. You can think of them as semi-opaque rather than opaque; their contents are partly visible to the shell. For example, the command:

rm “$PATH”

works like the command:

rm $PATH

In this case, the shell expands the value of the environment variable $PATH and therefore is unable to delete a file that has the same name as the environment variable.

So far, double quotes seem quite transparent. But, consider the following command:

rm “*”

Here, the double quotes protect the * from filename globbing. Rather than attempting to delete every file in the current working directory, this command attempts to find and delete a file named *. So, double quotes are not entirely transparent.

Mirrors: Back Quotes

The third kind of quoting, back quoting, also happens to be the most interesting. When the shell encounters text surrounded by back quotes, it executes the text as a command and then substitutes the result for the back quoted text. This is simple to say, but perhaps difficult to understand, so let’s look at an example.

Suppose you have a file named list that contains one line of text:

1 2 3

Now suppose that you execute the command:

cat list

The output of the command is the contents of the file:

1 2 3

You can use this to your advantage. Suppose you have a list of files that you want to delete. Enter the list into a file named list and execute the following command:

rm `cat list`

When the shell executes this command, it replaces the text ‘cat list’ with the contents of the file. So, the command operates as though you had typed rm followed by the names of the files you want to delete. For example, suppose the file named list contains the line:

1 2 3

The command rm `catlist` operates as though you’d typed:

rm 1 2 3

Try accomplishing that with some puny, point-and-click user interface!

Recall that double quotes were termed “semi-opaque.” Another reason for seeing them as such is that the shell will peek inside double-quoted text, looking for back-quoted text. If it finds any back-quoted text, the shell will execute the text as a command despite the presence of the double quotes.

When Quotes Don’t Break Free: Escapes

Another way to protect text from the shell is to escape the text. To do so, put a backslash in front of each character you want to protect. For example, the command:

rm *

will not delete every file in the current working directory. The command is interested only in the file named *.

One handy use of the backslash escape is to incorporate metacharacters, particularly quotes, inside output text. For example, to output the text:

The value of $PATH is
   “/usr/bin:/bin:/usr/bin:
   /usr/X11R6/bin:/usr/local
   /bin:/home/bmccarty/bin”

you could use a command such as this:

echo “The value of $PATH
   is ”$PATH”.”

By escaping the $, you prevent the shell from seeing $PATH as the name of an environment variable. And, by escaping the inner double quotes, you prevent the shell from confusing them with the double quotes that surround the text. Table One recapitulates Linux’s quoting and escaping features.

Strategic Teamwork: Combining Quotes with Other Shell Facilities

In journalism (I’m told), a writer has skillfully used quotes when they are woven into a strong narrative. The same is true in Linux; quotes can be powerfully used in combination with other elements, such as redirection. (If you are not familiar with redirection, brush up on it by checking out the December 2000 Tech Support column on the Web at http://www.linux-mag.com/2000-12/tech_support_01.html.) Let’s consider some examples.

First, let’s take the killall command. This command sends a specified signal, often a termination signal, to every process executing a specified command. On most Linux systems, killall is implemented as a binary executable. However, let’s see how we might implement a similar facility using only the shell.

Let’s begin with the ps command, which lists processes. By using the -C flag, we can cause the ps command to list all processes running a specified command. For example, the command shown here:

ps -C bash

lists all processes running the BASH shell. The output of the command resembles the following:

PID   TTY     TIME     CMD
1989  pts/41  00:00:00  bash
2138  pts/54  00:00:00  bash
2338  pts/76  00:00:00  bash

To send a signal to a process, we need only the process ID, which appears in the first column. Let’s use another flag to suppress the unneeded output:

ps -C bash -o pid

Now the output looks like:

PID
1989
2138
2338

To learn more about these and other flags associated with the ps command, view the command’s man page by issuing the command man ps.

The kill command sends a signal to a process. By default, the signal sent is a termination signal. By using kill, ps, and some quotes, we can create our own simplified version of the killall command. Place the following line in a file named mykillall:

kill ‘ps -C $1 -o pid’

Keep in mind that $1 stands for the first argument passed to the mykillall command.

Now, suppose that many unwanted copies of the process named test have somehow clogged your system. To kill them, issue the command:

sh mykillall test

from the directory containing the file mykillall.

You can test your workmanship by creating several processes running the sleep command. To do so, first be sure you’re not logged in as root; otherwise, you may disturb important system processes. Next, issue the following command several times:

sleep 60 &

To see the unwanted processes, you can issue the command ps; you should see something like:

ps
PID   TTY     TIME      CMD
2539  pts/23  00:00:00  bash
3071  pts/23  00:00:00  sleep
3072  pts/23  00:00:00  sleep
3073  pts/23  00:00:00  sleep
3074  pts/23  00:00:00  ps

To kill the processes, issue the following command:

sh mykillall sleep

You should see output that resembles the following:

sh mykillall sleep
kill: No such pid PID
[1]  Terminated      sleep 60
[2]- Terminated      sleep 60
[3]+ Terminated      sleep 60

The first line of output is due to the heading line output by the ps command. The kill command stupidly attempts to delete a process named PID. Using more advanced shell techniques, we could eliminate this superfluous message; for the sake of simplicity, we simply ignore the message. You might like to investigate this issue on your own. If so, study the –no-headers option of the ps command (which is the easy way) or study the tail command (which is the harder, but more general way. See the tail man page for more on this).

To demonstrate that the sleep processes have been terminated, issue the ps command:

ps -C sleep

The command should not report any sleep processes.

Splitting File Names

Abraham Lincoln, America’s 16th president, was known as the “Illinois rail splitter” because of his prowess in splitting logs. It’s unclear if his frontier skills would be up to the challenges of splitting Linux file names; however, you can do so easily with the aid of back quotes and the commands dirname and basename.

The dirname command prints all but the last component of the file name that is specified as its argument. For example:

dirname /usr/bin/test

prints /usr/bin. The basename command prints the last component of the file name specified as its argument. For example:

basename /usr/bin/test

prints test. You can think of these two commands as opposite sides of a coin. The basenamecommand has an additional capability related to a second argument, which specifies a file name extension. This extension is removed from the result of the command before the result is printed. For example:

basename/user/bin/test.txt .txt

prints only test, not test.txt.

Now, consider the following script built using these commands:

cd ‘dirname $1′
cp $1 ‘basename $1 .txt’.old
$EDITOR $1

Suppose that the environment variable EDITOR has the value vi, and you execute this script with an argument such as /home/mccarty/list.txt. The commands executed are actually:

cd /home/mccarty
cp /home/mccarty/list.txt /home/mccarty/list.old
vi /home/bmccarty/list.txt

That is, the command takes you to the directory that contains the specified file (/home/mccarty), creates a backup copy of the file (list.old), and launches the vi editor on the file.

You can probably devise endless variations on this theme, some of which you’ll likely find useful in your daily work. Go on, have it your way; that’s what Linux is about. If you want to try this script, be sure to define and export a value for the EDITOR environment variable. Here’s one way to do so:

export EDITOR=vi

Quote Away

Now that you have the quoting rules of the Linux shell at your disposal, you should never again ignorantly misquote either your own words or those of another shell programmer.

Next month, we’ll look at another issue of shell use, continuing our quest toward shell mastery. Until then, happy quoting!

Bill McCarty is an associate professor at Azusa Pacific University. He can be reached at bmccarty@apu.edu.