The time has come to leave Hogwarts, young wizard! We wrap up our ten-part series on becoming a command-line wizard with a look at more utilities you should know.
The time utility shows system resource usage. Compare the linear search to a binary search:
$ time look -b gas
Note that a binary search matches the search string exactly as a prefix. To use the
-d (dictionary order) or
-f (case-insensitive) search options with a binary search, the file must be sorted in that order!
You can also create files to search later with look.
$ nice sort -t: -k1,1 invoices_raw > invoices
$ look 172-102i: invoices
nice reduces the scheduling priority of sort, reducing the impact that process has on other users and processes.
mail, sendmail, …
Want to send a quick email message? Years ago, everyone used mail to send and read plain-text messages. Now it’s half-forgotten but still useful for sending email. Also, if sendmail or another interface to the system mail transfer agent is available (and configured correctly), you can send email with almost-complete control over the message header and body.
Different versions of mail and sendmail support different options. Study and test to see what works on each system.
mail sends text from the keyboard or standard input. From the keyboard, end text with CTRL-D or a dot (period) on a line by itself. The
-s option specifies a subject:
$ mail email@example.com firstname.lastname@example.org
Subject: test message
$ myprog | mail -s 'myprog output' email@example.com
$ mail -s 'afile contents' firstname.lastname@example.org < afile
With most versions of mail, you don’t supply a message header. But sendmail sends a complete message with a full header and body. (Header and body are separated by exactly one newline character: an empty line.) The
-t option tells sendmail to read addresses from the message header. It’s a good idea to add the
-f option with a return address (the SMTP FROM, or envelope sender).
Before sending a message with sendmail, we’ll check it with
cat -e, which puts
$ at the end of each line. It shows that the line between the header and body is empty. sendmail typically isn’t in users’ shell search paths, so you may need to use its full pathname:
$ cat -e msg
From: Joe Smith <email@example.com>$
To: Ann Doe <firstname.lastname@example.org>,$
Subject: Test from sendmail$
$ /usr/sbin/sendmail -email@example.com -t < msg
A good way to send batches of messages with MIME-formatted bodies is with sendmail and a shell loop. Listing One shows the setup: a message and a list of addresses, one per line. The loop prepends a
To: address field to the stock message, then sends the message to that address with sendmail. A
sleep 3 command pauses three seconds after each message to avoid flooding the system mailer; this may not be needed:
$ cat msg
From: Joe Smith <firstname.lastname@example.org>
Subject: Please vote for me
Content-Type: multipart/mixed; ...
$ cat addrs
"Ann Doe" <email@example.com>
"Zoe Jones" <firstname.lastname@example.org>
$ cat addrs |
> while read addr
> echo "To: $addr" |
> cat - msg |
> /usr/sbin/sendmail -t -email@example.com
> sleep 3
There are tools available to MIME-encode messages, but one of the easiest ways is to send a message to yourself with a MIME-capable program like Thunderbird, save the message in a file, and clean up its header before re-sending it with sendmail.
od and friends
Being able to really see what’s in a file, byte by byte, can help you debug a lot of problems. The Swiss Army Knife of byte-by-byte viewing is od. Here’s an example: the first few bytes of a JPEG image file:
$ od -c -w6 clinica.jpg
0000000 377 330 377 340 \0 020
0000006 J F I F \0 001
0000014 001 001 \0 H \0 H
0000022 \0 \0 377 341 \0 026
0000030 E x i f \0 \0
-c option, od shows bytes as characters if they have ASCII representations; it shows escape sequences for others (like
\0 for NUL bytes) and octal values for the rest.
Linux has useful utilities other than od:
cat -vte encodes only “non-printable” characters and TABs, and (as we’ve seen earlier) it marks the ends of lines with
- The flexible pager less will ask “filename may be a binary file. See it anyway?”, then show hexadecimal values of “non-printable” data in reverse video.
- The strings utility searches files for text strings — for instance, finding filenames embedded in an executable binary.
There’s more about od and non-text files in “Bits and Pieces: Comparing Binary Data (and More)”
There’s a lot to say about these powerful networking utilities, and much of it has been said in other Linux Magazine columns over the years. “Transfer Tips, Part I,”, shows many command-line examples. The most important point to take away from that article is that ssh starts a shell on the remote system and that:
- It’s helpful to understand how wildcards and other shell operators are handled on both the local and remote systems,
- Your command line can redirect the standard input and standard output of either or both the local and remote shells,
- The remote shell has a context (shell and environment variables, a current directory that you can change with cd, and more).
Let’s close with an example: using tar to transfer a local directory tree to a different directory on a remote system:
$ tar czf - adir | ssh remhost 'cd subdir && tar xzvf -'
If that surprises you, a good look at ssh will be worth your time.
The ten articles in this “Wizard Boot Camp” series have covered a lot of ground: certainly not everything you need to know about Linux systems, but (we hope) a good starting point. Use your wizardly Linux powers well… and keep digging for more!