dcsimg

Wizard Boot Camp, Part 10: Utilities You Should Know

The time has come to leave Hogwarts, young wizard! We wrap up our ten-part series on becoming a command-line wizard with a look at more utilities you should know.

The time utility shows system resource usage. Compare the linear search to a binary search:

$ time look -b gas
gas
gas's
...

real    0m0.004s
user    0m0.001s
sys     0m0.003s

Note that a binary search matches the search string exactly as a prefix. To use the -d (dictionary order) or -f (case-insensitive) search options with a binary search, the file must be sorted in that order!

You can also create files to search later with look.

$ nice sort -t: -k1,1 invoices_raw > invoices
  ...later...
$ look 172-102i: invoices
172-102i:2008-05-17:Smith, Alan:...

nice reduces the scheduling priority of sort, reducing the impact that process has on other users and processes.

Other uses for the system word list

The system word list, often in the directory /usr/share/dict, can be searched with more than look. Need an eight-character word starting with n and a vowel, and ending in ly?

$ cd /usr/share/dict
$ grep -i '^n[aeiou]....ly$' words
narrowly
normally

mail, sendmail, …

Want to send a quick email message? Years ago, everyone used mail to send and read plain-text messages. Now it’s half-forgotten but still useful for sending email. Also, if sendmail or another interface to the system mail transfer agent is available (and configured correctly), you can send email with almost-complete control over the message header and body.

Different versions of mail and sendmail support different options. Study and test to see what works on each system.

mail sends text from the keyboard or standard input. From the keyboard, end text with CTRL-D or a dot (period) on a line by itself. The -s option specifies a subject:

$ mail joe@foo.xyz ann@bar.xyz
Subject: test message
test test
that's all
.
$ myprog | mail -s 'myprog output' joe@foo.xyz
$ mail -s 'afile contents' ann@bar.xyz < afile

With most versions of mail, you don’t supply a message header. But sendmail sends a complete message with a full header and body. (Header and body are separated by exactly one newline character: an empty line.) The -t option tells sendmail to read addresses from the message header. It’s a good idea to add the -f option with a return address (the SMTP FROM, or envelope sender).

Before sending a message with sendmail, we’ll check it with cat -e, which puts $ at the end of each line. It shows that the line between the header and body is empty. sendmail typically isn’t in users’ shell search paths, so you may need to use its full pathname:

$ cat -e msg
From: Joe Smith <joe@foo.xyz>$
To: Ann Doe <ann@bar.xyz>,$
  zoe@plum.xyz, ron@kumquat.xyz$
Subject: Test from sendmail$
$
test test$
$ /usr/sbin/sendmail -fjoe@foo.xyz -t < msg

A good way to send batches of messages with MIME-formatted bodies is with sendmail and a shell loop. Listing One shows the setup: a message and a list of addresses, one per line. The loop prepends a To: address field to the stock message, then sends the message to that address with sendmail. A sleep 3 command pauses three seconds after each message to avoid flooding the system mailer; this may not be needed:

$ cat msg
From: Joe Smith <joe@foo.xyz>
Subject: Please vote for me
MIME-Version: 1.0
Content-Type: multipart/mixed; ...
...
$ cat addrs
"Ann Doe" <ann@bar.xyz>
"Zoe Jones" <zoe@plum.xyz>
...
$ cat addrs |
> while read addr
> do
>   echo "To: $addr" |
>   cat - msg |
>   /usr/sbin/sendmail -t -fjoe@foo.xyz
>   sleep 3
> done

There are tools available to MIME-encode messages, but one of the easiest ways is to send a message to yourself with a MIME-capable program like Thunderbird, save the message in a file, and clean up its header before re-sending it with sendmail.

od and friends

Being able to really see what’s in a file, byte by byte, can help you debug a lot of problems. The Swiss Army Knife of byte-by-byte viewing is od. Here’s an example: the first few bytes of a JPEG image file:

$ od -c -w6 clinica.jpg
0000000 377 330 377 340  \0 020
0000006   J   F   I   F  \0 001
0000014 001 001  \0   H  \0   H
0000022  \0  \0 377 341  \0 026
0000030   E   x   i   f  \0  \0
...

With its -c option, od shows bytes as characters if they have ASCII representations; it shows escape sequences for others (like \0 for NUL bytes) and octal values for the rest.

Linux has useful utilities other than od:

  • cat -vte encodes only “non-printable” characters and TABs, and (as we’ve seen earlier) it marks the ends of lines with $.
  • The flexible pager less will ask “filename may be a binary file. See it anyway?”, then show hexadecimal values of “non-printable” data in reverse video.
  • The strings utility searches files for text strings — for instance, finding filenames embedded in an executable binary.

There’s more about od and non-text files in “Bits and Pieces: Comparing Binary Data (and More)”

ssh/scp

There’s a lot to say about these powerful networking utilities, and much of it has been said in other Linux Magazine columns over the years. “Transfer Tips, Part I,”, shows many command-line examples. The most important point to take away from that article is that ssh starts a shell on the remote system and that:

  • It’s helpful to understand how wildcards and other shell operators are handled on both the local and remote systems,
  • Your command line can redirect the standard input and standard output of either or both the local and remote shells,
  • The remote shell has a context (shell and environment variables, a current directory that you can change with cd, and more).

Let’s close with an example: using tar to transfer a local directory tree to a different directory on a remote system:

$ tar czf - adir | ssh remhost 'cd subdir && tar xzvf -'

If that surprises you, a good look at ssh will be worth your time.

In closing

The ten articles in this “Wizard Boot Camp” series have covered a lot of ground: certainly not everything you need to know about Linux systems, but (we hope) a good starting point. Use your wizardly Linux powers well… and keep digging for more!

Comments on "Wizard Boot Camp, Part 10: Utilities You Should Know"

Hi fantastic blog! Does runnikng a blog such as this require a great deal oof work?
I’ve no expertise in programming but I hadd been hoping to start my own blog in the near
future. Anyhow, if you have any idess or techniques for new blog
owners please share. I know this is off topic howevdr I just wanted to ask.
Cheers!

My blog :: cheap car insurance

It’s amazing for me to have a web site, which is beneficial in support oof my knowledge.

thanks admin

Here is my weblog; cheap car insurance

Excellent web site you have got here.. It’s hard too find high quality writing llike yours thesze
days. I seriously appreciate individuals like you!

Takee care!!

my web site … Cheap car insurance

My coder is trfying to convince me to move to .net from PHP.
I have always disliked the idea because oof the costs.But he’s tryuong none
thhe less. I’ve been using WordPress on various websites for ahout a year
and am anxious about switchig to another platform.
I have heard good things about blogengine.net.Is there a way I can import all my wordpress content inyo it?

Any kind of help would be greatly appreciated!

Look at my web blig – cheap car insurance in ohio

Does your site hhave a contact page? I’m havin a tough time locating it
but, I’d like tto senhd you an e-mail. I’ve
got some ideas for your blog you might be interested in hearing.
Either way, greeat blog and I look forward to seeing it
grow over time.

Visit my weeb blog – cheap car insurance

Thanks ffor another informative web site.
The place else may just I am getting thqt kind of information written in such a perfect manner?
I’ve a challenge that I’m simply now running on, and I’ve been at the look
out for such info.

Also visit my webpage Cheap Car Insurance

I am extremely inspired along with your writing abilities and
also with the format to your weblog. Is that this a paid subject or did you modify it yourself?
Either way stay up the nice high quality writing, it is uncommon to look a nice blog like this one today..

Also visit my blog: cheap car insurance

Heya i’m for the first time here. I found this board and I find It really useful & it helped me out a lot.
I hope to give something back and help others like you helped
me.

Here is my page :: cheap car insurance

Very good blog article.Really thank you! Cool.

Very neat post. Really Cool.

Check beneath, are some entirely unrelated web-sites to ours, even so, they’re most trustworthy sources that we use.

Very few websites that come about to be detailed below, from our point of view are undoubtedly effectively really worth checking out.

Here are some of the web sites we suggest for our visitors.

Every after in a although we pick out blogs that we read. Listed below are the most up-to-date web-sites that we opt for.

This blog w?s… how ddo I ?ay ?t? Relevant!!

F?nally I’v? found s?mething tha helped me.
?hank y?u!

Feel free to visit m? website – adult swinging

Leave a Reply