Making sure that users' electronic mail gets sent out and delivered is one of the system administrator's most important jobs, and it's also one that becomes extremely visible should things go wrong. Inevitably, administering e-mail is time-consuming and frustrating, at least intermittently.
Making sure that users’ electronic mail gets sent out and delivered is one of the system administrator’s most important jobs, and it’s also one that becomes extremely visible should things go wrong. Inevitably, administering e-mail is time-consuming and frustrating, at least intermittently.
This column begins a three-part look at some key aspects of the usual Linux electronic mail system. This month, we will begin with an introduction to how electronic mail gets created and look at some of the complexities involved in mail addressing. In future months, we will take a look at the key component responsible for most aspects of mail transportation and discuss how electronic mail may be filtered.
Parts of the Mail System
As with regular postal mail, a fully functioning electronic mail system depends on a series of distinct, and often geographically separated, facilities and processes working together (or at least with mutual cooperation). A mail system is made up of the following types of components:
- Programs that allow users to read mail and submit new messages — such programs are known as “user agents.” There are a variety of such mail programs available under Linux, ranging from the traditional (and primitive) mail command to character-based, menu-driven programs such as elm, mutt, and pine to Internet-integrated graphical packages like Netscape. In addition, some users prefer the mail facilities embedded within their favorite editor (such as emacs). In any case, user agents do not typically require a lot of administrative time or attention.
- Programs, typically run as daemons, that accept outgoing e-mail (“submission agents”), send it on its way, and begin the process of delivering it to its final destination (system and user). The latter functions are the responsibility of mail “transport agents.” sendmail is the traditional Unix transport agent and frequently functions as a submission agent as well, although some user agents now incorporate this capability themselves. Current estimates indicate that about 75 percent of all e-mail is transported by sendmail. Other transport agents include Postfix, qmail, and smail. Transport agents use the Simple Mail Transfer Protocol (SMTP) to exchange data.
- A program that actually places the message into the appropriate user’s mailbox. Once mail arrives on its destination system, the transport agent hands it off to a “delivery agent.” There may be separate delivery agents for different types of messages (e.g., local vs. remote) and different transport protocols (e.g., SMTP vs. UUCP).
- The final destination of a mail message from the point of view of the transport agent may not be the actual location from which the user will access it. There are two situations in which this occurs. The first is when a user, organization, or entire site has only an intermittent connection to the Internet. Messages will be stored for them on their ISP’s site until they are ready to collect them. A “retrieval agent” will then periodically establish a connection to the ISP to send new messages and collect those that are waiting. The most common of these programs is fetchmail.
The second situation is when a user accesses e-mail on a different computer system from the one where the official mailbox is located (the target location for the delivery agents). The storage location is known as a “message store.” The user agent must then connect to the message store in order to view, access, manipulate, and potentially download the messages, using the Post Office Protocol (POP3) or Internet Message Access Protocol (IMAP) for communication.
|Figure One: How an e-mail from Hamlet makes its way to Ophelia’s computer via the Internet.|
Figure One illustrates some of these components and concepts using a sample mail message sent from a user named Hamlet at uwitt.edu to his friend Ophelia at firstname.lastname@example.org.
Hamlet composes his message to Ophelia using a mailer program like pine or mutt on one of the common workstations in his department (hostname philo). Depending on his user agent and its exact configuration, it will either forward the message to the local sendmail process using port 587 (allowing sendmail to submit it to the mail facility), or it may do the submission operation itself, communicating with sendmail via SMTP on port 25 (the transport agent standard port). In our example, pine has been configured to function as a submission agent as well as a user agent, while mutt relies on sendmail for mail submission.
At this site, all outgoing mail is funneled through a single mail relay host named apollo, The sendmail process on philo passes the message along to the corresponding process on apollo, which routes it to the Internet. From there, the message will eventually be sent to ophe624@ elsinore.gov, which is diverted (via DNS MX records) to some system at the ISP used by the elsinore.gov site.
When convenient, the incoming mail server at elsinore.gov (poste) connects to the ISP and uses the fetchmail program to retrieve waiting messages. fetchmail forwards the data to sendmail using the SMTP protocol and port 25, thereby simulating normal incoming TCP/IP mail. The sendmail process on poste sends the messages for user ophe624 to the sendmail process on polonius, where the procmail program places it in the correct mailbox on host polonius (/var/spool/mail/ ophe624).
From the point of view of the sendmail transport agent, the mail is now delivered. However, Ophelia still has not seen the message. She reads her mail on her laptop. To do so, she has configured the mail program in Netscape to connect to the message store (her mailbox) on polonius, providing the appropriate username and password for authentication. Netscape will then display information about the messages in her mailbox, showing her the actual message when requested, retrieving all data via the IMAP protocol. Ophelia can delete the message, download it to her laptop, or file it away into one of her mail folders on polonius (or even leave it in her incoming mailbox).
If the site elsinore.gov had a direct Internet connection, the initial delivery of mail message to their site would be somewhat different. Instead of using fetchmail to retrieve messages from a remote ISP site, mail would arrive at the site’s firewall. In this case, a daemon that forwarded SMTP packets to a designated host inside the firewall (poste could again be used for the latter) would run on the firewall. Such a daemon is known as an SMTP proxy; the most widely used SMTP proxy facility is the combination of smtpd (to receive and store incoming SMTP data) and smtpfwdd (to forward SMTP data to the incoming mail server) from www.obtuse.com.
Mail System Components Available for Linux Systems
User Agents: mail, elm, mutt, pine*, mh/nmh*, Netscape*, emacs’ rmail (Starred items can also function as submission agents.)
Submission/Transport Agents: sendmail, Postfix, smail, qmail, exim.
Delivery Agents: mail, deliver, mail.local (part of the sendmail package), procmail, uux with rmail (for UUCP-transported mail) — Mail piped to programs is delivered by /bin/sh (bash actually) or smrsh (part of the sendmail package).
Access Agents: Any mailer program supporting the POP3 or IMAP protocol (these include pine, mutt, mh/nmh, and Netscape) — When the message store is kept on a Linux computer system, it will need to run the appropriate daemons; these may be found at http://www.washington.edu/imap. Some Linux distributions also include IMAP and POP daemons in a package with a name beginning with pop (followed by a version number string).
Retrieval Agent: fetchmail is the program of choice for these functions.
Mail Address Subtleties
So far, we have considered only the most straightforward mail-addressing scenario — a message is addressed to a user at a particular site. However, some complications can arise:
- DNS MX records can redirect messages directed at some host to a different host.
- E-mail aliases can redirect incoming messages for a user to a different host and/or user (or even group of users).
- Mail-forwarding mechanisms can also redirect mail to a different destination address (typically used for users who are away from their home site for an extended period or have left an organization altogether).
We will consider each of these in turn.
DNS MX records specify the host that handles e-mail for a designated computer system. They cause e-mail to that host to be preempted and rerouted to the new destination system. MX records have the general format:
host [ttl] IN MX n destination
where host is the computer to which the record applies, n is a number indicating the record priority (lower numbers indicate higher priority), and destination is the name of the host to which mail should be redirected (note that it can be the same as the host itself). (ttl is the usual optional time-to-live parameter applying to DNS caching. In case you need it, you can find a more general overview of DNS on our Web site at http://www.linux-mag.com/2000-09/dns_01.html.) Figure Two shows some examples of this.
Figure Two: Sample MX Records
dalton IN MX 10 dalton
IN MX 20 postal
IN MX 90 remote.mysite.com
newton IN MX 10 apple
IN MX postal
mysite.com IN MX 10 granada.mysite.com
IN MX 20 laguna.mysite.com
Host dalton normally receives its own mail since it is listed as its own highest-priority destination host. If dalton is unavailable, mail will be redirected first to host postal and then to host remote.mysite.com. In contrast, e-mail destined for host newton is redirected to host apple under normal circumstances. If apple is unavailable, mail will go to postal instead. Thus, postal serves as a backup mail server for both hosts (and may do so for an entire network).
The final two lines specify a default mail destination system for mail addressed to email@example.com; by default, mail will go to the system granada, which serves as the incoming mail server for that site. System laguna is specified as a backup.
Mail aliases are another way of rerouting e-mail. In contrast to DNS MX records, these operate on a per-user basis. Mail aliases are defined in the file /etc/ aliases; this feature is part of the sendmail package (and also of other transport agents). Note that some mailer programs also allow personal mail aliases to be defined, but these apply only to outgoing messages and won’t be considered here.
Entries in the aliases file have the following format:
Here are some example entries:
The first two entries illustrate user account aliases. In this case, mail for aef would be redirected to aefrisch. The name aefrisch is itself an alias and expands to aefrisch@dalton, so mail coming to this system for aef would go to aefrisch@dalton (at least to start with). Note that aliases continue to be expanded as long as is necessary. The third entry defines an alias for max:maxwell@newton. This is a terminal alias; the initial backslash prevents any further expansion.
The second group of sample entries is used to define some mailing lists. The first two lists have three members each. The third list, science, has two other mailing lists as its members (along with max). Any duplicates in the resulting list will automatically be removed by sendmail. Note also that entry order is irrelevant in the aliases file. Thus, the alias jones does not need to precede its use in the chem mailing list.
Whenever you edit the aliases file, you must convert the text file to the actual binary databases used by sendmail by running the newaliases command (equivalent to sendmail -bi). Aliases may also be used to redirect mail to a file or a program; we will not concern ourselves with these capabilities here.
Mail forwarding is the third mail-redirection mechanism we will consider. It is very similar to mail aliasing. There are two main mechanisms for accomplishing it. First, a file named .forward may be created in a user’s home directory. It should contain one or more e-mail addresses to which e-mail should be forwarded (the simplest format is to put each address on a separate line). For example, if the .forward file in user chavez‘s home directory contained the single line firstname.lastname@example.org, then her e-mail would be forwarded to the specified address. If she wanted to keep a local copy of the mail as well, she could use this .forward file:
This file would forward the mail to the same address and also place a copy of each forwarded message into the file mail_pile in her home directory.
The second mail-forwarding mechanism is an internal feature of sendmail, and we will discuss it next month.
Putting It All Together
How do all of these separate pieces interact to affect mail delivery? Basically, at each attempted delivery host, MX records are examined to see whether e-mail is rerouted at the DNS level or not. If so, the mail is sent to the same user at the new host, whose own MX records are then checked.
If no MX record has an effect, then the address is examined for aliasing via the aliases file and then the forwarding mechanism. Either of these has the potential to redirect the mail to a different user and/or host. If the host changes, the message is routed to the specified host, where MX record- checking occurs again.
If all of the aliasing does not redirect the message to a different host, then the message gets delivered to the final resulting user on the local system.
Another example should be able to make this clearer. Suppose a message is addressed to email@example.com. It arrives first at the incoming mail server poffice.doe.com (designated by the MX record for the domain itself). At this point in time, the MX records for poffice are examined, and its highest-priority record redirects the mail to host sorter. Thus, the address now becomes firstname.lastname@example.org. com, and the mail is redirected at the DNS level.
Now suppose sorter‘s MX record points to itself, but an alias on that system defines jane as jane@incognito. The address now becomes jane@ incognito, and the mail is transported accordingly. Host incognito in turn has an MX record pointing its mail to erewhon, so the address becomes jane@erewhon (and the sendmail process on incognito never handles the message).
Finally, on erewhon (which has an MX record that points to itself), jane is aliased as jsmith. User jsmith has a .forward file which consists of the entry email@example.com. So, the mail is then readdressed to firstname.lastname@example.org, and the various sendmail processes in this domain send it back out onto the Internet. When the message finally arrives at faraway.org, the entire process begins again.
More Next Month…
Hopefully, the information in this column has given you a good feeling for what is involved in making an organization’s mail system work successfully. Next month, we will consider many of these concepts in more detail as we discuss configuring sendmail for various, common uses.
Æleen Frisch is the author of Essential System Administration. She can be reached at email@example.com.