ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


FreeBSD Basics

Understanding E-Mail

08/30/2000

One of the first things every FreeBSD user wishes to configure on his computer is e-mail access. However, a look in the mail section of the ports collection yields an amazing 148 applications dealing with e-mail. To make matters worse, some claim to be MUAs, MTAs, POP3 clients, or IMAP4 servers. These descriptions are not helpful if you are unfamiliar with these terms and just want to be able to send and receive e-mail.

In this article, I'll give a "readers digest" version of some important e-mail concepts, with references to additional information. Then I'll walk you through the steps of configuring your FreeBSD system to send and retrieve e-mail, as well as configuring Pine.

Ever since networks were invented, people have wanted a method of sending messages to each other. This led to the development of many messaging programs, most of which were proprietary, meaning you could only send a message to someone who used the same messaging system. Early messaging systems allowed you to send a message directly to another person's computer. However, this method doesn't work if the other computer is turned off or there is a problem with the physical connection between the two computers.

Most messaging systems prefer to have all users send their messages to a centralized server for storage; since this server is always available, it can act like a post office. If a user has a mailbox on that server, he can periodically check his mailbox to see if he has received any messages.

As the need for messaging grew, standards were developed to allow users to send messages to any user, regardless of the messaging system they used. One of these standards was X.400; another was SMTP, which became the standard used on the Internet. Two interesting points of views on these standards can be found at X.400 - The Better Internet and X.400 is Dead, Long Live X.400.

Unless you work at a company that uses an internal X.400 messaging system, you probably use SMTP to send e-mail. The e-mail addresses you are used to seeing are actually SMTP addresses and look like this: anyuser@anycompany.com You'll note that an SMTP address looks similar to a web address, except it has a @ instead of the first period. The Internet already uses DNS (the Domain Name System) to locate a computer's IP address. When an administrator sets up a mail server, he will create an MX (mail exchange) record in his DNS database to indicate which computer in his network is the mail server. The name to the left of the @ indicates the name of a user's mailbox on that mail server.

When the Internet messaging standard was being developed, it was decided to use the centralized server (post office) approach where users would check their mailboxes. The Internet uses TCP/IP, so new TCP/IP protocols needed to be created. Remember from last week that protocols define the rules of communication and that each TCP/IP protocol has an associated port number and at least one RFC which describes its operation.

SMTP (simple mail transfer protocol) describes the communications between mail servers. Its port number is 25, and its behavior is described in RFC 821.

As its name implies, SMTP merely transfers mail from one server to another until it reaches the correct mail server. In order to ensure that the message makes it into the right mailbox on that server, POP3 (post office protocol 3) is used. POP3 uses port number 110 and is defined by RFCs 1734, 1957, 2449, and STD0053 (standard 53).

POP3 is also responsible for the management of the mailboxes on a mail server. POP3 will not let a user retrieve the messages in their mailbox until they have been authenticated.

If you are unfamiliar with the commands used by POP3 and SMTP, try the SMTP with Telnet tutorial.

This tutorial shows how you yourself can issue SMTP and POP3 commands to send and receive e-mail. These are the same commands that are usually issued by messaging systems and e-mail programs.

One other term you may come across is IMAP4 (Internet Message Access Protocol version 4). IMAP4 is similar to POP3, but with extra features. POP3 assumes that a user will download all of the mail in their mailbox so the copy on the mail server can be deleted; the user will then disconnect from the mail server to read his e-mail "offline." With IMAP4, a user can download just the headers of his e-mail so he can decide which messages he wants to retrieve at this time; he can also delete messages directly at the mail server without having to download them first.

Messaging software is usually divided into two separate components: the MTA (message transfer agent) and the MUA (message user agent). The MTA runs on the mail server and understands both SMTP and POP3. Therefore, it is able to send messages to other SMTP mail servers, receive messages from other SMTP mail servers, and store messages in the appropriate user mailboxes. Sendmail is the most common MTA in use on the Internet and comes bundled and ready to go on your FreeBSD system. Exim, Postfix, and Qmail are examples of other MTAs that you can build using FreeBSD's ports collection.

The MUA is used by users to compose and read e-mail messages. Some MUAs have a POP3 client built-in, meaning you can check for new mail using the MUA; some do not, and will only display e-mail you've retrieved using a separate POP3 client. MUAs differ widely in their features; which MUA to use is a matter of personal preference. There are dozens of MUAs in the ports collection; some of the most popular are Mutt, Pine, and Xfmail.

So, how do all these components work together when you send an e-mail message? You use your MUA (for example, Pine) to create an e-mail message; Pine will pass the message to the MTA (Sendmail), who will query DNS to find the address of the SMTP server hosting the mailbox of the user you are sending the message to. Sendmail will then use SMTP commands to transfer the message to that mail server. That mail server will use POP3 to put the message in the correct mailbox. Your recipient will use POP3 commands to retrieve the message and will use their favorite MUA to read the message and possibly compose a reply to it.

Let's look at the configuration of a FreeBSD system that will be using Sendmail (the MTA), fetchmail (the POP3 client), and Pine (the MUA). Let's assume a very quick and dirty scenario in which I have one user on a single FreeBSD computer who wishes to use e-mail.

Related Reading:

sendmail

sendmail, 2nd Edition
By Bryan Costales & Eric Allman

Like any MTA, Sendmail is a highly configurable and complex program capable of providing messaging for large companies and even ISPs. However, you don't want to start messing with the default Sendmail configuration files unless you know what you're doing. If you ever need to get into more complicated Sendmail configurations, you should first read the book sendmail, 2nd Edition by Brian Costales and Eric Allman. You may also wish to investigate other alternatives such as Qmail or Exim.

Fortunately, the default Sendmail configuration on your FreeBSD system will issue SMTP commands so you can send out e-mail to the world. Sendmail is enabled by default; if you want to double-check that you haven't disabled it at some point, issue the following command:

more /etc/defaults/rc.conf | grep sendmail
sendmail_enable="YES"	# Run the sendmail daemon (or NO).
sendmail_flags="-bd -q30m" # Flags to sendmail (if enabled)

If the sendmail_enable line is set to "NO", you'll have to become the superuser to change it to "YES" using your favorite text editor. Be sure to check your change carefully for typos before saving the file. While you're still the superuser, close all programs running on the computer and issue the following from the Alt-F1 terminal:

shutdown now

When you receive a prompt, press the Enter key, then type:

exit

Watch the messages that appear on the screen; you should see a line similar to this:

starting standard daemons: sendmail.

Now that your MTA is running, you'll want to build the MUA. If you have installed the ports collection on your computer, become the superuser and:

cd /usr/ports/mail/pine4
make && make install

When that is finished, you'll also want to build the POP3 client so:

cd ../fetchmail
make && make install

If you don't use the ports collection, both Pine and Fetchmail have packages that can be downloaded from FreeBSD Ports.

As root:

mkdir /usr/packages

Save the downloaded packages to this directory, then:

cd /usr/packages
pkg_add pine* 
pkg_add fetc*

Now that we have the necessary software, we need to create a user account to use when sending and receiving e-mail. You can either create a user with the same name as the mailbox portion of your e-mail address, or you can create an alias to map an existing user to the mailbox name. Since my e-mail address is genisis@istar.ca, to keep my life simple on my home FreeBSD box, I create a user called genisis. When I wish to access my e-mail, I log in to my FreeBSD system as the user genisis. This trick is useful on small systems as it saves you editing an alias file.

If you need to create a new user, become the superuser and type /stand/sysinstall and select Configure | User Management | User. The login ID should be the same as the name of the mailbox, so in my case it is genisis. Input a password you'll remember and tab over to OK. Arrow over to Cancel twice, then arrow over to Exit Install.

Leave the superuser account and log in as the new user. Now type pine -- you should see the following message:

Creating subdirectory "/home/yourusername/mail" where Pine will store its mail folders.

This will be followed by Pine's greeting text. You'll notice that Pine's commands are always listed at the bottom of your screen. The ^ symbol means use your CTRL key along with the letter next to the ^. Once you've read the greeting, use ^E (Exit This Greeting). This will bring you to Pine's main menu.

Since this is the first time you are running Pine as this user, you'll need to press S to enter Setup, then C to enter Config. The first three items should be edited as follows:

You'll notice that Pine supports a lot of configuration values; if you find one that sounds interesting, highlight it and press the ? key. This will give a short, and usually helpful, description of what this configuration parameter does. I usually change the following on mine:

Once you've finished your configurations, press E to exit Setup and Y to commit changes. This will return you to the main menu where you can press A to edit the address book. The @ key will let you add an e-mail address; when you're finished, type ^X and the address should now show in your address book.

Highlight a user you've added to your address book and press C to compose an e-mail message to them. Arrow down to Cc and input your e-mail address so you'll be sent a copy of the message. Once you've composed your message, make sure you are connected to your ISP and press ^X to send your message. Type y when Pine asks you if you want to send the message. It should say [Sending Mail], then [Message sent and copied to 'sent-mail'.] Use your < key to return to the main menu, then L to enter the folder list. If you arrow over to sent-mail and press enter, you should be able to view and read the message that you sent.

One last point on Pine: it contains a built-in editor called Pico. Pico is a very easy editor to learn as its commands are always listed on the bottom of the screen. You don't have to run Pine to use Pico; if you want to edit a file with Pico, do this:

pico filename 

Now that you've sent your first e-mail with Pine, you'll want to check that you received your copy of the e-mail by using the fetchmail program. Open up another virtual terminal, and log in as your user. I use fetchmail like so:

fetchmail istar.ca
Enter password for genisis@istar.ca: 
fetchmail: IMAP connection to istar.ca failed: Connection refused
1 message for genisis at istar.ca (868 octets).
reading message 1 of 1 (868 octets)  flushed

When you check your e-mail, substitute the value to the right of the @ in your e-mail address for istar.ca. If everything worked, fetchmail will download at least one message. Return to the terminal where you are running Pine and navigate to your inbox to read it.

Note that fetchmail added genisis to my e-mail address for me because I am logged in as genisis. If I happened to be logged in as another user, say biko, it would use the address biko@istar.ca instead.

The fetchmail command would then fail as I don't have a mailbox called biko on Istar's mail server. However, if I'm logged in as biko and invoke fetchmail like so:

fetchmail -u genisis istar.ca

This will tell fetchmail to use the mailbox name of genisis instead of my login name of biko so I can retreive genisis' email.

Fetchmail also has a verbose mode which can be used to view the various POP3 commands between fetchmail and the mail server. You can invoke verbose mode with -v and very verbose mode with -vv. Here is a portion of a session with fetchmail in verbose mode:

fetchmail -v istar.ca
Enter password for genisis@istar.ca:
fetchmail: 5.3.0 querying istar.ca (protocol auto) at Mon, 31 Jul 2000 11:16:28 -0400 (EDT)
fetchmail: 5.3.0 querying istar.ca (protocol IMAP) at Mon, 31 Jul 2000 11:16:28 -0400 (EDT)
fetchmail: IMAP connection to istar.ca failed: Connection refused
fetchmail: 5.3.0 querying istar.ca (protocol POP3) at Mon, 31 Jul 2000 11:16:28 -0400 (EDT)
fetchmail: POP3< +OK iSTAR POP3 Server at istar.ca Ready v1.13 12.2.1996 <398597cc398597cc@istar.ca
fetchmail: POP3> USER genisis
fetchmail: POP3< +OK Complete authentication with the PASS command for genisis@istar.ca
fetchmail: POP3> PASS *
fetchmail: POP3< +OK Authentication successful.
fetchmail: POP3> STAT
fetchmail: POP3< +OK 5 29820
fetchmail: POP3> LAST
fetchmail: POP3< +OK 0
5 messages for genisis at istar.ca (29820 octets).
fetchmail: POP3> LIST
fetchmail: POP3< +OK 5 messages (29820 octets)
fetchmail: POP3< 1 8777
fetchmail: POP3< 2 8961
fetchmail: POP3< 3 4837
fetchmail: POP3< 4 2806
fetchmail: POP3< 5 4439

You should be able to recognize the USER, PASS, and LIST commands from the tutorial.

Fetchmail is highly configurable through its use of switches; its manpage is well worth reading.

Hopefully, we've cleared up some of the mystery regarding how e-mail works. Next week, we'll look at setting permissions.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.


Read more FreeBSD Basics columns.

Discuss this article in the Operating Systems Forum.

Return to the BSD DevCenter.

 

Copyright © 2009 O'Reilly Media, Inc.