Procmail Basics
Pages: 1, 2

Related Reading Programming Internet Email By David Wood Table of Contents Index Sample Chapter Full Description

The .procmailrc file has two sections. The first section contains your path and environment variables; the second section contains your filtering recipes. Once you've copied the default .procmailrc into your home directory, immediately edit the first section before you try to download any email. I'll open up my .procmailrc with my favorite text editor to demonstrate:

# Please check if all the paths in PATH are reachable,
# remove the ones that are not.

PATH=$HOME/bin:/usr/bin:/usr/ucb:/bin:/usr/local/bin:.

FreeBSD doesn't have a /usr/ucb, so remove that bit, along with the extra colon, so the line looks like this:

PATH=$HOME/bin:/usr/bin:/bin:/usr/local/bin:.

MAILDIR=$HOME/Mail # You'd better make sure it exists

Because I use pine, my mail goes to a directory called mail, so I'll change that capital M to a small m. If you don't use pine as your email reader, do an ls -F of your home directory. Somewhere in the output you should have a directory where you store your email; it is usually called mail or Mail. If you do an ls of that directory, you should be able to recognize your mail folders (e.g. sent, saved, etc.) It is very important that your MAILDIR line reflects the correct directory.

The next three environment variables can be left as is. The DEFAULT variable tells procmail to create a directory called mbox, where it will store all messages that don't match any of your recipes. The LOGFILE variable creates a directory called from, where your procmail logs will be stored.

DEFAULT=$MAILDIR/mbox
LOGFILE=$MAILDIR/from
LOCKFILE=$HOME/.lockmail

After the environment variables is the section containing some sample recipes:

:0                  # Anything from thf
* ^From.*thf@somewhere.someplace
todd                # will go to $MAILDIR/todd

:0                  # Anything from people at uunet
* ^From.*@uunet
uunetbox            # will go to $MAILDIR/uunetbox

:0                  # Anything from Henry
* ^From.*henry
henries             # will go to $MAILDIR/henries

# Anything that has not been delivered by now will go
# to $DEFAULT using LOCKFILE=$DEFAULT$LOCKEXT

Once you've edited your path and environment variables, the procmail utility has enough information to run correctly; until you customize those recipes, it will simply place all of your email in a folder called mbox. For now, I'll remove the sample recipes. I'll also test to make sure that I haven't broken anything by restarting fetchmail and opening up pine. I notice that I have a new folder in my folder list called mbox, and after a few moments, there's some new mail in it. So far so good.

Now the fun part begins, as I devise recipes to organize my email. I'll start by taking a look at the mailing lists I'm subscribed to, as they represent the biggest chunk of my email. I'm subscribed to freebsd questions, several mailing lists from securityfocus.com, and the Internet drafts list. So I'll start by making three recipes to sort those three general topics.

Every recipe has the same basic syntax and requires a minimum of three lines. The recipe can be as simple or as complicated as you want to make it; there is no right or wrong way to create a recipe, as long as it follows the correct syntax and gets the job done. Let's start by looking at that three line syntax:

:0              #first line
*               #second line
file            #third line

You'll note that comments are allowed (and recommended, so you remember why you created that recipe) and are indicated by a #.

The first line indicates the beginning of a recipe and is easy for me to remember, as it sort of looks like my expression when I'm in the kitchen. You'll sometimes see more characters in the first line if someone is doing something a bit more complicated, but we'll save complicated for later.

The second line is the condition. In simple recipes, it will start with an * and go on from there; this is the part of the recipe that can be as complicated as you want. You can also make as many conditions as you want in a recipe.

The third line is the action, which usually means the folder in which you want procmail to put the email message that met your condition.

Now, let's translate all this into three recipes to organize my mailing lists. Recipes to sort mailing lists are the simplest to create, as we can use procmail's built-in ^TO_ expression. This expression will scan the To: and Cc: lines of an email message's header. This is ideal for finding mailing lists, as a person either sends a message "to" the mailing list, or they reply to the original posting, which will send a "cc" back to the mailing list.

Before I create the actual recipes, it's helpful to know what procmail does when it reads my recipes. By default, it will scan the headers of incoming email messages for the conditions I've created in my recipes. And, by default, it will stop reading when it finds a matching condition. This means that order is important for two reasons. One, procmail will work much faster if I put the recipes that will generate the most matches at the top of the recipe section; for example, most of my email comes from the freebsd questions mailing list, so that recipe should be near the top. Second, if I want a recipe that scans for virii, it should be placed before any other recipes, so it will scan every message as it arrives.

One last thing before creating those recipes: since procmail scans headers by default and most of your recipes will reflect this, it is helpful to understand what an email header is and what type of information one would expect to see in the various header fields. If you're rusty on this subject, there is an excellent tutorial here.

Okay, on with the first three recipes. Since most of my messages come from freebsd questions, followed by the security lists, followed by the Internet drafts, my recipes look like this:

:0                        # Anything from questions@freebsd.org
* ^TO_*questions@freebsd.org
questions                       # will go to $MAILDIR/questions

:0                      # Anything from a security mailing list
* ^TO_*security
security                         # will go to $MAILDIR/security
 
:0                         # Anything regarding Internet drafts
* ^TO_*ietf
drafts                             # will go to $MAILDIR/drafts

I've used the same logic in each of the recipes. The only semi-complicated bit is the syntax of that expression in the condition line; it should look like this:

* ^TO_*

and be immediately followed by the expression you want procmail to search for in the To: or Cc: fields of a message's header. When you're creating your own recipes, take a look at those fields in your email messages and look for the common expression. For example, if I had told the first recipe to search for the expression "freebsd" instead of "questions@freebsd.org," it would catch all messages from any freebsd mailing list (e.g. newbies, advocacy, security, etc.). I was a bit more particular and specified that I just wanted it to catch messages from the questions list at freebsd.org.

I kept my second recipe more general. It actually catches every message from "securityfocus.com," "security@freebsd.org," and even my daily security output, since they all share the common expression "security." This way any email that is security-related will be placed into a common folder, which is what I wanted.

For the Internet drafts, I noticed that they always came from ietf.org. Since the expression ietf is fairly unique, I used that as my condition.

Each action specifies a folder for the messages to be placed in. The procmail utility automagically created the folders questions, security, and drafts for me the first time it came across a message that required that action. Any message that didn't match those three recipes was put in the default folder of mbox. This is handy when you're creating your recipes, as you'll be able to see which messages don't match a recipe and see if there is a need to lump them together and create a recipe to do that for you.

There're a few other recipes you may want to include to get you started. The first is out of man procmailex and deals with duplicate messages. It should be the very first recipe, and looks like this:

:0 Whc: msgid.lock
| formail -D 8192 msgid.cache   #prevent duplicate messages

:0 a:
duplicates         #but store them instead of deleting them

This variant of the recipe will send all duplicate messages to a folder called duplicates. That manpage also gives an alternate recipe that will instead delete any duplicate messages, but also warns that you might lose other email messages if you have a scripting issue in a complicated recipe. You may want to stick with the above recipe until you reach procmail guru status.

The last recipe I'll demonstrate today is an example of a virus-scanning recipe. The week I wrote this article, the "Badtrans" virus was rampant and was becoming a bit of an irritation. I borrowed the following recipe from the freebsd questions mailing list and placed it right below my duplicate recipe:

# Stupid BadTrans virus
:0:
* ^From: .+ \<_.*\>$
garbage

:0:
* ^From: _.+ \(.+\)$
garbage

This is an example of a more complicated expression. Don't despair if you don't understand the expressions in those condition lines; the important thing is that this recipe works. Remember, when you want to create a recipe, you don't have to reinvent the wheel. There's a good chance that someone else has already created a recipe that does what you want to do. This is especially true when new virii are released, as procmail users will post their recipes to the procmail mailing list. You can search for those recipes here.

If I do a search for "badtrans," I'll come across several recipes which vary in their complexity, but which all get the job done.

This should get you started on using procmail. In the next article, we'll look at some more complicated expressions you can use in your recipes and also take a look at procmail's logging statistics.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.

---

Read more FreeBSD Basics columns.

Return to the BSD DevCenter.