ONLamp.com: Mail Server Filtering

Most of what arrives at my mail servers is unwanted: viruses, spam, and executable garbage. Even if you're running something other than Windows on the desktop, the sudden appearance of a new virus can overwhelm your inbox. If you're an administrator, your users likely aren't as reliable about not clicking on attachments as we'd all like. Combined with the flood of spam and random garbage, putting a mail server on the Internet without filtering is like covering yourself with barbecue sauce and breaking into the Charity Home for Badgers with Rabies. Decent spam and virus protection measures can save you a lot of time and effort.

While the systems I use for this article are FreeBSD, the tools and techniques will work on any UNIX-like operating system running modern Sendmail.

Many commercial vendors provide garbage mail protection for Sendmail, but if I go around buying software then I'm never going to be able to afford that 17" laptop. Instead, I use MIMEDefang for generic content handling, SpamAssassin to identify unwanted bulk mail, and ClamAV to reject viruses. This combination eliminates almost all unwanted mail, while letting the good stuff through.

Sendmail(8) provides a milter (or mail filter) API for third-party programs. This means that it's fairly straightforward to add functionality to Sendmail with very little overhead. We can attach MIMEDefang to Sendmail via milter. MIMEDefang can call both SpamAssassin and ClamAV. To make this work, however, you must install and configure the various programs in the correct order.

Install the ClamAV Antivirus Scanner

Install ClamAV from /usr/ports/security/clamav by running make install. Do not run make clean yet! The work subdirectory contains some sample viruses that we'll use to confirm that the program works correctly. The port will install the main virus scanner, clamd(8), a command-line console clamscan(1), a preliminary virus signature database, and an assortment of documentation and ancillary programs. Under normal operation, the clamd(8) virus scanner should be running at all times. Other programs can send files to clamd(8) to learn if they are infected.

Start clamd from the command line. Though it requires configuration, we'll be able to test it before doing our custom setup. While you're still in the port directory, use the console command to scan the work subdirectory for viruses.

The -r recursively scans the current directory and the -l tells clamscan to log the output to the text file testoutput.txt. When you run this command, clamscan will check every file under the current directory for viruses and print out its status. A condensed version of the results, containing only the path to the infected files and a list of statistics, will appear in the logfile. For ClamAV 0.65, this test should discover five infected files.

Now, let's configure ClamAV to cooperate with MIMEDefang. The main configuration file is /usr/local/etc/clamav.conf. Many of the settings are tweakable as you desire. See the clamav.conf(5) man page for all of the options. The most important change is that you should set the user to mailnull, the same user that Sendmail and MIMEDefang run as.

By changing this setting, you also need to change the permissions on the various directories to which ClamAV writes.

Now that you have a basic virus scanner, you can update your virus definitions. Virus definitions are maintained by volunteers from around the world. When a major new virus hits, you can expect to see a definition available within hours. As I write this, the ClamAV database has signatures for 10,131 popular viruses. First, run freshclam as root to confirm that your software can successfully contact one of the virus signature mirrors and download the latest definitions. You should see the program check the freshness of main.cvd and daily.cvd before returning to the command line.

Once you know the update process works, enable the freshclam daemon to check for updates. There is one minor complication, however; freshclam runs as clamav and clamd runs as mailnull, so by default you won't be able to write freshclam reports in the clamav log directory. While you can muck around with file and directory ownership or change the user that freshclam runs as, the simplest thing to do is put your freshclam log elsewhere. When you're done, your /etc/rc.conf should have the following new lines:

The next time you reboot, the system will start the ClamAV virus scanner and updater. You might need to touch /var/log/freshclam.log and chown clamav:clamav /var/log/freshclam.log if freshclam(8) has trouble starting.

Related Reading

BSD Hacks
100 Industrial Tip & Tools
By Dru Lavigne

Table of Contents
Index

Read Online--Safari
Search this book on Safari:

Integrating ClamAV with MIMEDefang

By default, ClamAV puts its UNIX socket in /var/run/clamav/clamd. MIMEDefang expects to find it in /var/spool/MIMEDefang/clamd.sock. One or the other must change. If you do not change the socket, you will see errors in the mail log where MIMEDefang cannot communicate with the virus scanner. I consistently change ClamAV to use the MIMEDefang location, simply because it doesn't make any difference to the programs, but it's easier for me to remember. Change this in the clamav.conf file under the LocalSocket setting.

Once you change this in the configuration, however, ClamAV will not run until you've installed MIMEDefang. Despite advances in computing technology, placing sockets in nonexistent directories still presents difficulties.

Install SpamAssassin

SpamAssassin is perhaps the most celebrated piece of anti-spam software. It's a Perl package that uses pattern-matching to assign each piece of email a score. Key phrases, such as "Make money fast" and "Work from home," will increase the mail's score, as will bogus headers and an origination IP of a known spam source. You can set your mail client to delete or filter mail that has a score above your preferred limit.

As SpamAssassin is managed as part of MIMEDefang, we don't need to configure SpamAssassin itself; its configuration has been assimilated into MIMEDefang. Proceed directly to installing MIMEDefang.

Install MIMEDefang

MIMEDefang can add the common legalese found at the bottom of some email messages, provide very fine-grained access control, and alter emails in almost any other way desired. Our main interest is in using MIMEDefang to strip out unwanted attachment types and coordinate ClamAV and SpamAssassin activities. MIMEDefang's port lives under /usr/ports/mail/mimedefang. Like SpamAssassin, it's written in Perl. Like ClamAV, it runs in daemon mode, so that the system can avoid the massive overhead of starting up a Perl program for every email that passes through the system. When you install MIMEDefang, the configure script will automatically detect that you have both ClamAV and SpamAssassin and will build itself appropriately.

MIMEDefang's configuration file, /usr/local/etc/mimedefang/mimedefang-filter, contains fragments of Perl code that integrate with the main MIMEDefang daemon. If you are not comfortable with Perl, don't worry; just be certain that you implement new functions exactly as they are shown and you won't have any trouble. (I also highly recommend that you spend a few hours with an introductory Perl text; even though I'm a systems administrator and not a programmer, Perl is far too useful in my day-to-day work.)

You should set a few variables in mimedefang, such as $AdminAddress and $AdminName. If you have a separate MIMEDefang administrator, you can enter a specific name and email address, but otherwise, just use your network's generic mail management information. (For those of you unfamiliar with Perl, the single and double quotation marks are very important and must be as shown.)

MIMEDefang will occasionally send an email about an action it has taken. Set the $DaemonAddress variable to the email address you want it to use.

Everything else in this file is strictly optional. Take a look at some of the settings, however, as you might find them useful in your network. There are examples of blocking emails with too many MIME parts, as well as configuring where MIMEDefang will place its alerts. The default includes antispam and antivirus functions, as well as mail quarantine.

Personally, I find that MIMEDefang's mail quarantine functions are reliable enough that I'm comfortable simply rejecting viral emails. There's a comment in the mimedefang-filter file much like this:

By commenting out the line beginning with action_quarantine, MIMEDefang will make Sendmail reject incoming viruses as soon as possible.

Finally, the variable $bad_exts contains a list of extensions that MIMEDefang will block. Extend this list as needed. For example, if some evil mastermind were to write a high-powered virus that propagated via files with the .doc extension, you could block them by adding .doc to this list and restarting MIMEDefang.

To make MIMEDefang start at boot time, go into /usr/local/etc/rc.d and copy mimedefang.sh-dist to mimedefang.sh. By default, this program does not start when the system boots. You must have clamd(8) and MIMEDefang running; SpamAssassin is included as part of MIMEDefang, so you don't need spamd(8) as you would for a standalone SpamAssassin installation.

Configuring SpamAssassin for MIMEDefang

MIMEDefang includes a configuration file for its SpamAssassin calls: /usr/local/etc/mimedefang/spamassassin/sa-mimedefang.cf. If you want to make any changes to the global SpamAssassin settings, change this file. Editing other SpamAssassin configuration files will have no effect whatsoever on SpamAssassin. You can configure SpamAssassin in almost endless ways; we'll cover only the bare bones.

Adjust the sensitivity of the filter with the required_hits variable. The default is 5. When a piece of email scores more than 5 points on the spam-o-meter, SpamAssassin flags it as a piece of spam. Users can choose to filter as they see fit.

Another important feature is the whitelist, which allows you to list email addresses that should never be considered spam sources. For example, my sister sends HTML-laden, image-heavy email from a known spam sewer. By listing her email address on a whitelist_from line, SpamAssassin will let it pass, even if she forwards me a piece of pornographic spam and asks me where it came from. Similarly, blacklist_from variables allow me never to see email from chosen addresses.

Making Filtering Easier

One surprise for users familiar with SpamAssassin is that MIMEDefang does not allow SpamAssassin to alter the email in any way. Instead, SpamAssassin reports a score back to MIMEDefang and lets MIMEDefang change the message. The MIMEDefang FAQ includes several suggestions for editing the filter configuration so as to display SpamAssassin information as desired.

By default, MIMEDefang displays a single header that contains the SpamAssassin score and a series of asterisks, much as the example below shows.

Many email clients have difficulty filtering on this header; their filtering rules will not let them compare the numerical score. By making one minor change in how MIMEDefang marks spam email, you can make life much easier for these users.

If you exchange the $score and the $hits variables as shown below, your users will be able to filter on the number of asterisks instead.

You can write a rule that searches the X-Spam-Score header for a certain number of asterisks and filters those messages away. With this header setup, users can easily adjust their own spam tolerances; you can be harsh on spam detection on the server, knowing that all you're really doing is adding a header. If a user decides to set his sensitivity to two asterisks and loses some vital email, that's not your fault.

Integrating with Sendmail

Now that you have a working MIMEDefang/SpamAssassin/ClamAV installation, how do you tie all this into Sendmail's milter interface? A modern Sendmail system includes Makefiles that simplify creating a configuration file. You need a custom Sendmail .mc file, so go into /etc/mail and copy freebsd.mc to a file named after the host. For example, on bewilderbeast.blackhelicopters.org, I use a .mc file called bewilderbeast.mc. Next, enter this filename in /etc/make.conf as the SENDMAIL_MC variable.

Whenever you rebuild your system, this tells make(1) to use your custom .mc file instead of the default. This retains your customizations, instead of making your spam protection fail after every make installworld. (After an upgrade, be certain you compare the newly updated freebsd.mc with your custom file, just in case something important changes.) Now add the lines below to the end of your custom .mc file.

Stay in the /etc/mail directory and run

make all install
restart

. This will rebuild your sendmail.cf file from your customized configuration file and restart Sendmail with it.

Conclusion

If you tail -f /var/log/maillog, you will be able to see log entries from MIMEDefang as it processes every message. Viruses will bounce at the border, while spam will be conveniently flagged for your users. Even a small mail server such as mine is hit with hundreds of viruses and thousands of pieces of spam a day; now, it's all flagged or rejected. Just as much junk will travel across the Internet, but you won't be so aware of it. Today, that's as good as you're going to get.