I planned to write about
pf log file analysis tools and techniques, but the mail I received after the last installment of Securing Small Networks With OpenBSD made me change my mind, because many readers want to know more about the way
So I'll leave
pf log analysis until the next installment, and answer two questions: one about
pf rules and one about problems with sendmail.
Why are "in" rules used to filter outbound traffic?
That question pops up quite frequently in mail I receive and in conversations with my students. The answer lies in the difference between how
pf looks at packets and interfaces and how we view them. Unlike the administrator writing the rules,
pf knows nothing about the routes that packets have to follow. Its only job is to check packets arriving or leaving network interfaces on the machine it runs on; all it does is check their type, flags, shape, state, origin, and destination. Inbound packets are matched against in rules and outbound packets are matched against out rules. That's the whole magic.
In order to better understand the whole process, let's have a closer look at the main steps on the route that packets sent from a machine on the internal private network have to follow in order to reach a host located somewhere on the Internet (we'll borrow the host and network addresses, and interface names from the first installment of Securing Small Networks With OpenBSD:
pfmatches the inbound packets against the NAT rules and performs address translation and redirection.
pfmatches the inbound against the in rules for the ne1 interface.
pftests are let into the firewall and their destination is checked against the routing table stored on the firewall. The routing table specifies which interface a packet should be sent to in order to reach its destination.
tun0external interface. They are ready to leave, but before the firewall machine sends them on,
pfkicks in once again.
pfmatches the outbound packets against the out rules for the
pfmatches the outbound packets against the NAT rules and performs address translation and redirection.
pftests are sent to the network the
tun0is connected to. How they reach the destination host is no longer of interest to the firewall; the routers managing traffic on other networks will take care of it.
Packets arriving from the hosts located on the Internet follow a similar route, but in the opposite direction, and with other
pf table sections applied to them.
Why is my
/var/spool/clientmqueue directory filled with strange files?
Why is sendmail being run at regular intervals after I turn it off in rc.conf?
What are these strange noises my firewall is making at regular intervals?
Why is root mail filled with messages from cron?
Why is the disk on the firewall filling up very quickly?
Believe it or not, all of these questions often have the same answer: you are probably having problems with scripts run by the
cron daemon (
man 1 crontab, and
man 5 crontab explain how cron works in detail).
There are three such scripts installed by default in OpenBSD:
/etc/monthly. They perform daily, weekly, and monthly security checks and basic system housekeeping, and send the results to user root.
cron runs a script, it will send the script's output to the user specified in the MAILTO variable in the
crontab from which the script was run, or the owner of that
crontab. That's why the root mail account is being filled with "strange" mail.
cron daemon uses sendmail to deliver messages. It doesn't matter that you set the
sendmail_flags=NO flag in
/etc/rc.conf, such settings only tell the system not to start the sendmail daemon -- the sendmail itself can still be run from the command line, a script, or another program. That is why top shows sendmail on the list of active processes from time to time.
As long as sendmail is being run by
cron after running one of the default scripts, or one of your own scripts, you have nothing to worry about. The strange noises your machine makes at regular intervals are the sounds of the systems plowing through the filesystem.
When the firewall machine is set up in the way described in Part 1 or 2 of Securing Small Networks With OpenBSD, sendmail cannot deliver messages sent by cron and fills
/var/spool/clientmqueue with strangely named files. These are undelivered messages and their headers. You can read them using the
less command, followed by the name of the file you want to read. Each message/header pair has a different prefix followed by an identical number. But since that way of reading mail is not very convenient, you can make your life easier in two ways: either relax the firewall rules, which will be the subject of a future article, or modify the daily script and the root
crontab in the following way:
sh /etc/security 2>&1 > $OUT if [ -s $OUT ]; then mail -s "$host daily insecurity output" root < $OUT fi
sh /etc/security 2>&1 > $OUT.TMP if [ -s $OUT.TMP ]; then echo "$host daily insecurity output" > $OUT cat $OUT.TMP >> $OUT rm $OUT.TMP /usr/libexec/mail.local root > $OUT fi
crontab for user root with:
$ crontab –u root –e
30 1 * * * /bin/sh /etc/daily 2>&1 | tee /var/log/daily.out | mail -s "`/bin/hostname` daily output" root
30 3 * * 6 /bin/sh /etc/weekly 2>&1 | tee /var/log/weekly.out | mail -s "`/bin/hostname` weekly output" root
30 5 1 * * /bin/sh /etc/monthly 2>&1 | tee /var/log/monthly.out | mail -s "`/bin/hostname` monthly output" root
30 1 * * * /bin/sh /etc/daily 2>&1 | tee /var/log/daily.out | /usr/libexec/mail.local root
30 3 * * 6 /bin/sh /etc/weekly 2>&1 | tee /var/log/weekly.out | /usr/libexec/mail.local root
30 5 1 * * /bin/sh /etc/monthly 2>&1 | tee /var/log/monthly.out | /usr/libexec/mail.local root
These changes replace the mail command (which calls sendmail) with
mail.local, which delivers mail straight to the local user's mailbox, bypassing sendmail.
As for the undelivered files from
/var/spool/clientmqueue, you can remove them with:
$ rm /var/spool/clientmqueue/*
If the system complains about the length of the argument list, as can happen when this directory is filed with thousands of files, you should use
ls, and wait for it to display all files. When the command prompt is visible again, look at filenames and try to spot identical prefixes you can use to construct a wildcard, such as:
qfg3MM and you can use:
$ rm qfg3MM*
to remove them.
After removing several smaller groups of files, you can try to shorten the wildcard to make it encompass more files:
$ rm qfg*
If you still get errors, use
ls, construct a longer wildcard, and repeat the process as many times as necessary to clean the
/var/spool/clientmqueue directory. After you remove all files, make sure to monitor this directory for new, undelivered messages.
Alternately, you can look into xargs to help with the long command lines.
Until next time.
Jacek Artymiak started his adventure with computers in 1986 with Sinclair ZX Spectrum. He's been using various commercial and Open Source Unix systems since 1991. Today, Jacek runs devGuide.net, writes and teaches about Open Source software and security, and tries to make things happen.
Return to the BSD DevCenter.
Copyright © 2009 O'Reilly Media, Inc.