ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Securing Small Networks with OpenBSD Archiving PF Firewall Logs

by Jacek Artymiak
07/25/2002

Welcome back.

In the previous installment of this series we learned how to fine-tune the process of saving and rotating pf logs to match our preferences. Today we'll look at the problem of automating the transfer of logs from the firewall to one of the workstations connected to the internal private network segment. But, you may ask, why won't we analyze pf logs on the firewall instead? Well, while we could analyze the logs on the firewall, it is usually more convenient, efficient and safe to do it on another computer with a faster processor, more memory, and larger hard disks. We shouldn't really ask the firewall to do anything more than packet filtering. The task of log archiving and analysis is best left to another computer.

The monitoring workstation need not be connected to the internal private network segment. Logs can be sent to a trusted machine located on the outside of our network, although that is less common in small to medium-size networks. Some larger networks connecting computers at more than one physical location, though within the same organization often send logs to one central location for analysis, but still keep the copies of logs archived on-site as a security precaution.

Many administrators managing several client's networks from a single location, send logs to a single machine located inside the administrator's own network. Although this might seem a little unwise from the point of view of security, if strong encryption is used and the administrator follows all basic security precautions, the chances of hackers getting at the precious data is very low. One important drawback of sending logs outside your own network is the fact that they consume a lot of bandwidth, and bandwidth costs money.

Which method of archiving logs is best for you?

The choice of the method for remote logging and archiving depends on many factors, but in most cases, it will all boil down to the security policy of your organization, the amount of time and money you can devote to overseeing and monitoring the whole setup, as well as what you feel is most convenient to you. But in any situation, it is highly advisable that you choose a method that is the easiest to build, manage and secure within the constraints of law, money and time.

There are literally dozens of solutions for archiving logs and we cannot possibly describe them all in this article, because the exact details of every configuration are highly dependent on each site's security policies and the personal preferences of its administrators. However, since we are using OpenBSD, we can assume that certain things are going to look and work the same on most OpenBSD installations. Software tools like pf, newsyslog, or syslogd are available on all OpenBSD 3.0 (and later) systems; similarly ssh and Perl are also standard fare on all free and commercial UNIX derivatives (Linux, BSD, Mac OS X, etc.).

Click for larger view
Figure 1: Monitoring station connected to the internal private network segment. (You can click on the figure to open a full-size view.)

For the sake of simplicity we'll use a network split into two segments: private network and DMZ (Figure 1), both protected by the same firewall, which will automatically collect and send logs to one of the workstations connected to the internal private network segment. (The details of the configuration of such network design were discussed in part 1 and part 2 of this series, and can be easily adapted to other network configurations.)

This solution is convenient and cost-effective, because the administrator can log into the monitoring station from other hosts and can use the same workstation for doing things other than log collection, archiving, and analysis. Also, because the monitoring station is connected to the network, which itself is connected to the outside world, upgrades of the station's operating system and tools are quick and easy. (The monitoring station need not be running one of UNIX variants, but our job will be much easier, if it does.)

Click for larger view
Figure 2: Monitoring station connected to a separate network segment. (You can click on the figure to open a full-size view.)

Also in Securing Small Networks with OpenBSD:

Changes in pf: Packet Filtering

Changes in pf: More on NAT

NAT with pf

Patching OpenBSD

Downloading Files from Behind the Firewall

There is a lot to like about this configuration, but the administrator should be careful to not overload the internal network with additional traffic generated when logs are transmitted from the firewall to the monitoring station. If our firewall is set up to log all traffic going in and out of the private segments and the DMZ segment, then sending the logs to the workstation connected to the internal private segment of your network (Figure 1) more than doubles the amount of data transferred in that segment. When the network becomes congested, there are two things we can do:

If you want to connect the monitoring station via a separate interface, you will need to do the following things:

Configuring the firewall

Before you start rebuilding your network, it would be a good idea to test the software side of this configuration without touching the hardware, that way you will have fewer things to debug. After you have the software working, reconfiguring the hardware is rather easy. Also, to avoid the risk choking your network with log transfers during the testing, either choose a time when the traffic is low or log only a small fraction of traffic, just to test things on a smaller scale before you let the whole thing run on full power.

To limit the size of logs, copy /etc/pf.conf to /etc/pf.conf_old, open /etc/pf.conf in a text editor, and add the log-all keyword to one of the pass out rules for the external interface. That way you will be logging only the outbound traffic leaving the external interface and originating from your private network. For example, change this rule

# allow TCP IPv4 connections to the outside world, keep state 
 
pass  out on $ExtIF inet proto tcp all flags S/SA modulate state

to this

# allow TCP IPv4 connections to the outside world, keep state 
 
pass  out log-all on $ExtIF inet proto tcp all flags S/SA modulate state 

You can begin logging all traffic again, when you are sure that the monitoring station is receiving logs.

While you are editing the pf configuration file, you should also add a line that prevents pf from logging packets sent from the firewall to the monitoring station to port 22, where you should have ssh listening for connections. If you don't do that, you will create a loop which will bring your network to a halt (the packets carrying the logs will be logged and sent to the monitoring station and logged again, and sent to the monitoring station, and so on ...).

The following rule should be added at the end of the pf rule section for the internal private network segment interface (you may need to modify it to match your network configuration, but don't add the log or log-all keywords):

# allow pfloggin on a remote host 
# (there are two lines below) 
 
pass  out on $PrvIF inet proto tcp  from xxx.xxx.xxx.xxx 
		to yyy.yyy.yyy.yyy port 22 modulate state 
 
pass  out on $PrvIF inet proto udp  from xxx.xxx.xxx.xxx 
		to yyy.yyy.yyy.yyy port 22 keep state

Save the modified /etc/pf.conf and reload it with

# pfctl -R /etc/pf.conf

Your firewall is now logging only the outbound traffic. It is not sending the logs to the monitoring station, for that we need to use a script, which we'll write in Perl. Using Perl is not a requirement, you could use shell, AWK, or any other language, but since the Perl interpreter is installed as a part of the default OpenBSD configuration, there is little point to not use it.

Sendpflog

We are now ready to write the script that sends the logs to the firewall. For the sake of simplicity, we will try to make as little changes to the default setup as possible. The only intrusion will be one small change in /etc/newsyslog.conf and the addition of the sendpflog script. The following script will handle the rest of the work:

#!/usr/bin/perl -W 
# 
#                                   Copyright 2002 Jacek Artymiak 
#                                                License: XFree86 
#---------------------------------------------------------------- 
# section  1: open sendpflog.log -- the log used by sendpflog to 
#                                              store its messages 
 
open (LOG, ">> /var/log/sendpflog.log"); 
select (LOG); 
$|=1; 
 
#---------------------------------------------------------------- 
# section  2: define the logme function used to write messages to 
#                                                   sendpflog.log 
 
sub logme { 
 
$datetime = `date`; 
chop $datetime; 
 
$logentry = $datetime . ": sendpflog[$$]: $_[0]" . "\n"; 
 
print LOG $logentry; 
} 
 
#---------------------------------------------------------------- 
# section  3: define the loganddie function used to clean up 
#                                           before sendpflog dies 
 
sub loganddie { 
 
$datetime = `date`; 
chop $datetime; 
 
$logentry = $datetime . ": sendpflog[$$]: Fatal error: $_[0]" 
. ": Exiting ...\n"; 
 
print LOG $logentry; 
 
$logentry = $datetime 
. ": sendpflog[$$]: Closing pflog ...\n"; 
 
print LOG $logentry; 
 
close (INFILE); 
 
$logentry = $datetime 
. ": sendpflog[$$]: Closing output stream ...\n"; 
 
print LOG $logentry; 
 
close (OUTFILE); 
 
$logentry = $datetime 
. ": sendpflog[$$]: Removing sendpflog.pid ...\n"; 
 
print LOG $logentry; 
 
`rm /var/run/sendpflog.pid`; 
 
$logentry = $datetime 
. ": sendpflog[$$]: Exiting.\n"; 
 
print LOG $logentry; 
 
die ($logentry); 
} 
 
#---------------------------------------------------------------- 
# section  4: define the rotatelogs function, which closes 
#             /var/log/pflog and /var/log/sendpflog.log, 
#             sends SIGHUP to pflogd, and reopens /var/log/pflog 
#                                      and /var/log/sendpflog.log 
 
sub rotatelogs() { 
 
logme ("Closing rotated pflog."); 
 
close (INFILE); 
 
logme ("Closing sendpflog.log."); 
 
close (LOG); 
 
open (LOG, ">> /var/log/sendpflog.log"); 
select (LOG); 
$|=1; 
 
logme ("sendpflog.log rotated."); 
 
logme ("Sending SIGHUP to pflogd."); 
 
kill HUP => `cat /var/run/pflogd.pid`; 
 
logme ("Trying to open new pflog ..."); 
 
open (INFILE, "< /var/log/pflog") or 
loganddie( "Could not open pflog:" . $! . '.'); 
 
logme ("pflog opened."); 
} 
 
#---------------------------------------------------------------- 
# section  5: define the open_output function which connects to 
#             the monitoring station 
 
sub open_output { 
 
logme ("Trying to open output stream ..."); 
 
open (OUTFILE, 
'| ssh -q -2 -l username xxx.xxx.xxx.xxx "cat >> pflog"') 
or loganddie ("Could not open output stream: " . $! . '.'); 
 
sleep(5); 
select (OUTFILE); 
$|=1; 
 
logme ("Output stream opened successfully."); 
} 
 
#---------------------------------------------------------------- 
# section  6: set signal handlers 
 
$SIG{HUP}  = 'rotatelogs'; 
$SIG{INT}  = 'loganddie'; 
$SIG{QUIT} = 'loganddie'; 
$SIG{KILL} = 'loganddie'; 
$SIG{TERM} = 'loganddie'; 
$SIG{STOP} = 'loganddie'; 
$SIG{TSTP} = 'loganddie'; 
$SIG{PIPE} = 'IGNORE'; 
 
#---------------------------------------------------------------- 
# section  7: we're waking up 
 
logme ("Starting sendpflog ..."); 
 
#---------------------------------------------------------------- 
# section  8: write the current process ID (PID) to 
#                                          /var/run/sendpflog.pid 
 
logme ("Creating sendpflog.pid ..."); 
 
open (PIDFILE, "> /var/run/sendpflog.pid") or 
loganddie ("Unable to create sendpflog.pid: " . $! . '.'); 
 
logme ("Writing PID to sendpflog.pid ..."); 
 
syswrite PIDFILE, $$; 
 
logme ("Closing sendpflog.pid"); 
 
close PIDFILE; 
 
#---------------------------------------------------------------- 
# section  9: open /var/log/pflog for reading 
 
logme ("Trying to open pflog ..."); 
 
open (INFILE, "< /var/log/pflog") or 
loganddie ("Unable to open pflog: " . $! . '.'); 
 
logme ("pflog opened successfully."); 
 
#---------------------------------------------------------------- 
# section  10: read /var/log/pflogand send it to the monitoring 
#                                                         station 
 
open_output(); 
 
for (;;) { 
while (<INFILE>) { 
if (!(syswrite OUTFILE, $_)) { 
logme("The monitoring station is not responding"); 
close (OUTFILE); 
open_output(); 
} 
} 
 
sleep 1; 
seek (INFILE, 0, 1); 
}

Here is a short summary of what this script does:

Make sendpflog executable, owned by root and a member of the wheel group with these commands (you need to be logged in as root):

# chmod 0700 sendpflog 
# chown root sendpflog 
# chgrp wheel sendpflog 

Move sendpflog to /root/sendpflog with

# mv sendpflog /root/sendpflog

Next, open /etc/newsyslog.conf and replace the following line

/var/log/pflog 600 3 250 * ZB /var/run/pflogd.pid

with these two lines:

/var/log/pflog          600  3    250  *     ZB /var/run/sendpflogd.pid 
/var/log/sendpflog.log  600  3    250  *     ZB /var/run/sendpflogd.pid 

The changes will tell newsyslog to monitor /var/log/pflog and /var/log/sendpflog.log and rotate them when their size meets the preset criteria.

Next we need to generate a key pair used to automatically log into the monitoring station without having to give the password. This is done with the ssh-keygen utility. You can use it to generate three types of keys: rsa1 (for RSA authentication connections made using the SSH1 protocol), rsa (for RSA authenitconnections made using the SSH2 protocol), or dsa (for DSA connections made using the SSH2 protocol). If the monitoring station is running the latest release of OpenSSH, then you can use RSA authentication, older releases of SSH may need to use DSA or RSA1.

Log in as root and run ssh-keygen as follows

# ssh-keygen -t dsa 
Generating public/private dsa key pair. 
Enter file in which to save the key (/root/.ssh/id_dsa): 

Hit Enter, the computer responds with

Enter passphrase (empty for no passphrase):

Hit Enter, the computer responds with

Enter same passphrase again:

Hit Enter, the computer responds with

Your identification has been saved in /root/.ssh/id_dsa. 
Your public key has been saved in /root/.ssh/id_dsa.pub. 
The key fingerprint is: 
6f:38:13:90:04:89:55:e6:50:6f:69:50:5f:77:15:e7 root@localhost 

The /root/.ssh/id_dsa.pub file is the public key which you will copy to the monitoring station to the home directory of an ordinary user which you will need to create for the sole purpose of collecting, archiving and analyzing pf logs (the name of that user must be given in section 5). Do not use the root or any privileged account for that purpose. Also, create an ordinary group that will have only one member-our unprivileged user.

After you create the new account on the monitoring station, copy /root/.ssh/id_dsa.pub to the normal user directory on the firewall, e.g. /home/joe. Then, use scp to copy id_dsa.pub to the monitoring station. Then delete id_dsa.pub from /home/joe and copy (on the monitoring station) id_dsa.pub to the log collecting user's .ssh/authorized_keys2 file. For example, if the log collecting user is called scooter, use this command:

$ cat id_dsa.pub >> /home/scooter/.ssh/authorized_keys2

Note that if the monitoring station is running some other implementation of SSH, the authorized_keys2 file may reside in the .ssh2 directory. Also, if you use the RSA key (generated with ssh-keygen -t rsa), the id_rsa.pub file goes into authorized_keys2 as well, while the RSA1 key, the identity.pub file (generated with ssh-keygen -t rsa1) goes into .ssh/authorized_keys.

OK, it is now time to test the connection; login on your firewall as root and issue the following command:

# ssh -2 -l username xxx.xxx.xxx.xxx

(Replace username and xxx.xxx.xxx.xxx with real user name on the monitoring station and the station's IP number or host name).

If all goes well, you should see a message similar to this one:

RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx. 
Are you sure you want to continue connecting (yes/no)?

Answer yes, and you should be logged in on the monitoring station. Be careful, if ssh asks for the password, then there is something wrong and you cannot log in with the dsa key. To fix it, generate rsa or rsa1 keys and try logging in again (if you use the rsa1 key, make sure you replace -2 in section 5 of sendpflog with -1, this tells ssh to use the SSH1 protocol instead of SSH2). One of them is bound to work (but remember that rsa is more secure than rsa1).

Configuring the monitoring station

Related Reading

Network Security with OpenSSL
Cryptography for Secure Communications
By John Viega, Matt Messier, Pravir Chandra

While you are logged on the monitoring station, create a named pipe in the log collecting user's directory on the station with this command

$ mkfifo -m 0600 pflog

Notice that the name and the location of that named pipe must be given in section 5, after the cat >> command.

You are now ready to test the connection between the firewall and the monitoring station. To do that, log on the firewall as root and start sendpflog with

# /root/sendpflog

Then, log on the monitoring station as the user receiving logs and type this command

$ cat pflog >> pflog-current

Let the computers do their job for a while, browse a few web sites (to generate traffic), and then press Ctrl+C. Type

$ ls -l pflog-current

and you should see something like this (the user and group names and the file size will be different)

-rw-r----- 1 scooter scooter 1256 Jul 10 23:35 pflog-current

If that is the case, you can congratulate yourself, the logs are being transferred to the monitoring station. But if it doesn't work, you should check ssh parameters in section 5, particularly the username, the host IP number or the name of the log file.

You can now read the logs on the monitoring station with tcpdump -r ./pflog, but that is not very convenient. What we need to do is write a script that works a little like newsyslog. And this is something we'll deal with in another installment of this series.

The method of automated log transfers described in this article could be used to transfer other logs /var/log/pflog. This will be left as an exercise to the reader. I will only say that every log will have to be transferred via a separate ssh connection, and the named of the scripts, log files and named pipes should be changed to match the name of a particular log.

The sendpflog script is a part of the OpenBSD Administrator Toolbox project hosted on SourceForge.

Well, that's it for today!

Until next time...

Jacek Artymiak started his adventure with computers in 1986 with Sinclair ZX Spectrum. He's been using various commercial and Open Source Unix systems since 1991. Today, Jacek runs devGuide.net, writes and teaches about Open Source software and security, and tries to make things happen.


Read more Securing Small Networks with OpenBSD columns.

Return to BSD DevCenter.

Copyright © 2009 O'Reilly Media, Inc.