ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Securing Small Networks with OpenBSD Managing Advanced PF Logs

by Jacek Artymiak
08/22/2002

Welcome back.

In part 6 and part 7 we learned how to send logs over an ssh connection and how to make that connection more secure. We also did some math to find out how much storage space we'll need to keep old logs for analysis. Today we need to roll up our sleeves again, and do some coding in Perl, because the pflog fifo pipe created on the monitoring station's hard disk in part 6 is a bit like an oil well, gushing with black stuff that needs some additional piping to turn it into an orderly flow.

Catching and taming that wild stream of data is the job of the readpflog script. Written in Perl, it runs in the background and does three basic things: 1) reads pf logs from the pflog fifo pipe, 2) archives logs on the monitoring firewall, and 3) sends them to another fifo pipe so that log analysis software can pick them up for analysis.

#!/usr/bin/perl -W
#
#                                   Copyright 2002 Jacek Artymiak
#                                                License: XFree86
#----------------------------------------------------------------
# section  1: basic setup  

use Fcntl;
use POSIX qw(:errno_h);

$rdelay = 3600;  # log archiving delay (in seconds, 0 turns off); 

#----------------------------------------------------------------
# section  2: check if readpflog.pid exists

if (-e "/home/scooter/readpflog.pid") {

    print "Looks like readpflog is already running, if it is " . 
               "not running, delete /home/scooter/readpflog.pid";
    exit;
}

#----------------------------------------------------------------
# section  3: open readpflog.log --  the log used by readpflog to 
#                                              store its messages

open (LOG, ">> /home/scooter/readpflog.log"); 
select (LOG);
$|=1;

#----------------------------------------------------------------
# section  4: define the logme function used to write messages to 
#                                                   readpflog.log

sub logme {

 $datetime = `date`;
 chop $datetime;

 $logentry = $datetime . ": readpflog[$$]: $_[0]" . "\n";

 print LOG $logentry;
}

#----------------------------------------------------------------
# section  5: define the loganddie function used to clean up 
#                                           before readpflog dies 
 
sub loganddie {

 $datetime = `date`;
 chop $datetime;

 $logentry = $datetime
                      . ": readpflog[$$]: Fatal error: $_[0]" 
        . ": Exiting ...\n";

 print LOG $logentry;

 $logentry = $datetime 
                      . ": readpflog[$$]: Closing pflog ...\n";

 print LOG $logentry;

 close (INFILE);

 $logentry = $datetime 
              . ": readpflog[$$]: Closing pflog-current ...\n";

 print LOG $logentry;

 close (OUTFILE);

 $logentry = $datetime 
              . ": readpflog[$$]: Removing readpflog.pid ...\n";

 print LOG $logentry;

 `rm ~/readpflog.pid`;

 $logentry = $datetime 
              . ": readpflog[$$]: Exiting.\n";

 print LOG $logentry;

 die ($logentry);
}

#----------------------------------------------------------------
# section  6: define the rotatelogs function, which closes and
#                                         reopens ~/readpflog.log
 
sub rotatelogs() {

 logme ("Closing readpflog.log.");

 close (LOG);

 open (LOG, ">> /home/scooter/readpflog.log");
 select (LOG);
 $|=1;

 logme ("readpflog.log rotated.");
}

#----------------------------------------------------------------
# section  7: we're waking up 

logme ("Starting readpflog ...");

#----------------------------------------------------------------
# section  8: write the current process ID (PID) to 
#                                                 ~/readpflog.pid

logme ("Creating readpflog.pid ...");

open (PIDFILE, "> /home/scooter/readpflog.pid") or 
       loganddie ("Unable to create readpflog.pid: " . $! . '.');

logme ("Writing PID to readpflog.pid ...");

syswrite PIDFILE, $$;

logme ("Closing readpflog.pid");

close PIDFILE;

#----------------------------------------------------------------
# section  9: open ~/pflog for reading

sub opensource {

    logme ("Trying to open pflog ...");

    open (INFILE, "< /home/scooter/pflog") or 
                 loganddie ("Unable to open pflog: " . $! . '.');

    logme ("pflog opened successfully.");

    select (INFILE);
    $|=1;
}

#----------------------------------------------------------------
# section 10: open ~/pflog-current for writing

sub opentarget {

    logme ("Trying to open pflog-current ...");

    open (OUTFILE, ">> /home/scooter/pflog-current") or 
         loganddie ("Unable to open pflog-current: " . $! . '.');

    logme ("pflog-current opened successfully.");

    select (OUTFILE);
    $|=1;

    alarm $rdelay;
}

#----------------------------------------------------------------
# section 11: rotate ~/pflog-* archive

sub rotatetarget {

    close (OUTFILE);

    $d_t = `date "+%Y-%m-%d-%H-%M-%S"`; 

    unless (fork) {
        system ("mv /home/scooter/pflog-current /home/scooter/pflog-" . $d_t);
        system ("gzip -9 /home/scooter/pflog-" . $d_t);
        exit;
    }

    opentarget();
}

#----------------------------------------------------------------
# section  12: opens ~/pflog-pipe fifo pipe for writing 
#

sub openpipe {

    sysopen(LPIPE, "/home/scooter/pflog-pipe", O_NONBLOCK|O_RDWR)
                                  or die "Can't open pipe: $!\n";

    select (LPIPE);
    $|=1;
}

#----------------------------------------------------------------
# section  13: set signal handlers

$SIG{HUP}  = 'rotatelogs';
$SIG{INT}  = 'loganddie';
$SIG{QUIT} = 'loganddie';
$SIG{KILL} = 'loganddie';
$SIG{TERM} = 'loganddie';
$SIG{STOP} = 'loganddie';
$SIG{TSTP} = 'loganddie';
$SIG{PIPE} = 'IGNORE';
$SIG{ALRM} = 'rotatetarget';

#----------------------------------------------------------------
# section  14: open input and output files

opensource();
opentarget();
openpipe();

#----------------------------------------------------------------
# section  15: read ~/pflog, write it to ~/pflog-current, and 
#                                                    ~/pflog-pipe

for (;;) {

    while (<INFILE>) {

        if (length ($_) != 0) {

            $buf = $_;

            if (!(syswrite OUTFILE, $buf)) {

                close (OUTFILE);
                opentarget();
            }

            if (!(syswrite LPIPE, $buf)) {

               close (LPIPE);
               openpipe();
            }
        }

    }

    sleep 1;
    seek (INFILE, 0, 1);
}

Here is a short summary of what this script does:

Perl for System Administration

Related Reading

Perl for System Administration
Managing multi-platform environments with Perl
By David N. Blank-Edelman

Log in as the user scooter (or whatever username you used for the user receiving logs), copy the script, and save it as readpflog. Now we need to make readpflog executable, and make it owned by scooter and a member of the scooter group with these commands (you need to be logged in as scooter):

# chmod 0700 readpflog
# chown scooter readpflog
# chgrp scooter readpflog

The user and the group need to be created on the monitoring station, with minimal privileges.

Next, we need to create the /home/scooter/pflog-pipe fifo pipe:

$ mkfifo -m 0600 pflog-pipe

In the last step, we need to add the following line to /etc/newsyslog.conf:

/home/scooter/readpflog.log        600  3    250  *     ZB /home/scooter/pflogd.pid

(Note that newsyslog will only rotate readpflog.log files, not pflog-* files.)

You might want to set $rdelay to a lower value; say, 60 seconds. If everything is working fine, you should see a list of archives similar to this one:

-rw-r--r--  1 scooter  scooter   212558 Jul 30 23:50 pflog-2002-07-30-23-50-54.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:51 pflog-2002-07-30-23-51-14.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:51 pflog-2002-07-30-23-51-24.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:51 pflog-2002-07-30-23-52-03.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:52 pflog-2002-07-30-23-52-18.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:52 pflog-2002-07-30-23-52-33.gz
-rw-r--r--  1 scooter  scooter       46 Jul 30 23:52 pflog-2002-07-30-23-52-48.gz
-rw-r--r--  1 scooter  scooter     4510 Jul 30 23:52 pflog-2002-07-30-23-53-03.gz
prw-r--r--  1 scooter  scooter        0 Jul 30 23:52 pflog-pipe

Now change $rdelay to a higher value, and have fun. You will most probably want to write another script, or modify readpflog (you're free to make any changes you like within the scope of the terms of the XFree86 license) to write archives to tape, or other external storage devices.

As before, you can find the listing of the script from this issue in the OpenBSD Administrator Toolbox.

Until next time!

Jacek Artymiak started his adventure with computers in 1986 with Sinclair ZX Spectrum. He's been using various commercial and Open Source Unix systems since 1991. Today, Jacek runs devGuide.net, writes and teaches about Open Source software and security, and tries to make things happen.


Read more Securing Small Networks with OpenBSD columns.

Return to the BSD DevCenter.


Copyright © 2009 O'Reilly Media, Inc.