News flash: computers are expensive, film at 11.
Many people use a dedicated computer as a firewall and NAT device. It's quite true that a high-end 486 running an open source UNIX-like operating system is perfectly capable of handling network connectivity in a SOHO environment, but computers have costs beyond the base hardware. These systems are bulky, and old hard drives frequently stagger on the verge of death. Plus, if you're in an enterprise environment, your manager does not want to hear statements like, "You know that desktop that was so abominably slow that you refused to let the receptionist endure it any longer? I put it back into service as the corporate firewall." Of course, you can always use your IT budget to purchase a nice new machine with a full warranty, but we all have things we'd rather buy than a computer that will sit 99% idle. All of this adds up to "expensive."
It's enough to make even the most rabid open source advocate consider purchasing an inexpensive consumer firewall appliance, losing a bit of flexibility in exchange for low-power silence.
Or, you can build a small firewall box yourself. While there are several alternatives for small, quiet systems, my favorites are the Soekris models. Soekris boxes are roughly the size of a paperback book, fanless, low-power, and designed specifically for open source operating systems. They're sold either as a plain board or mounted in a case.
For this article, I used the Soekris NET4801 -- a high-end miniature computer with a 266MHz processor and 128MB RAM, case-mounted and with a DC power supply, all purchased direct from Soekris.
While the NET4801 can use a laptop hard drive, that's a bit much for a basic network gateway. I chose to use a compact flash card for the operating system, as firewalls generally don't have to write to the local hard drive.
To install OpenBSD on a compact flash card, you need a CF drive and an OpenBSD system with a CF reader. The good news is that the OpenBSD system doesn't have to be a server-grade system; it's perfectly acceptable to take that secretary's obsolete desktop and use it as your bootstrap platform. I used a very recent OpenBSD-current on my bootstrap station. Using this method installs the same version of OpenBSD on the Soekris as on the workstation, so your choice of OpenBSD version is important. (You can install other releases on the Soekris, but that's beyond the scope of this article.)
Take the cover off of your Soekris, insert the CF card, and then hook a null modem
cable to the console port and a serial port on your bootstrap station. On your
bootstrap station, open up a
tip(1) window at 19200 baud.
# tip -19200 tty00 connected
Plug in the Soekris, and you'll see the following in your
comBIOS ver. 1.20 20030721 Copyright (C) 2000-2003 Soekris Engineering. net4801 0128 Mbyte Memory CPU Geode 266 Mhz Pri Mas Hitachi XXM2.3.0 LBA 993-16-63 500 Mbyte PXE-M00: BootManage UNDI, PXE-2.0 (build 082)
The LBA space gives the disk geometry that the Soekris sees. The Soekris BIOS believes this CF card has 993 cylinders, 16 tracks per cylinder, and 63 sectors per track. Take note of those numbers; you'll probably need them. These numbers are for my 512MB flash card, which turned out to be vastly too large. My complete install, with applications, used about 20MB, or roughly 4% of that disk. A 32MB or 64MB card would have been more than sufficient and far less expensive.
Now get Chris Cappuccio's flashdist.
flashdist is a set
of tools to install a stripped-down OpenBSD on an attached hard drive device.
In addition to a shell script that does most of the work, it includes some
/etc scripts designed for embedded systems, kernel
configurations tuned for Soekris hardware, and a text file listing every
component to install to provide basic OpenBSD networking features.
I started by building a Soekris kernel using the NET4801 configuration in
# cp NET4801 /sys/arch/i386/conf # config NET4801 # cd ../compile/NET4801/ # make depend && make
Once this finishes churning, copy the bsd file somewhere safe. I recommend renaming it to something like NET4801-bsd and moving the copy to your flashdist directory, just so you don't get confused and install it on your bootstrap station. (The Soekris kernel configurations don't bother with unnecessary hardware such as PC keyboards or video cards, so your bootstrap station would probably behave badly with that kernel.)
Now remove the flash card from the Soekris and put it in the bootstrap station's CF reader. Take note of which device the CF card shows up as, and then run flashdist.sh from the directory in which you extracted it.
# ./flashdist.sh -d sd0 flashsmall.txt ./NET4801-bsd /
-d flag to tell
flashdist which local
device to install onto. In this example, the CF card is
so we use
We also need a list of files to install on the system, provided in
as flashsmall.txt. The third argument is the name of the kernel
file to be installed on the disk, and finally we have the directory path from
flashdist should copy the files. In this case, we're copying the system
files from the installed OpenBSD system. You could extract the distribution
files elsewhere and use that path here, to install a different version of
OpenBSD than is on your bootstrap station.
The flashdist.sh script will walk through an install, making its best
guess at the disk geometry and prompting you for items such as the IP address
of a logging server and a root password for the complete system. Once complete, remove the CF card from your workstation, insert the CF in the Soekris, and power the
Soekris back on. You should see a standard OpenBSD bootup in your
tip(1) window, ending with a command prompt.
Congratulations! You have a working embedded OpenBSD system. It doesn't do much yet, but at least it boots. Now to make it useful.
You can set several variables in flashdist.sh to make the script stop asking you questions. You will probably be repeatedly re-installing on the CF card, so making the install as silent as possible is a good thing. Once you know the install works the way you want, you can even comment out the "hit enter to continue" sections and make the script run perfectly silently.
sectorstrack variables come straight from the Soekris BIOS screen.
(While several other disk variables appear in the script, flashdist.sh can calculate them on the fly.) You can also define variables for the syslog server IP address, or a password in either encrypted
or plain text format. The
etccopyfiles variable lists files
to copy from the flashdist directory to the target disk's /etc
directory. We'll make copious use of this later.
Also edit the rc file included in
flashdist. Unlike the standard BSD-style
rc system with lots of shell tricks and nifty variables, this rc script is just
a basic shell script of commands to be run upon boot. There is no
rc.conf or equivalent. The most obvious things to change are the
ifconfig statements, to assign the appropriate IP addresses to each
interface. Add any local startup scripts you have to this file.
Run flashdist.sh again once you have set all of the variables and configured the rc file, and confirm that the bare OpenBSD install still works. This is a fairly complicated process, and I recommend you reinstall until you have a customized installation process that works perfectly every time.
flashdist includes a very minimal OpenBSD install, you can probably
strip it down further. Edit the flashsmall.txt file to remove
things you don't need. This particular system will be a firewall, so it
doesn't need /usr/sbin/dhcpd, /sbin/dhclient, or
/sbin/dhclient-script. There is no wireless card, so
wicontrol(8) can go away. Do you need
tcpdump? If it's not needed in your
environment, remove it. Once you've gutted flashsmall.txt,
reinstall again to be sure the system still works; if you trim too
enthusiastically and without sufficient comprehension, you can easily create a
non-working system. Be sure your base system works before adding programs!
This particular system needs two additional programs, Snort with MySQL and
snmpd. The basic process is to build the
program on the bootstrap server, identify the files absolutely necessary for
the program to run, and add them to flashsmall.txt. Let's start
Build and install Snort with the appropriate flavors on the bootstrap server, then get it running on your bootstrap system exactly as you want it to be run on your firewall. This will tell you which configuration files Snort requires. In my case, I installed the configuration under /etc/snort.
Add the list of required configuration files to flashsmall.txt. While this will handle the instructions the binary needs, what about the associated programs and shared libraries the program binary needs?
The port places a list of all its installed files in the
+CONTENTS file under /var/db/pkg. This file also
contains port instructions and comments, all of which are prefaced with the "at"
@). Get rid of those first.
# grep -v '@' /var/db/pkg/snort-2.0.0p1-mysql-flexresp/+CONTENTS > \ $HOME/stripped-snort-contents
This creates a list of all files installed by the port. We won't be reading
man pages on the firewall, so get rid of them. We won't install sample rules,
so zap them. At the end, only one program file remains:
Even though the Snort port didn't install additional binaries, that doesn't
mean that the compiled Snort binary doesn't expect a certain shared library
infrastructure to be available. If you ask nicely with
the program binary will tell you which libraries it expects.
# ldd /usr/local/bin/snort /usr/local/bin/snort: Start End Type Ref Name 00000000 00000000 exe 1 /usr/local/bin/snort 001d1000 201d5000 rlib 1 /usr/local/lib/libnet.so.0.0 088ca000 288d0000 rlib 2 /usr/lib/libz.so.2.0 0c47a000 2c487000 rlib 1 /usr/lib/libpcap.so.2.0 01a30000 21a37000 rlib 2 /usr/lib/libm.so.1.0 09fdf000 29fe8000 rlib 1 /usr/local/lib/libmysqlclient.so.10.0 055d5000 2560e000 rlib 1 /usr/lib/libc.so.30.1 08f9a000 08f9a000 rtld 1 /usr/libexec/ld.so
This really isn't surprising. Snort requires four standard libraries from
/usr/lib, all of which are already appear in
flashsmall.txt. We have to add in the shared libraries from
/usr/local/lib, in this case
libnet.so.0.0. Add these files to
By default, OpenBSD only checks for shared libraries under
/usr/lib. You must tell the Soekris'
check for additional shared libraries under /usr/local/lib. Add
the following lines to the
flashdist rc file. I put them just
ldconfig ldconfig -m /usr/local/lib
With these additions to flashsmall.txt, re-run the installation and boot the Soekris.
In this particular case, I found that
snort would not run. It
complained about not being able to find
ldconfig -r showed that the shared library was
After losing several clenched fists' worth of hair, I learned about
libtool. By design, shared libraries are not supposed to call
other shared libraries.
libtool allows a shared library to break
this restriction. The MySQL libraries use
libtool to glue them
together. We have the following files under
libmysqlclient.so.10.0 libmysqlclient.a libmysqlclient.la
.so.10.0 is a standard binary shared library. The
.la file is a text file that configures
support for this library. Buried in this file we find the following:
# The name of the static archive. old_library='libmysqlclient.a' # Libraries that this one depends upon. dependency_libs=' -lz -lm'
So we need the libmysqlclient.a file. This library also
depends on two others,
libm. Add these to
flashsmall.txt, if they're not there. We also need the
libtool binary and whatever libraries that requires. Some work
ldd(1) gives the following extra files:
./usr/local/bin/libtool ./usr/local/lib/libltdl.a ./usr/local/lib/libltdl.la ./usr/local/lib/libltdl.so.1.2
/usr/local/lib/mysql to the
paths in the Soekris' /etc/rc file.
ldconfig ldconfig -m /usr/local/lib ldconfig -m /usr/local/lib/mysql
Finally, add the
snort startup to the very tail end of
echo snort: starting daemon... /usr/local/bin/snort -c /etc/snort/snort.conf -Dz &
After all of this, following the same process for
You will almost certainly have to do some troubleshooting on every application you install. I've found applications with files that look important but are irrelevant, while the developers have apparently disguised vital files as Frozen Bubble levels.
After all this, however, you will have a small silent machine that takes up a fraction of the power of a standard PC, with sufficient processing power for small network tasks. You can roll out hundreds of small systems while only having to back up the bootstrap station. By alternating flash cards, you can easily revert when an upgrade goes bad. These small systems make it possible to deploy complicated systems with a minimum of fuss.
Once you have hundreds of homemade embedded boxes, though, shuffling CF cards turns into an annoyance. Wouldn't it be easier to just skip the flash card and have the Soekris boxes create their own minds? We'll do that next time.
Michael W. Lucas
Read more Big Scary Daemons columns.
Return to the BSD DevCenter.
Copyright © 2009 O'Reilly Media, Inc.