Homemade Embedded BSD Systems
Pages: 1, 2
You can set several variables in flashdist.sh to make the script stop asking you questions. You will probably be repeatedly re-installing on the CF card, so making the install as silent as possible is a good thing. Once you know the install works the way you want, you can even comment out the "hit enter to continue" sections and make the script run perfectly silently.
sectorstrack variables come straight from the Soekris BIOS screen.
(While several other disk variables appear in the script, flashdist.sh can calculate them on the fly.) You can also define variables for the syslog server IP address, or a password in either encrypted
or plain text format. The
etccopyfiles variable lists files
to copy from the flashdist directory to the target disk's /etc
directory. We'll make copious use of this later.
Also edit the rc file included in
flashdist. Unlike the standard BSD-style
rc system with lots of shell tricks and nifty variables, this rc script is just
a basic shell script of commands to be run upon boot. There is no
rc.conf or equivalent. The most obvious things to change are the
ifconfig statements, to assign the appropriate IP addresses to each
interface. Add any local startup scripts you have to this file.
Run flashdist.sh again once you have set all of the variables and configured the rc file, and confirm that the bare OpenBSD install still works. This is a fairly complicated process, and I recommend you reinstall until you have a customized installation process that works perfectly every time.
Customizing the Installation
flashdist includes a very minimal OpenBSD install, you can probably
strip it down further. Edit the flashsmall.txt file to remove
things you don't need. This particular system will be a firewall, so it
doesn't need /usr/sbin/dhcpd, /sbin/dhclient, or
/sbin/dhclient-script. There is no wireless card, so
wicontrol(8) can go away. Do you need
tcpdump? If it's not needed in your
environment, remove it. Once you've gutted flashsmall.txt,
reinstall again to be sure the system still works; if you trim too
enthusiastically and without sufficient comprehension, you can easily create a
non-working system. Be sure your base system works before adding programs!
This particular system needs two additional programs, Snort with MySQL and
snmpd. The basic process is to build the
program on the bootstrap server, identify the files absolutely necessary for
the program to run, and add them to flashsmall.txt. Let's start
Build and install Snort with the appropriate flavors on the bootstrap server, then get it running on your bootstrap system exactly as you want it to be run on your firewall. This will tell you which configuration files Snort requires. In my case, I installed the configuration under /etc/snort.
Add the list of required configuration files to flashsmall.txt. While this will handle the instructions the binary needs, what about the associated programs and shared libraries the program binary needs?
The port places a list of all its installed files in the
+CONTENTS file under /var/db/pkg. This file also
contains port instructions and comments, all of which are prefaced with the "at"
@). Get rid of those first.
# grep -v '@' /var/db/pkg/snort-2.0.0p1-mysql-flexresp/+CONTENTS > \ $HOME/stripped-snort-contents
This creates a list of all files installed by the port. We won't be reading
man pages on the firewall, so get rid of them. We won't install sample rules,
so zap them. At the end, only one program file remains:
Even though the Snort port didn't install additional binaries, that doesn't
mean that the compiled Snort binary doesn't expect a certain shared library
infrastructure to be available. If you ask nicely with
the program binary will tell you which libraries it expects.
# ldd /usr/local/bin/snort /usr/local/bin/snort: Start End Type Ref Name 00000000 00000000 exe 1 /usr/local/bin/snort 001d1000 201d5000 rlib 1 /usr/local/lib/libnet.so.0.0 088ca000 288d0000 rlib 2 /usr/lib/libz.so.2.0 0c47a000 2c487000 rlib 1 /usr/lib/libpcap.so.2.0 01a30000 21a37000 rlib 2 /usr/lib/libm.so.1.0 09fdf000 29fe8000 rlib 1 /usr/local/lib/libmysqlclient.so.10.0 055d5000 2560e000 rlib 1 /usr/lib/libc.so.30.1 08f9a000 08f9a000 rtld 1 /usr/libexec/ld.so
This really isn't surprising. Snort requires four standard libraries from
/usr/lib, all of which are already appear in
flashsmall.txt. We have to add in the shared libraries from
/usr/local/lib, in this case
libnet.so.0.0. Add these files to
By default, OpenBSD only checks for shared libraries under
/usr/lib. You must tell the Soekris'
check for additional shared libraries under /usr/local/lib. Add
the following lines to the
flashdist rc file. I put them just
ldconfig ldconfig -m /usr/local/lib
With these additions to flashsmall.txt, re-run the installation and boot the Soekris.
In this particular case, I found that
snort would not run. It
complained about not being able to find
ldconfig -r showed that the shared library was
After losing several clenched fists' worth of hair, I learned about
libtool. By design, shared libraries are not supposed to call
other shared libraries.
libtool allows a shared library to break
this restriction. The MySQL libraries use
libtool to glue them
together. We have the following files under
libmysqlclient.so.10.0 libmysqlclient.a libmysqlclient.la
.so.10.0 is a standard binary shared library. The
.la file is a text file that configures
support for this library. Buried in this file we find the following:
# The name of the static archive. old_library='libmysqlclient.a' # Libraries that this one depends upon. dependency_libs=' -lz -lm'
So we need the libmysqlclient.a file. This library also
depends on two others,
libm. Add these to
flashsmall.txt, if they're not there. We also need the
libtool binary and whatever libraries that requires. Some work
ldd(1) gives the following extra files:
./usr/local/bin/libtool ./usr/local/lib/libltdl.a ./usr/local/lib/libltdl.la ./usr/local/lib/libltdl.so.1.2
/usr/local/lib/mysql to the
paths in the Soekris' /etc/rc file.
ldconfig ldconfig -m /usr/local/lib ldconfig -m /usr/local/lib/mysql
Finally, add the
snort startup to the very tail end of
echo snort: starting daemon... /usr/local/bin/snort -c /etc/snort/snort.conf -Dz &
After all of this, following the same process for
You will almost certainly have to do some troubleshooting on every application you install. I've found applications with files that look important but are irrelevant, while the developers have apparently disguised vital files as Frozen Bubble levels.
After all this, however, you will have a small silent machine that takes up a fraction of the power of a standard PC, with sufficient processing power for small network tasks. You can roll out hundreds of small systems while only having to back up the bootstrap station. By alternating flash cards, you can easily revert when an upgrade goes bad. These small systems make it possible to deploy complicated systems with a minimum of fuss.
Once you have hundreds of homemade embedded boxes, though, shuffling CF cards turns into an annoyance. Wouldn't it be easier to just skip the flash card and have the Soekris boxes create their own minds? We'll do that next time.
Read more Big Scary Daemons columns.
Return to the BSD DevCenter.