oreilly.comSafari Books Online.Conferences.


OpenBSD PF Developer Interview

by Federico Biancuzzi

OpenBSD's PF packet filter has grown in power and appeal since its introduction in OpenBSD 3.0. With the imminent release of OpenBSD 3.5, Federico Biancuzzi interviewed several leading OpenBSD developers for their thoughts on PF and new features.

This is Part 1 of a two-part series. BSD PF Developers Interview, part two is also online.

Federico Biancuzzi: Can you give us a short introduction about yourself and your role as open source developer?

Daniel Hartmeier: I'm 29 years old and live near Zug in Switzerland. I dropped out of university after the first year and am working as programmer since. Unmarried, no kids, two cats.

It was around November 1999 when I installed OpenBSD (2.5 by then) for the first time. I was working at a small company and we were looking for an OS to use as an Internet gateway. I didn't have much prior experience with Unix, but after a while, that gateway handled most network services we needed. Since then, I run OpenBSD on all servers and desktops I use.

Henning Brauer: Well, I am 25, running an ISP here in Hamburg/DE, mostly doing infrastructure, server- and backend-stuff, and, of course, and unfortunately, some administrivials.

I've been an OpenBSD developer since sometime early in 2002, doing a lot of stuff in pf (biggest single thing was the altq merge I lead and did big parts of myself); Apache maintenance in our tree, where I wrote the chroot extensions as well; various stuff in the network area from NIC drivers to ip layer, and so on, as well as various userland stuff.

Mike Frantzen: I live outside Washington D.C. where I enjoy the biking and hiking trails. I'm an engineer at NFR Security working on its intrusion detection system, where I tell my boss that I browse the web all day long. I hack on OpenBSD in my copious lack of spare time.

Cedric Berger: I'm 34 years old and live in the French-speaking part of Switzerland, near the city of Neuchatel. Five years ago, I went to California with my wife in the hopes of improving my English skills, and after nine months there I started to work for Wireless Networks, Inc., a startup company that currently provides wireless B2B services for the banking industry.

There we developed a little wireless router, and we decided to use OpenBSD TCP/IP stack in it because it was, at that time, the only free OS that had decent IPSec support. So, besides doing hardware design, network/database design, and everything else you'd have to do in a startup, I extracted part of OpenBSD kernel and ported it to a real-time kernel on an ARM processor. This is how I got involved with the Unix world.

O'Reilly Open Source Convention.

Two years ago I came back to Switzerland, when my wife was expecting. I still work for Wireless Networks as a consultant, and when I'm not hacking or babysitting, I like hiking and skying, like the average Swiss person. :)

Ryan McBride: When I wear a suit, I'm an information security consultant; nowadays, I'm just an open source software addict trying to get my daily fix. I currently live in the woods near a small town north of Vancouver, Canada, so when I'm not working on OpenBSD things I'm likely outside chasing deer from my garden or chopping firewood.

Can Erkin Acar: I am 32 years old and I live in Ankara, Turkey with my wife Zeynep. I have (finally) completed my PhD work last June and got married in July. I have been the network administrator of the department during my graduate studies, and now I am working for a network security company.

Federico: How did you join OpenBSD?

CB: I've used PF since 3.0 in an environment where I need to filter thousands of IP addresses individually, and that configuration was not handled very efficiently with early version of PF. So, two years ago I started to think about the table extension and wrote a working prototype of the "table" feature in December 2002. I submitted it to the PF mailing list, most people liked it, and I got invited to join the OpenBSD developer team by Daniel and Theo.

HB: Well, I was using OpenBSD for quite some time at work, and at a certain point the time had come to get more involved. In my case that is strongly related to pf; I apparently was the first one who tried to use pf in a major setup after 3.0 was released. It didn't quite work out, so I had a longish debugging session with Daniel. Turned out I had triggered a bug in an error path in pf, but it took us weeks to find out. After posting my results to misc@ I got this mail from Theo...

MF: Dug Song tricked me! Bad Dug. I wrote a firewall called mongoose for fun during a break from university. A year later OpenBSD dropped IPF in a copyright dispute and Dug invited me up to the MIT hackathon. We rightly chose to go with Daniel's PF for a replacement firewall which was more complete than mongoose and mine had a bunch of portability cruft to let it work under both OpenBSD and Solaris (hey, I like sparc gear).

RM: I've been involved with OpenBSD as a user since 2.3 or so; As far as development... When PF was first added to the tree, I was not working full-time and had been playing around with IPv6. I decided that PF should support IPv6, and the in keeping with the "shut up and hack" motto of OpenBSD development, I sat down and wrote it.

CEA: While searching for an operating system that would replace our Netware servers at the department, I installed OpenBSD 2.7 and instantly felt at home. I was thrilled by pf and subsequent pace of development and started using it immediately. Felt the need and wrote pflogd, and it was imported in to the tree. Also wrote pftop, a state visualization tool, and contributed some minor fixes and improvements to pf. I was invited to the last hackathon by Henning and became a developer.

I am mostly working on userland tools these days, especially privilege separation. There is too much to explore/learn and hack and very little time.

DH: In short, there was a dispute over IPFilter's license (we assumed it had a BSD license, Darren Reed said he never granted the right to distribute modified versions, which is what OpenBSD wanted), and Theo removed IPFilter from OpenBSD after 2.9 was released. I had some spare time and wanted to give it a try. I think it came as a surprise to everyone involved that pf was working so soon and got imported into CVS. Looking back, I'm grateful for the license argument, as I doubt there would have been enough motivation to replace and, eventually, surpass the existing code otherwise.

Federico: What PF parts have you worked on from 3.0 to 3.4?

HB: All.

For a lot of stuff in pf, there is a leading developer, the one who had the idea, or who started working on what was obvious. The result however is in almost all cases a joint effort.

That said, the biggest subproject I did in pf was the altq merge — again, a joint effort, without kenjiro laying the grounds from altq's side, theo's visions and quite some help from daniel this would not have worked. At the last hackathon I did the tagging stuff, which is very powerful. Over the time I did a [lot] of work in the parser, bringing more flexibility to the language, and learned to love yacc while doing so. And much more, but I don't remember everything offhand ... it's been some time and a lot of changes.

Related Reading

BSD Hacks
100 Industrial Tip & Tools
By Dru Lavigne

MF: The TCP state engine (shamelessly based on Guido van Roij's). The scrubber/normalizer which fits right in with the IDS I work on professionally (err, all the web browsing I do at work). The passive fingerprinting. Making fun of Henning and how little beer he can drink.

RM: After the IPv6 work, I added support for multiple translation or routing target addresses, which allows PF to do various types of load balancing of connections; it makes possible things such as using a PF firewall with two different ISP's, or balancing incoming http traffic against multiple web servers. But besides adding new features, I've done a fair bit of work cleaning up and simplifying the internals. I always feel best about my commits which result in a net reduction in the pf codebase.

CEA: I have written the pflogd pre 3.0 and improved the original (binary) log format of pf logs. What started as a project to safely parse pflogs for generating ASCII logs resulted in security extensions to bpf (berkeley packet filter which is used for capturing packets from the net and reading the pf logs) and privilege separation of pflogd, and tcpdump.

Federico: The 3.4 release page shows five new features. How does each one work?

  1. Packet tagging (filtering on tags added by a bridge based on MAC address).

    HB: Well, packet tagging allows you — as the name says — to add a "tag" to a packet, and read it out later on. Basically, that's it. It is so simple, and that is what makes it so powerful. You can use tags to express trust relations between interfaces (tag int1 on interface int1, and allow all packets with this tag out on int2 unconditionally), you can use tags to split classification and policy (tag packets so you end up with several "groups", where each group is one tag, and do the pass/block decision based on tags only), and you can even tag from outside pf — that's what i did for the bridge filters, we can tag packets from there as well, based on mac address for example.

  2. Stateful TCP normalization (prevent uptime calculation and NATdetection)

    MF: Stateful TCP normalization is a set of techniques to remove or resolve ambiguities in network traffic. One of the techniques most important to the average user is TCP timestamp modulation. Most operating systems with high performance networking include a timestamp in every TCP packet.

    Since that timer starts ticking when the machine was booted, a server (or anyone in between) can look at a packet and know the machine's uptime. An attacker could look at a machine's responses to know it hasn't been rebooting since the last patch came out so it is probably still vulnerable. Alternately a stingy internet service provider that charges extra for home networks can look at all of the timestamps coming from a link and count the number of NATted machines by the number of unique timestamps. The PF firewall can scramble both uptime calculation and NAT detection by modulating the timestamps with a random number. There are a variety of other normalization techniques done and others still in development.

  3. Passive OS detection (filter or redirect connections based on source OS).

    MF: Passive operating system fingerprinting has been suprisingly controversial feature. The underlying premise is that various operating systems' TCP stacks have evolved and diverged in different ways over the years. The firewall can look at packets and determine which operating system they came from by looking at those differences. Not only can it differentiate between Linux and Windows, but it can tell between Linux 2.2 and 2.4; it can even determine if you're using Opera. The integration into the firewall allows the administrator to filter or redirect connections based on the operating system of the client. Your enterprise has phased out Windows 98? Redirect them to a web site telling people to upgrade to a newer version (or a free Unix *hint* *hint*). Hate SCO? Redirect all SCO Unixware/OpenServer connections to a web page rant. Find email worms annoying? Block mail that came directly from Windows machines instead of going through a UNIX mail server. Getting slashdotted? Give Unix web browsers a larger share of the bandwidth than non-Unix users. Passive OS fingerprinting in PF has been my gift to BoFHs everywhere ;-)

  4. SYN proxy (protect servers against SYN flood attacks).

    DH: When a TCP connection is established, the so-called TCP handshake takes place. The source of the connection sends a SYN packet, the destination replies with a SYN+ACK, and the source replies with an ACK. After that, the TCP connection is established, and both peers can send data.

    A malicious party can cause a denial of service (DoS) attack by sending large amounts of TCP SYN packets to a server, while picking random fake source addresses (spoofing). The server will send SYN+ACK replies to the random source addresses and wait for the handshake to get completed by an ACK. But the recipients of the SYN+ACKs have never sent the SYN which initiated the handshake, so they will either reply with an RST or not reply at all. In both cases, resources are allocated on the server. Though this is only temporary, the server can suffer to varying degrees of the resource starvation caused by a significant flood of SYNs. Not all operating systems deal with this optimally, some may even become completely unresponsive.

    pf's synproxy sits in between the vulnerable server and potential attackers (usually in form of a border firewall). Instead of forwarding TCP handshake packets as they are seen (when they are valid and allowed), the synproxy intercepts the SYN packet and first completes the TCP handshake itself with the source peer. Afterwards, it replays the SYN with the destination peer, completing the handshake itself again. Once the handshake is completed with both peers, further packets are forwarded as usual.

    This has little impact on legitimate connections, but in case of the attack described above, the first handshake will never complete (as the true sender of the spoofed packet will not see the SYN+ACK, and the recipient of the SYN+ACK does not expect a connection). Before the first handshake is completed, no packet reaches the protected server. The attack can no longer allocate resources on the server. Instead, the resources are allocated on the firewall, which is built with this case in mind and can withstand such attacks more gracefully. In short, synproxy takes the burden of a SYN flood attack off the real server.

    The way synproxy is implemented in pf causes no additional resource allocation, when filtering statefully. A state entry (created on the initial SYN packet) already contains sequence numbers modulators (used to randomize initial sequence numbers, another feature of pf), which is all the synproxy needs. No additional connection tracking or SYN cookies are needed. And all existing features dealing with state table size (adaptive timeouts, limiting state entries created from specific rules, etc.) can be used to address attempts to starve resources on the firewall.

  5. Adaptive state timeouts (prevent state table overflows under attack)

    HB: That is easy — we can scale the state timeouts down based on the number of current states. It helps fighting state table exhaustion.

    When your state table is completely full (i.e., you hit the states limit), no new connections are possible. Thus, you really want to prevent this. Now, when you don't have many states in the table, you might want to run with the "normal" timeouts (set optimization normal). The closer you get to the state table limit, the more agressive you want to be about timing out old states to prevent the said state stable exhaustion. This is exactly what adaptive state timeouts are for. One you hit the adaptive.start number of states, pf starts to scale down timeouts — the closer you get to adaptive.end, the more.

Pages: 1, 2

Next Pagearrow

Sponsored by: